Leap second bug in linux kernel

Sarath Pillai's picture

On 30th June Sheeri K. Cabral of Mozilla foundation posted a blog on their website. The blog post was regarding some issue with their mozilla servers who's cpu utilization went so high at around 17.00 UTC. The servers who's cpu utilization spiked were having MYsQl and Java programs running.

 

And they eventually ended up fixing the issue by setting the date once again by the below method

 

 

 

 

[root@myvm1 ~]# date -s "`date`"
Tue Sep 25 02:17:39 PDT 2012

Now UTC is time used by many servers and data centers because that remains consistent across different countries. UTC is used mainly to coordinate with others with one time standard. There is something called as Leap second which is added occasionally to keep UTC close to solar mean time. You can read more about Whats a Leap second from here Leap Second Wikipedia  

And the last time this second was inserted was on June 30, 2012.

One important fact to note here is that, Mysql and Java are not the culprits here but consider them as a victim.

The bug is that when a leap second is added the, kernel time is one second ahead of the system time(which means that system time lags one second behind the kernel time)

So in that case imagine a situation where an application asks a kernel function with a duration of less than one second, then the kernel imagines that the time is over of finished even before the counter is set. And the program keeps on asking again and again in an endless loop sometimes resulting in high CPU utilisation.

This problem is reported to have affected all kernel versions from 2.6.26 to 3.3.

The high CPU utilization resulted in power wastage, reported several data centers from around the world when the leap second was added.

John Stultz who is working on he kernels hrtimer code for years is trying to fix the issue.

Resetting the date as shown above previously without restarting the server is an easy and fast fix to the bug.

And Now there is a good news for linux users that they fixed this Leap second bug in the new 3.5 kernel release. Other than the Leap Second Bug fix, Hybrid graphics support is the noticable thing in the new kernel.

This problem is reported to have affected all kernel versions from 2.6.26 to 3.3.

Rate this article: 
Average: 3.4 (34 votes)

Comments

What i understood in this article, kernel time is one second ahead of system time.
We can synchronize the system time(Kernel time) using hwclock command.

# hwclock --hctosys ( set the system time from the hardware clock )

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.