View Issue Details

IDProjectCategoryView StatusLast Update
0005726CentOS-6libvirtpublic2015-10-15 17:57
Reporterg-pulido@gmx.net 
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionsuspended 
Platformx86_64OSCentOS 6.2OS Version6.2
Product Version6.2 
Target VersionFixed in Version 
Summary0005726: Suspended VMs have wrong time on wake up
DescriptionIf a Virtual Machine running under KVM on a CentOS 6.2 host is suspended, and then is powered on after a few hours, the time in the VM will not sync with the real time, it will continue from the moment it was suspended. The machine uses kvm-clock as time source (recommended according to docs I've found). Even using NTP will not help because the time shift is too big for NTP to fix it.
Steps To ReproduceCreate a KVM VM
Verify it has the correct time.
Suspend the VM and wait some time (an hour should be enough)
Power on the VM
Verify it shows the time about an hour back.
Additional InformationThis happened with several guest OSs: CentOS 6 x86 and x86_64, FreeBSD 9.0, Ubuntu 12.04 and OpenBSD 5. Same behavior on all of them.
TagsNo tags attached.

Activities

toracat

toracat

2012-05-12 19:58

manager   ~0015073

I suspect what you described is the default behavior. When you suspend a KVM guest and then restore it, all is restored to the same state as before including the time. So, you'd need to adjust it manually or reboot the guest.
g-pulido@gmx.net

g-pulido@gmx.net

2012-05-12 20:55

reporter   ~0015074

Thanks for your pointer, toracat. However, intentional or not, this is different from the behavior of (at least) VMware ESX, Hyper-V and VirtualBox, therefore I see it as a bug. In my experience, on those other platforms the clock in the guest is synchronized with the host at resume time.
toracat

toracat

2012-05-14 15:59

manager   ~0015079

We are probably not talking about the same thing ...

When you say, "suspend and power on", what exactly would you do? In my earlier note, I was referring to 'virsh save/restore'.

If I "pause" a KVM guest, the time gets adjusted as soon as it is "unpaused". This is equivalent to doing 'virsh suspend/resume'. I tested this with a few guests on 2 different KVM hosts after pausing 1 hr to 12 hrs. It always worked for me.

???
g-pulido@gmx.net

g-pulido@gmx.net

2012-05-14 16:22

reporter   ~0015080

Hi toracat

I think I didn't explain well, sorry for that. What I meant is exactly what you tested: pause a VM and then resume it. The way I test is exactly this:

1. I have some VMs running on my host.
2. I shutdown the host, so all the running VMs are saved.
3. The host stays powered off some hours.
4. I turn the host back on, so the VMs are resumed.
5. I check the time on the guests, they keep the same time they had when suspended (approximate, because obviously time starts running again when the VM is resumed).
6. If I turn another VM on, it gets the time fine.
7. If I reboot a VM, the time problem persists, but if I power it off and on again, the time is fixed.

I tested this with CentOS 6.2 x64. This is the only host I have, so I can't test with another one, but if hardware details are needed, please feel free to ask, I'll provide any info needed to solve this. I also found this link, that describes my exact issue: http://serverfault.com/questions/334698/how-to-keep-time-on-resumed-kvm-guest-with-libvirt

Thanks!
laroche

laroche

2012-05-15 20:19

reporter   ~0015084

Hello,

I can only confirm above reports:

With kernel 2.6.32-220.13.1.el6.x86_64 for kvm-server as well as
guest, a "virsh suspend/resume" sets the time to the current time
and ntpd keeps running for me.

With "virsh save/restore" the time is kept to the time of the "save",
so then "ntpd" is also getting into trouble.

Overall the clock source in the kernel is heavily patched and very dependent
on the underlying hardware details, but this looks more like a design decision
on when/if to sync the time to the current kvm-server time.

Try reporting this to Red Hat / upstream.

best regards,

Florian La Roche
g-pulido@gmx.net

g-pulido@gmx.net

2012-05-16 02:53

reporter   ~0015087

Reported on upstream's bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=821988

Thanks!
toracat

toracat

2012-05-16 16:47

manager   ~0015092

For the record, there is another person who confirmed the issue as seen in this mailing list thread:

http://lists.centos.org/pipermail/centos/2012-May/126015.html

( it's OT there :-( )

As a workaround (not a solution), one can add a line:

tinker panic 0

to the *top* of the /etc/ntp.conf file. A quote from a VMWare KB article [1]:

"The configuration directive tinker panic 0 instructs NTP not to give up if it sees a large jump in time. This is important for coping with large time drifts and also resuming virtual machines from their suspended state."

[1] http://kb.vmware.com/kb/1006427
toracat

toracat

2012-08-01 03:23

manager   ~0015567

@g-pulido

In the BZ 821988 you filed, someone referred to:

http://www.mail-archive.com/kvm@vger.kernel.org/msg67881.html

as the patch for the issue reported. And the patch appeared in the 6.3 GA kernel. Can you confirm that the problem was actually resolved in CentOS 6.3 ?
g-pulido@gmx.net

g-pulido@gmx.net

2012-08-02 17:29

reporter   ~0015571

Hi!

I just updated (yum update) my CentOS 6.3 KVM host and tested again. The behavior remains exactly the same, so it's not fixed yet.
toracat

toracat

2012-08-02 18:13

manager   ~0015572

That means that the patch referenced in https://bugzilla.redhat.com/show_bug.cgi?id=821988#c5 is not a fix.

You want to post your finding there?
toracat

toracat

2013-04-09 00:47

manager   ~0017175

To people who encountered the problem, does this still exist in CentOS 6.4?
centos.admin

centos.admin

2013-04-11 08:25

reporter   ~0017195

Tested on 6.4, problem still exists. Saved VM will have wrong date after Restore. Virtual reboot of VM does notfix time.
michaeljmuller

michaeljmuller

2013-05-21 22:16

reporter   ~0017487

I also see this problem on 6.4. Using VMWare Fusion. Suspend VM, and when I resume it's still got the old time...
toracat

toracat

2015-10-15 17:57

manager   ~0024609

Closing due to inactivity.

Issue History

Date Modified Username Field Change
2012-05-12 18:08 g-pulido@gmx.net New Issue
2012-05-12 19:58 toracat Note Added: 0015073
2012-05-12 20:55 g-pulido@gmx.net Note Added: 0015074
2012-05-14 15:59 toracat Note Added: 0015079
2012-05-14 16:22 g-pulido@gmx.net Note Added: 0015080
2012-05-15 20:19 laroche Note Added: 0015084
2012-05-16 02:53 g-pulido@gmx.net Note Added: 0015087
2012-05-16 16:47 toracat Note Added: 0015092
2012-05-16 16:47 toracat Status new => confirmed
2012-08-01 03:23 toracat Note Added: 0015567
2012-08-01 03:23 toracat Status confirmed => feedback
2012-08-02 17:29 g-pulido@gmx.net Note Added: 0015571
2012-08-02 17:29 g-pulido@gmx.net Status feedback => assigned
2012-08-02 18:13 toracat Note Added: 0015572
2013-04-09 00:47 toracat Note Added: 0017175
2013-04-11 08:25 centos.admin Note Added: 0017195
2013-05-21 22:16 michaeljmuller Note Added: 0017487
2015-10-15 17:57 toracat Note Added: 0024609
2015-10-15 17:57 toracat Status assigned => closed
2015-10-15 17:57 toracat Resolution open => suspended