Re: Issue with time going backwards on OSX

[Warner Losh]

On Tue, Sep 28, 2021 at 8:48 AM Joe Tanen <jtanen@fb.com> wrote:

Hello,

 

I posted this on qemu-discuss, but it doesn't seem like anybody else has seen this issue. I figured I'd try here to see if anybody had any thoughts.

 

I'm using qemu-system-aarch64 v5.2 on OSX 11.6. While reading cntvct_el0, I've seen the ticks count go backwards. I dug into the qemu source a bit, and this register is eventually backed by a call to clock_gettime(CLOCK_MONOTONIC), which should not go backwards. This code seems to be the same on master as the branch I'm on.

 

Before I started delving into debugging qemu, I wanted to verify that this wasn't an OS issue. I wrote a simple test that read CLOCK_MONOTONIC every 100ms, compared the values to make sure they always increased, and let it run overnight. Lo and behold, I saw time go backwards. I ran a concurrent test with CLOCK_MONOTONIC_RAW, and I did not see time go backwards. Successive reads of the CLOCK_MONOTONIC after it jumped backwards tracked the new time, so the jump wasn't a spurious error. When I ran this test on a Fedora 34 PC, I never saw time go backwards.

 

Someone suggested as a test disabling networking so the NTP daemon couldn't mess with the clock, but that's not something I can do IRL. In any case, my understanding is that CLOCK_MONOTONIC should never go backwards and that any adjustments are made by temporarily slowing down or speeding up the tick rate. I'm going to pursue what I believe to be the underlying issue separately, but my qemu questions are these:

 

- Has anybody else seen this or other timer registers go backwards under these or other conditions? I only see the behavior on OSX 11.6, but I'm a small sample size.

 

- Would it make sense to use CLOCK_MONOTONIC_RAW (if available) instead of CLOCK_MONOTONIC in qemu? My issue aside, CLOCK_MONOTONIC_RAW feels closer to giving the guest direct access to a hardware clock. e.g., if I wanted to write my own NTP time adjustment daemon, I'd prefer to have an unadjusted clock. The code already checks to see if CLOCK_MONOTONIC is available, and, if not, it reverts to a simple gettimeofday(). It was easy to add another check to init_get_clock() to add a case checking for the availability of CLOCK_MONOTONIC_RAW. I've tested this myself, but I wanted to see if there was upstream interest in this and/or if there were reasons to prefer the current implementation.

Yea, CLOCK_MONOTONIC going backwards violates POSIX's description of it not going backwards... Some operating systems with the not going backwards property provide CLOCK_MONOTONIC_RAW as an alias for CLOCK_MONOTONIC, so whatever you do, please make sure it works in this situation, or you'll break at least FreeBSD...

Warner

 

Thanks, and best regards,

Joe