December 31 2016 Leap Second

Leap Second's effect on PASSCAL RT-130 and Q330

Summary:

Q330:

Data time starting at 2017:001:00:00:00 is +1 second from truth for 1-2 minutes (seen in controlled tests) until a clock correction, followed by a gap of 2-40 seconds per channel depending on sample rate.

RT-130:

Data time starting at 2017:001:00:00:00 is +1 second from truth for 15min - 9:42hrs (seen in controlled tests) until a clock correction. Some data (~6secs in tests) around the time correction is mistimed, sometimes 11 years in the future, or missing.

Units that did not make the correction for 9:42hrs did not perform phase timing corrections during this period and could also have timing errors due to clock drift that can not be corrected by -1sec adjustment.

PASSCAL has developed software to adjust RT130 data for the December 31 2016 leap second. The new program for finding, viewing, and correcting Dec 2016 leapsecond mistimed RT-130 data recorded to CF media may be found here. A description of program parameters and usage can be found here, or by invoking rtleap.py -h.

Details:

A leap second was added on December 31, 2016 at 23:59:59. This has the effect of delaying the year rollover by one second. Let's say that a datalogger (DAS) is running with its oscillator keeping time, with the GPS off, and then the DAS turns the GPS on when it thinks it is Jan 1 2017 00:00:01. If the GPS could get a lock instantaneously, the time from the satellites would be Jan 1 00:00:00, and the DAS need to do a -1 second time correction. Details of how PASSCAL Q330 and RT130 dataloggers handled the leap second and what users may expect to see in their data are described below. Also note that the dataloggers on the PASSCAL benches during the leap second had no problems obtaining GPS locks. If a GPS in the field is having difficulty obtaining a lock on the satellites, the leap second correction could be delayed.

(click here to skip to RT130 Leap Second Correction details)

Q330 Leap Second Correction (1.145 firmware)

The Q330 dataloggers power the GPS antennnas and turn on the GPS routinely on the day rollover. It takes a moment for the GPS to get a lock but, for the units we had running on the bench at PASSCAL, within the first one to two minutes of the new year the units noted the time difference and made a time jump to correct. If the GPS happened to already be turned on when the leap second occurred, it's possible the time jump might be exactly -1 second. The GPS in our units had all been off for some time though so the jumps were not exactly -1 second (since some offset had built up while the GPS was off) but were all approximately -1 second. The negative time jump caused an ~1 second data overlap. However, this was immediately followed by a data gap, ranging from 2 to 21 seconds. The gap is noted in the Q330 LOG with the entry "New Digitizer Phase=Timemark Wait, Reason=Phase out of Range". For state of health channels, with slower sample rates, the gap could be longer, ranging up to 40 seconds. The figures below show the features of the Q330 behavior with regards to the leap second (click on images for larger views).

Figure 1: PQL plot of ~2 minutes of sine wave Q330 data channels centered on the year rollover with ~1 second overlap and 21 second gap.

Figure 2: PQL plot of ~2 hours of selected Q330 SOH channels, centered on the year rollover showing the up to 40 second gap.

Figure 3: For context, ~24 hour PQL plot of same selected Q330 SOH channels, centered on the year rollover, showing the additional GPS On cycle on the day rollover and the phase correction during that cycle.

Click here for an excerpt of the Q330 LOG from the beginning to the end of the GPS On cycle that occurred on the year rollover.

RT130 Leap Second Correction (3.4.3 firmware)

The RT130 dataloggers do an additional time set on the year rollover, similar to the time set done when the unit is powered up or rebooted. The units we had running on the bench at PASSCAL over the new year powered on their GPS (if it was off) at 00:00:00 or 00:00:01. Doing the time set on the year rollover is one thread in the 3.4.3 firmware. Correcting for leap seconds when they are detected is another thread in the 3.4.3 firmware. The exact timing of these two threads running together in this instance caused some variable results. Before describing these results, please note that we have already been in contact with RefTek, they explained the issue of the two software threads interacting and assured us that this been addressed and is no longer a issue in firmware version 3.7.3. We are starting our testing and verification procedures on version 3.7.3 and hope to begin updating units before the end of the year. With that said, Figure 4 below summarizes the different leap second results we saw among the 14 RT130 units running on the bench (click on the image for a larger view).

Figure 4: PQL plot of bench data (1 sec pulse once per minute) for channel 1 of four RT130 units from 2016 Dec 31 23:00:00 to 2017 Jan 1 11:00:00. The red arrows indicate when the leap second correction occurred and where there were gaps in the data plotted by PQL (see text).

Some units completed the correction at ~17:001:00:15:30, some at ~17:001:00:57:30, and some at ~17:001:09:42:59. Although the PQL plot in Figure 4 shows gaps in the data for part of the first hour of 2017 for some units, there are events starting at the beginning of the new year for all units in their raw data. When the leap second correction occurs, a new event starts (and ends at the next hour rollover). However, due to competition between the two threads running in the firmware, in some instances where the leap second was corrected within the first hour (first event) of the new year, some time stamps in the DT packets in the raw data became corrupted. We saw time stamps that jumped ahead nearly 11 years, and that is what caused the gaps in the PQL's plotting of the data. We have not seen corrupted time stamps in the data from units that were unable to make the leap second correction until 9:42, but it may happen in some instances. We don't know if the differences seen among the units on our bench represent all the permutations that could have resulted from these thread interactions.

Click here for an example of the corrupted DT packet time stamps.

Whether an RT130 made the leap second correction at 00:15 or 00:57 seems to have depended upon whether the the previous hour's GPS On interval had ended and the GPS was off when the year rolled over, or whether GPS happened to be still On. If the GPS was Off, then it was powered back On at 00:00:01 and completed the correction at ~00:15:30. However, if the GPS was still On when the year rollover occurred, it was extremely close to the end of the normal GPS On cycle, went ahead and ended that cycle on schedule, didn't have an extra On interval at the beginning of the year, and then took care of the leap second in the next regular On cycle starting at 00:40, completing the correction at ~00:57. Logpeek screen captures for these two instances are shown just below.

(Click an image for a larger view)

Figure 5: Logpeek screen capture for an RT130 where the GPS was Off when the leap second occurred, showing an extra GPS On cycle immediately after the year rollover with leap second correction at 001:00:15:33.

Figure 6: Logpeek screen capture for an RT130 where the GPS was On at the year rollover so there was not an extra GPS On cycle as is seen in Figure 5. This one did not catch the leap second when it occurred. Another RT130, also with GPS On at the year rollover, did realize there was a problem right at the end of that GPS cycle and gave the log message "001:00:00:01 EXTERNAL CLOCK ERROR - UNABLE TO CORRECT DRIFT". Both units made the leap second correction at 001:00:57, during the next regular GPS On cycle.

The reason some units did not make the leap second correction until ~09:42 is not known. But in these cases, there was a suspension of all Clock state of health messages in the log file for that entire period. Figure 7 below illustrates.

Figure 7: Logpeek screen capture for an RT130 where correction of the leap second was delayed ~9hr 43min. The four units on the PASSCAL bench that behaved this way, have clock log messages exactly the same. The clock was powered On at 001:00:00:00, then there are no other clock messages until clock lock is reported at 001:09:27:33 and the clock set at 001:09:42:59. The units in the lab were in a controlled temperature environment so there was very little drift over that nearly 10 hour time span. However, one unit that was running outside experienced a 9 degree C temperature drop over that interval and it had a 793usec phase error to additionally correct.

The last figure, below, shows just over 100 seconds of data around the time that the unit plotted in Figure 5 (9418) made the leap second correction. Also plotted, for comparison, is data from the unit in Figure 7 (955A) which didn't make the correction until later. For unit 9418, rt2ms reported a data overlap of -6.455 seconds and then a gap of 0.450 seconds. For unit 955A, when it made the correction at 09:43, rt2ms reported a similar overlap of -6.405 seconds with a gap of 0.400 seconds.

Figure 8: Closeup of data at the time unit 9418 (also in Figure 5) corrected for the leap second. The top two traces are unit 9418. The bottom trace, for comparison, is unit 955A (Figure 7) which made the correction later. Vertical lines have been drawn to show the alignment of the pulses before the correction, and the offset after the time reset on unit 9418.