NTP Sync Quality — Co-located VM Test

Does the timekeeping series change how chrony disciplines the clock on an idle guest, under a completely standard NTP setup (no PPS)? Four co-located VMs run patched/baseline kernels in periodic/tickless tick modes, all disciplined by chrony from Amazon Time Sync, and judged by an independent referee: the ENA PHC (/dev/ptp0), sampled at 1 Hz.

Test Matrix

Host	Instance	Tick Mode	Kernel	Socket
E	`i-0bbd4d706b525b2c6`	periodic (`nohz=off`, 1000Hz)	7.3.0-rc1-timekeeping+	host:1
F	`i-0a1209b28c3b15800`	periodic (`nohz=off`, 1000Hz)	7.3.0-rc1-baseline+	host:1
G	`i-051ba016b4648399a`	NO_HZ_IDLE	7.3.0-rc1-timekeeping+	host:0
H	`i-061779927cc7e5b35`	NO_HZ_IDLE	7.3.0-rc1-baseline+	host:0 (to be confirmed — replacement instance, see below)

Kernels

Baseline: Linux 7.3-rc1 (cee9395acd80).
Timekeeping: 7.3-rc1 plus the four timekeeping fixes under review: timekeeping: Allow tick_length changes to apply mid-tick, timekeeping: Bound idle sleep while a phase slew is in flight, timekeeping: Reinstate proportional correction of ntp_error, ntp: Recalculate skew_delta when the phase offset changes.
Both kernels additionally carry two identical test-instrumentation commits, so the referee measures the same quantity on both sides: timekeeping: Apply extrapolated ntp_error to clock snapshots (PHC bracket reads report the intended clock rather than the mult-quantised sawtooth) and a /sys/kernel/debug/tk_sample sampler exposing ntp_error, mult etc.
Identical .config (only CONFIG_LOCALVERSION differs): HZ=1000, NO_HZ_IDLE=y. Tick mode selected per host by nohz=off on the command line. Deployed by kexec; the grub default remains the stock Fedora kernel.

Setup

All 4 VMs are c8i.16xlarge co-located on the same physical droplet (7.233.99.247, DUB68), which is dual-socket: the two sockets (host:0 / host:1) have independent TSCs that differ by ~2.5 PPM. Patched-vs-baseline is therefore compared within a socket (shared TSC, shared NTP path); cross-socket frequency differences are a hardware artifact, not a kernel effect.
Fedora 44 Cloud, chrony 4.8. Stock configuration except server 169.254.169.123 prefer iburst (Amazon Time Sync) and log tracking measurements statistics. Polling is fully adaptive — the poll interval is a measured output, not a controlled input.
Referee: CLOCK_REALTIME − ENA PHC at 1 Hz (tightest of 5 bracketed reads via PTP_SYS_OFFSET_EXTENDED), independent of chrony's own beliefs. The PHC derives from the same AWS time infrastructure as the NTP service, without the network path.
Instances otherwise idle. Drift files and chrony logs cleared; chronyd started simultaneously on all four hosts at 2026-09-30 23:42:48 UTC.
Host H is a fresh instance (i-061779927cc7e5b35) launched into the slot vacated by the original H on the same droplet; its socket placement will be confirmed from the frequency signature in the data.

Why Co-located?

The earlier bare metal tests showed clear differences between patched and baseline kernels, but confounders (different crystals, different thermal environments, different network paths) made it hard to attribute differences purely to the kernel patches. By placing all 4 VMs on the same physical host, we eliminate:

Crystal variation — within a socket: each patched/baseline pair shares a socket and therefore the same TSC.
Thermal drift — same physical environment
Network path — same NTP route to the time source

Any remaining differences must come from the kernel's timekeeping code or the tick configuration.

PHC Referee

CLOCK_REALTIME minus the ENA PHC at 1 Hz per host — ground truth for how well each kernel's clock actually tracks, independent of chrony's self-assessment. Second panel: exponentially-weighted RMS (60-sample halflife) of the kernel's own ntp_error from the tk_sample sampler. Third: PHC read bracket width (measurement quality bound).

CDF of PHC Offset

Allan Deviation

Allan deviation measures clock stability at different averaging timescales. Lower is better. A clock that's well-disciplined by NTP should show decreasing ADEV at longer timescales (the servo corrects drift). The slope indicates the dominant noise type (white noise = −½, flicker = 0, random walk = +½).

Time Series

The "RMS Offset (chrony)" panel shows chrony's internally-tracked RMS which uses an exponentially-weighted history that predates our test window. The "RMS Offset (cumulative)" panel computes RMS directly from the offset samples collected during this test, giving a fairer comparison unaffected by each host's different convergence history.

Distribution (Box Plot)

CDF of Absolute Offset

Methodology

Two collectors run on each host:

chronyc -c tracking every 30 s → ntp_quality.csv (chrony's self-view)
phc_phase_log every 1 s → phc_phase.csv (PHC referee + adjtimex() readback + tk_sample kernel discipline state)

Scripts

ntp_quality_log.sh — chrony tracking collector (runs on each host)
phc_phase_log.c — 1 Hz PHC referee collector (runs on each host)
run-test.sh — post-kexec bring-up: re-enable the ENA PHC (the devlink enable_phc param is lost on kexec), ensure chrony log directive, start collectors
ntp_quality_graph.py — fetch data and generate graphs

Raw Data

ntp_quality.csv columns (from `chronyc -c tracking`)

Column	Description
offset_ns	Current offset from NTP source (instantaneous error)
freq_ppm	Frequency correction applied by chrony
rms_offset_ns	RMS of recent offsets (jitter measure)
skew_ppm	Rate of frequency change (stability measure)
root_delay_ns	Round-trip delay to stratum-1
root_disp_ns	Estimated maximum error from root

phc_phase.csv columns

Column	Description
phc_offset_ns	CLOCK_REALTIME − PHC, tightest of 5 bracketed reads
bracket_ns	Width of the winning bracket (quality bound)
adj_freq_scaled, adj_offset, adj_status	`adjtimex()` readback
snap_err_ns	Extrapolated ntp_error deducted by the snapshot correction
ntp_error, mult, ntp_err_mult, ntp_err_frac	Timekeeper discipline state from `tk_sample`

Data last fetched: —