Frontend Performance

CrUX vs Your Own RUM: Why the Two Numbers Disagree

This comparison belongs to RUM Beacons & Field Data Collection within Core Web Vitals & Measurement. Your dashboard says p75 LCP is 2.1s. Search Console, reading the Chrome UX Report, says 2.6s and marks the origin as needing improvement. Someone concludes the instrumentation is broken and the project loses a week.

Neither number is wrong. They describe different populations over different windows with different aggregation rules, and once you can name the four differences you can reconcile them in an afternoon.

Rapid Diagnosis: Which Difference Is Yours?

  • Compare browser mix first. If more than a quarter of your traffic is Safari or Firefox, browser coverage alone can explain a gap of several hundred milliseconds.
  • Check the window. CrUX is a rolling 28-day distribution. If you deployed a fix ten days ago, your 24-hour number and CrUX cannot agree yet by construction.
  • Check the unit. CrUX reports per origin and, for popular URLs, per URL group. If your dashboard averages across every route, you are comparing a weighted mixture against a differently weighted mixture.
  • Check delivery. Divide beacons received by page views. Loss is rarely uniform: the slowest sessions are the most likely to close before the flush, so loss makes your p75 optimistic.
Reconciling a self-reported p75 with CrUX Starting from 2.1 seconds, adding non-Chrome coverage, widening to the 28-day window, switching to origin-level aggregation and correcting delivery loss each add time until the number matches the 2.6 second CrUX figure. From 2.1s to 2.6s, one correction at a time your p75 2.10s +0.18s drop non-Chrome +0.15s 28-day window +0.09s origin-level +0.08s delivery loss CrUX 2.60s Reconciled means explained, not identical. A residual under about 5% is normal: sampling, opt-in population and rounding are not reproducible. A residual above 20% after all four corrections is a real instrumentation bug worth chasing.

Root Cause Analysis: The Four Structural Differences

1. Browser and opt-in coverage. CrUX contains Chrome users who have opted into usage statistics reporting, on public origins. Your beacon fires wherever the APIs exist. Safari does not report INP at all and reports LCP differently in places, so an iOS-heavy audience shifts your distribution relative to CrUX in ways that vary per metric.

2. The aggregation window. CrUX publishes a rolling 28-day distribution updated daily, and the monthly dataset is coarser still. Any dashboard reading a shorter window will diverge after every meaningful change, in the direction of the change, for up to four weeks.

3. The aggregation unit. CrUX is origin-level unless a URL has enough traffic for its own entry. Your table can slice per route. A site whose homepage is fast and whose search results are slow will report wildly different numbers depending on which routes are in scope and how they are weighted.

4. Delivery loss and its bias. Beacons that never arrive are disproportionately from sessions that ended abruptly — precisely the slow ones. A 15% loss concentrated in the tail can move a p75 by more than a hundred milliseconds, always making your number look better than reality. The delivery fixes are in sending Web Vitals with sendBeacon.

Step-by-Step Resolution: Build a Comparable View

1. Filter your data to a CrUX-shaped population.

sql
-- Chrome only, 28-day window, origin-level, cold navigations only.
SELECT approx_quantile(value, 0.75) AS p75_lcp, count(*) AS samples
FROM rum_events
WHERE name = 'LCP'
  AND browser = 'Chrome'
  AND nav_type = 'navigate'          -- exclude bfcache restores and reloads
  AND ts > now() - interval '28 days';
-- trade-off: this view is for reconciliation only. Keep your per-route,
-- all-browser view as the engineering dashboard — it is more useful precisely
-- because it is not CrUX-shaped.

Expected outcome: the gap typically closes by half to two-thirds in one step.

2. Weight by the same unit. If CrUX reports your origin, compare against your origin-wide number, not the average of per-route p75s — an average of percentiles is not a percentile. Compute the percentile over the pooled sample.

3. Estimate delivery loss and bound its effect. Take beacons received over page views from an independent counter. If you are at 88% delivery, recompute your p75 after discarding the fastest 12% of samples: that is the pessimistic bound, and the truth sits between the two figures.

sql
-- Pessimistic bound: assume every lost beacon was slower than everything you kept.
SELECT approx_quantile(value, 0.75 / 0.88) AS p75_pessimistic
FROM rum_events WHERE name = 'LCP' AND browser = 'Chrome';
-- trade-off: this assumes ALL loss is in the slow tail, which overshoots.
-- Use it as an upper bound to decide whether loss can explain the gap at all.

Expected outcome: a range, not a point — and usually enough to show the residual gap is small.

4. Only then look for a bug. If a residual above roughly 20% survives all four corrections, investigate the instrumentation itself: an observer registered after the metric already fired, a reportAllChanges configuration reporting interim values, or a sampling rule that quietly excludes a device class.

Verification

Re-run the comparison weekly for a month and track the residual rather than the absolute difference. A stable residual under 5% means your pipeline is sound and the two numbers are simply measuring different things. A residual that grows over time points at drift — usually a new route excluded from instrumentation, or a delivery regression on one platform.

Confirm your view responds to reality in the right direction. Deploy a change you know affects LCP — a hero image format change is ideal — and watch: your 24-hour view should move within a day, and the CrUX-shaped 28-day view should move gradually over four weeks. If your short window does not move at all, you are not measuring what you think you are.

Keep both views on the dashboard permanently. Engineers should read the per-route, all-browser number because it tells them what to fix. Stakeholders should read the CrUX-shaped number because it is what search reporting will show them, and having both visible prevents the recurring argument about which one is real.

FAQ

Can I make my RUM exactly match CrUX?

No, and it is not a useful goal. CrUX draws on an opt-in Chrome population you cannot enumerate, applies its own weighting, and rounds published values. What you can achieve is a reconciliation — an explanation of the gap in terms of coverage, window, unit and loss — that leaves a residual small enough to be uninteresting. Chasing an exact match wastes time that belongs on the pages that are actually slow.

Which number should the team be held to?

CrUX, because it is what search reporting uses and it is not something you can accidentally bias. But diagnose and prioritise with your own data: it has the attribution, the route granularity and the device segmentation that CrUX lacks. Score with one, work with the other.

Why is my CLS so much worse in CrUX than in my own data?

Usually because your beacon is missing the shifts that happen late in a visit. CLS accumulates until the page is hidden, so any delivery mechanism that fires early — on load, or on a timer — systematically under-reports it. Confirm the flush happens on the hidden transition and that a second flush cannot reset the accumulated value.

How often should I run the reconciliation?

Weekly is enough once the pipeline is stable, and the thing to record is the residual rather than either raw number. A residual that stays flat means both systems are healthy; one that drifts upward over a month points at a change in your instrumentation — a new route not covered, a delivery regression on one platform, a sampling rule that quietly excluded a device class. Reconciling more often than the underlying 28-day window updates mostly measures noise.

Can I use CrUX data programmatically to automate this?

Yes — the public API exposes per-origin and per-URL distributions, so the comparison can be a scheduled job rather than a manual exercise. Pull the origin-level histogram, compute the same view from your own table, and store both with the residual. The value of automating it is not the time saved but the fact that a slowly growing gap becomes visible on a chart instead of surfacing as an argument months later.

What if my site has too little traffic for CrUX?

Then your own RUM is the only field signal you have, and it becomes more important rather than less. Below the reporting threshold there is no external scoreboard, so instrument thoroughly, keep sampling at 100%, and lean on the segmentation you can do that CrUX never could. Compare against your own history rather than against an external number, and treat the absolute thresholds as the target.