Why is passing Core Web Vitals a binomial event?

The 75th percentile of a sample is at or below a threshold exactly when at least 75 percent of the samples are at or below that threshold. So the pass or fail decision depends only on a count: how many of the n samples in the window landed under the threshold. If each sample is an independent draw with true under-threshold probability q, that count follows a Binomial distribution with parameters n and q, and the window passes when the count reaches ceil(0.75 times n).

How many samples do I need before a Core Web Vitals p75 is trustworthy?

It depends entirely on how far your true under-threshold rate sits from 75 percent, not on how much traffic you have. Take 1.645 squared times q times one minus q, divided by the squared distance between q and 0.75. At a true rate of 78 percent that is about 520 samples. At 76 percent it is about 4,900. At exactly 75 percent no sample size ever stabilises the verdict, because the pass probability converges to one half.

What is flip risk for a Core Web Vitals badge?

Flip risk is the probability that the next reporting window returns the opposite pass or fail verdict from the current one, assuming nothing about the site changed and the true under-threshold rate stayed the same. It is the binomial tail on the wrong side of the 75 percent cut. A green field-data badge with a 30 percent flip risk will turn red roughly one month in three for no engineering reason at all.

Why use a Wilson score interval instead of the normal approximation?

The textbook Wald interval, the point estimate plus or minus z times the square root of p times one minus p over n, has badly wrong coverage when p is near 0 or 1 or when n is small, and it can produce bounds below 0 or above 1. The Wilson score interval inverts the score test instead, keeps its bounds inside 0 and 1 by construction, and holds close to nominal coverage even for small samples and extreme proportions. Since a good Core Web Vitals rate is usually 0.75 to 0.99, that matters.

What are the Core Web Vitals thresholds evaluated at the 75th percentile?

Largest Contentful Paint is good at 2500 ms or less and poor above 4000 ms. Interaction to Next Paint is good at 200 ms or less and poor above 500 ms. Cumulative Layout Shift is good at 0.10 or less and poor above 0.25. The supporting diagnostic metrics use 1800 ms and 3000 ms for First Contentful Paint and 800 ms and 1800 ms for Time to First Byte. All are assessed at the 75th percentile of page loads, segmented separately for mobile and desktop.

Does the order-statistic interval need a normal distribution?

No. The rank window is distribution free. It says only that the true 75th percentile of the population lies between the j-th and k-th smallest values of your sorted sample, with ranks derived from the binomial variance of the count below the quantile. Web performance distributions are heavily right skewed and multi modal, so any method that assumes normality of the underlying latency values would be wrong. Ranks are safe because they depend only on ordering.

Core Web Vitals p75 Sample Size Confidence

Your field data says you pass. Is that a real signal or a coin flip? A p75 passes exactly when at least 75 percent of samples land under the threshold, so the verdict is a Binomial event. This page computes the exact binomial pass probability, a Wilson interval on the true rate, the rank window your p75 value really sits in, the month-to-month flip risk, and the sample size that would settle it. Everything runs in your browser.

1. Metric and threshold

Metric

Good threshold (ms)

Your reported p75 (optional)

Headroom to threshold

2. Your measurement window

Samples in window (n)

Percent at or under threshold

or count at or under threshold

Confidence level

Design effect (clustering)

Effective sample size

Pass needs at least

A p75 is at or under the threshold precisely when ceil(0.75 × n) samples or more are at or under it, so the whole pass or fail decision is one count. The design effect divides n before any statistics are computed: real RUM samples are not independent because a handful of returning users, one slow region, or a bot fleet can supply many correlated page views. Leave it at 1.0 for a clean independent sample; 1.5 to 2.5 is a realistic penalty when a small share of visitors dominates the window.

3. Verdict

Trustworthy

—

This window

P(pass) at the estimated rate

Flip risk next window

—

Samples for a stable verdict

0% confident50%P(pass)100%

4. The binomial that decides it — distribution of the under-threshold count, with the 75 percent line

Counts that pass (at or above the 75 percent line) Counts that fail Pass line k* = ceil(0.75n) Your observed count

5. Wilson score interval on the under-threshold rate

Point estimate q-hat

Lower bound

Upper bound

0 pp

Margin of error

6. Where the p75 value itself really sits — distribution-free order-statistic rank window

—

Lower rank j

—

Upper rank k

—

Percentile band

—

Window width as share of n

7. Sample size you would need

Target margin of error (percentage points)

Power for A/B test

A/B baseline good rate (%)

A/B expected good rate (%)

Question	Effective n	Raw samples	Formula

The method, exactly

Core Web Vitals is not scored on an average. It is scored on the 75th percentile of real page loads, and a percentile has a property that makes the whole thing tractable: the sample p75 is at or below a threshold T if and only if at least 75 percent of the samples are at or below T. Using the nearest-rank definition, the sample p75 is the value at rank ceil(0.75n) in the sorted sample, so the pass condition collapses to a single count.

Step 1 — the pass event is Binomial

Let q be the true probability that a random page load comes in at or under the threshold, and let X be the number of the n samples in the window that do. If samples are independent draws, X follows a Binomial distribution, and the window passes when X reaches the pass rank.

X ~ Binomial(n, q) k* = ceil(0.75 × n) P(pass) = P(X ≥ k*) = Σ_i=k*ⁿ C(n, i) qⁱ (1−q)ⁿ⁻ⁱ

This page evaluates that tail exactly for windows up to 120,000 samples. The binomial coefficient overflows a double long before that, so each term is computed in log space with a Lanczos log-gamma:

log C(n, i) = lnGamma(n+1) − lnGamma(i+1) − lnGamma(n−i+1) log pmf(i) = log C(n, i) + i·ln q + (n−i)·ln(1−q)

The summation starts at the mode side of the cut and walks outward using the exact term ratio pmf(i+1)/pmf(i) = ((n−i)/(i+1)) · q/(1−q), so terms shrink monotonically and there is no catastrophic cancellation. Above 120,000 samples the tool switches to the normal approximation with a continuity correction, which by then agrees with the exact value to far more digits than anyone needs:

P(X ≥ k*) ≈ 1 − Φ( (k* − 0.5 − nq) / √(nq(1−q)) )

Step 2 — Wilson score interval on q

You never know q. You know the observed fraction q̂. The Wilson score interval inverts the score test, which keeps its bounds inside 0 and 1 and holds coverage far better than the naive Wald interval when the proportion is close to 1 — which is exactly where a healthy Core Web Vitals rate lives.

centre = (q̂ + z²/2n) / (1 + z²/n) half = z/(1 + z²/n) × √( q̂(1−q̂)/n + z²/4n² ) CI = centre ± half

The single question that matters: does that interval straddle 0.75? If it does, the true p75 could genuinely be on the other side of the threshold and your green or red badge is not evidence of anything. If the whole interval sits above 0.75, you pass for real.

Step 3 — an interval on the p75 value, not just the rate

Step 2 bounds the rate. To bound the value of the 75th percentile you need a distribution-free order-statistic interval, because web latency distributions are heavily right skewed and nothing about them is normal. The count below the quantile is again Binomial with p = 0.75, so its standard deviation is √(np(1−p)), and the ranks that bracket the true quantile are:

p = 0.75 j = floor( np − z√(np(1−p)) ) k = ceil ( np + z√(np(1−p)) ) + 1 CI on the true p75 = [ x_(j) , x_(k) ] from your sorted sample

Sort your raw samples ascending, read off the j-th and k-th values, and that pair brackets the population p75 at your confidence level. The width of that rank window as a share of n is the honest answer to "how sharply is my p75 pinned down". At n = 1,000 and 95 percent confidence the window runs from rank 723 to rank 778 — your "75th percentile" is really somewhere between the 72.3rd and the 77.8th percentile. On a steep part of a latency curve that can be hundreds of milliseconds.

Step 4 — flip risk

Flip risk answers the question engineering leads actually get asked: will this badge still be green next month if we change nothing? Hold q fixed at the estimate, draw a fresh window of the same size, and ask for the probability of the opposite verdict.

currently passing: flipRisk = P(X < k*) = 1 − P(X ≥ k*) currently failing: flipRisk = P(X ≥ k*)

The pathological case is a true rate sitting exactly at 0.75. Then the pass probability converges to one half no matter how much traffic you collect, and the badge is a coin toss forever. That is not a measurement bug; it is what happens when a continuous quantity is reported through a hard cut-off.

Step 5 — the sample sizes

Three different questions, three different formulas. To resolve q to a margin m:

n ≥ z² · q(1−q) / m²

To stop the verdict flipping more than a fraction α of the time, invert the normal approximation of the pass probability around the 0.75 cut, using the one-sided z:

n ≥ z_α² · q(1−q) / (q − 0.75)²

And to call a before-and-after or A/B change in the good-rate significant, the standard two-proportion z-test size per arm, with p̄ the pooled rate:

n_{per arm} = [ z_α/2√(2p̄(1−p̄)) + z_β√(p₁(1−p₁) + p₂(1−p₂)) ]² / (p₂ − p₁)²

Note the shape of the flip-risk formula: the required n grows with the inverse square of how far your true rate sits from 75 percent. At a true rate of 78 percent you need about 520 samples for a flip rate under 5 percent. At 76 percent you need about 4,900 — roughly a tenfold jump for a two-point move in the rate.

The thresholds this tool loads

Metric	Good (p75 at or under)	Poor (p75 above)
LCP — Largest Contentful Paint	2500 ms	4000 ms
INP — Interaction to Next Paint	200 ms	500 ms
CLS — Cumulative Layout Shift	0.10	0.25
FCP — First Contentful Paint	1800 ms	3000 ms
TTFB — Time to First Byte	800 ms	1800 ms

LCP, INP and CLS are the three Core Web Vitals. FCP and TTFB are diagnostic metrics reported alongside them, not part of the Core Web Vitals assessment. Every one of them is assessed at the 75th percentile, segmented separately for mobile and desktop.

Assumptions this model makes, stated plainly

Independence. The binomial assumes each sample is an independent draw. Field data is clustered by user, session, device, network and geography, which inflates the true variance. The design effect input is the standard correction: divide n by it and everything downstream widens honestly.
A stable underlying rate. Flip risk holds q constant between windows. If your traffic mix shifts — a campaign brings in a slower country, a new device tier arrives — the real month-to-month variation is larger than the number shown here, not smaller.
Nearest-rank percentile. The pass rank is ceil(0.75n). Tools that interpolate between order statistics differ by at most one rank, which is immaterial except at very small n, where nothing is trustworthy anyway.
CrUX bucketing. The Chrome UX Report publishes histogram buckets rather than raw values and derives percentiles from them, and it does not publish its minimum-sample threshold for a URL or origin to appear. Treat CrUX percentile figures as slightly coarser than the model here assumes; the rate-based conclusions still hold.
Not a field-versus-lab bridge. Lighthouse runs a single synthetic load. It cannot produce a p75 at all, so none of this applies to a lab score.

How to get the numbers to type in

If you run your own RUM, the count you need is one query: how many beacons in the window had the metric at or under the threshold, and how many beacons there were in total. If you only have the CrUX API, the histogram gives you the cumulative density under each bucket boundary directly, and the bucket edges for LCP, INP and CLS are exactly the good and poor thresholds above — so the density of the first bucket is your under-threshold proportion. What CrUX will not give you is n, which is why the design effect and the "samples for a stable verdict" figure matter most for low-traffic origins.

Frequently asked questions

My p75 improved from 2600 ms to 2450 ms. Did anything actually change?

Compare the two under-threshold rates, not the two millisecond figures. Put the before rate and the after rate into section 7 and read the per-arm sample size. If each window is smaller than that number, the movement is inside the noise band and you cannot attribute it to your change. A p75 crossing the threshold is the least informative way to detect an improvement, because it discards everything about the shape of the distribution except one count.

Why does the tool sometimes say "too few samples" even when the percentage looks great?

Because a great percentage on tiny n is not evidence. At n = 40 with 85 percent under the threshold, the 95 percent Wilson interval runs from roughly 71 to 93 percent, which straddles 0.75 comfortably. The verdict is flagged when the effective sample size is under 30 or when the margin of error exceeds 10 percentage points, since below that you cannot distinguish a genuine pass from a genuine fail.

Is the exact binomial really different from the normal approximation?

At the sizes where the answer matters, yes. For small n the binomial is discrete and skewed and the continuity-corrected normal can be off by several percentage points in the tail, which is precisely the region flip risk lives in. That is why this page sums the exact tail up to 120,000 samples and only falls back to the approximation above it, where the two agree to many decimal places.

What design effect should I use?

If you can, measure it: compute the observed variance of the good-rate across your reporting windows and divide by the binomial variance q(1−q)/n you would expect from independent sampling. That ratio is the design effect. If you cannot measure it, 1.0 is optimistic, 1.5 to 2.5 is a reasonable range for consumer traffic where a minority of returning users generates a majority of page views, and higher is plausible if a single slow market dominates your sessions.

Can a site permanently sit on the boundary?

Yes, and it is common. If your true under-threshold rate is genuinely 0.75, the pass probability is about one half at every sample size, and the badge will alternate more or less at random forever. No amount of extra traffic fixes it. The only fix is to move the underlying distribution, so aim for a real rate of 0.80 or above and treat the gap between your rate and 0.75 as the actual engineering target.

Does this replace a proper A/B test?

No. Section 7 gives you the sample size for a two-proportion z-test on the good-rate, which is the right test when you are comparing two independent groups on a pass or fail outcome. It does not handle sequential peeking, multiple comparisons across metrics, or covariate adjustment. If you check the numbers every day and stop when they look good, your real false-positive rate is far above the nominal one.

About the author. Built by Michael Lip, a solo developer shipping performance and developer tooling across the Zovo network. This calculator implements the exact binomial tail with Lanczos log-gamma summation, the Wilson score interval, and the standard distribution-free order-statistic quantile interval. Everything runs client side; nothing you type leaves the page.

Last updated 29 July 2026