Hill Country Bots

How to check Core Web Vitals for a whole site (in bulk)


Published November 12, 2026 by Hill Country Bots

PageSpeed Insights checks one URL at a time. That's fine for your homepage. It's not fine when the question is "which of our 300 pages are slow?" This guide covers the free ways to answer that first, then the paid tools with their public prices, then the bulk checker we built. Every number below comes from a run we did tonight on our own site, or from a page we link to.

The short answer

  1. Want to know how real visitors experience the site? Open the Core Web Vitals report in Google Search Console. It's free, but it only covers your own verified site, and it groups pages instead of listing every one.
  2. Want a score for every page? Pull the URLs from your sitemap, then loop over them with the PageSpeed Insights API (free key) or Lighthouse CLI on your own machine.
  3. Don't want to write the loop? Use a bulk tool. There's a price table below, and several free ones handle up to 500 URLs.

First, know which numbers you're looking at

There are two kinds of data, and most of the confusion about "bulk Core Web Vitals" comes from mixing them up.

Google's targets for a good page are LCP within 2.5 seconds, INP of 200 milliseconds or less, and CLS of 0.1 or less, measured at the 75th percentile (web.dev). Field data needs enough real traffic. Ours didn't have any (more on that below), which is normal for a small or new site.

Step 0: get the list of pages

Almost every bulk method starts with a URL list, and the sitemap is the quickest source. For a plain sitemap, this one-liner prints every URL into a text file:

curl -s https://example.com/sitemap.xml | grep -o '<loc>[^<]*' | sed 's/<loc>//' > urls.txt

On our site that gave 31 URLs. If your sitemap is an index that points at other sitemaps, you'll need to open each child too. Our guide on finding every page on a website covers that, and the free sitemap to CSV page does it in the browser.

Free method 1: the Search Console Core Web Vitals report

If you own the site, start here. It's Google's own view of real-user data for your pages, split into mobile and desktop, and grouped as Good, Needs improvement and Poor for LCP, INP and CLS.

Know its limits before you rely on it. Straight from Google's help page:

So it's great for "is the blog template slow on mobile?" and not much use for "give me a sortable table of all 300 pages."

Free method 2: loop over the PageSpeed Insights API

This is the same engine as the PageSpeed Insights website, called from a script. Google documents it on the getting started page.

Get an API key first. Google's page says a key is recommended for automated queries, but when we sent a request without one at 8:42 PM Central on November 12, 2026, Google answered 429 "Quota exceeded" on the very first try. A free key is created in Google Cloud. Google doesn't print the quota on the PSI docs, so read it on your own project's quota page (APIs & Services, then PageSpeed Insights API, then Quotas). The Lightscore write-up on PSI rate limits got the same keyless 429 and goes deeper on this.

Here's a small loop that writes one CSV row per page. It needs curl and jq:

KEY="your-api-key"
echo "url,performance,lcp_ms,cls,tbt_ms" > psi.csv
while read -r u; do
  curl -sG "https://www.googleapis.com/pagespeedonline/v5/runPagespeed" \
    --data-urlencode "url=$u" -d strategy=mobile -d key="$KEY" -o page.json
  jq -r --arg u "$u" '[$u,
    (.lighthouseResult.categories.performance.score * 100),
    .lighthouseResult.audits["largest-contentful-paint"].numericValue,
    .lighthouseResult.audits["cumulative-layout-shift"].numericValue,
    .lighthouseResult.audits["total-blocking-time"].numericValue] | @csv' page.json >> psi.csv \
    || echo "\"$u\",error" >> psi.csv
  sleep 2
done < urls.txt

We tested the jq part on real output tonight, including the error case, where a 429 response becomes an error row instead of killing the loop. Each call takes a while because Google loads the page for real, so a few hundred URLs is a "go get coffee" job.

One heads-up: Google says it plans to stop including CrUX field data in this API and points people to the CrUX API instead (same getting started page). If you need field data in bulk, method 3 is the long-term answer.

Free method 3: the CrUX API for real-user data

If you only care about real-user Core Web Vitals, skip Lighthouse and ask CrUX directly. It's fast because nothing gets loaded. Per Chrome's docs, it's limited to 150 queries per minute per Google Cloud project, offered without charge, and you can't pay for more. One request looks like this:

curl -s -X POST "https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/pricing", "formFactor": "PHONE"}'

Swap "url" for "origin": "https://example.com" to get the whole site in one number. The catch is coverage: pages without enough Chrome traffic return nothing, and on a small site that can be most of them.

Free method 4: Lighthouse CLI on your own machine

Lighthouse is open source, and the command line version runs the same lab audit locally. Per Chrome's Lighthouse docs, you need Google Chrome and the current LTS version of Node.

npm install -g lighthouse
i=0
while read -r u; do
  i=$((i+1))
  lighthouse "$u" --only-categories=performance --output=json \
    --output-path="lh-$i.json" --quiet --chrome-flags="--headless=new"
done < urls.txt
jq -r '[.finalDisplayedUrl, .categories.performance.score * 100,
  (.audits["largest-contentful-paint"].numericValue | round)] | @csv' lh-*.json

No key and no quota. The trade-offs: it only gives lab data (Screaming Frog's docs say the same thing about running Lighthouse locally: no CrUX), it's slower because it runs one page at a time on your hardware, and the numbers depend on that hardware. We ran exactly this loop on our 31 URLs tonight with Lighthouse 13.6.0. It took 5 minutes 45 seconds.

If you'd rather not write the loop, Unlighthouse is a free, MIT-licensed tool that finds your pages and runs Lighthouse across all of them with one command (npx unlighthouse@beta --site https://example.com on their homepage today). Honestly, for a one-off local audit it's the nicest free option we found.

The tools, side by side

Prices and limits below are from each tool's public page, read on November 12, 2026, between 8:40 and 8:45 PM Central. If one has changed, email [email protected] and we'll fix it.

OptionHow many URLsReal-user dataPrice
Search Console CWV reportGrouped sample of your indexed pages, 200-row tableYes, that's all it showsFree, your own verified site only
PageSpeed Insights API + your scriptWhatever your project quota allowsYes for now (Google plans to drop it)Free key; you write and keep the script
CrUX API150 queries a minuteYes, only thatFree, no paid tier
Lighthouse CLI or UnlighthouseNo cap, runs on your machineNo, lab onlyFree, open source
EXPERTE.com Core Web Vitals checkUp to 500 URLs; crawl limited to 60 secondsSays it uses the same data as PageSpeed InsightsNo price or sign-up shown on the tool page
SpeedVitals bulk CWV checkerUp to 500 URLs, sitemap importYes, from CrUXFree plan (15 speed tests a week); paid plans listed from $19 to $399 a month (pricing)
PageSpeed Plus10 URLs on the free tool; full sitemap scans on a planLab via the PSI API; scheduled weekly scans€20, €60 or €200 a month; 14-day trial
Screaming Frog SEO SpiderFree up to 500 URLs; more with a licenceYes, through its PageSpeed Insights API connection$279 per licence a year (1 to 4 licences)
Bulk PageSpeed Checker (we built it)No small cap; you set a maxYes when Google has it, page and origin$3 per 1,000 successful audits; failed ones free

Where the others win, plainly: for a one-time look at up to 500 pages, the free SpeedVitals and EXPERTE checkers are hard to argue with. If you already pay for Screaming Frog, its PSI connection gets you this inside a crawl you're running anyway. PageSpeed Plus gives you dashboards and history out of the box. And Search Console is the only one here that shows Google's grouped view of your real visitors.

Our tool, and what it found on our own site

We built the Bulk PageSpeed Checker, and it runs on Apify. It does the method 2 loop for you: paste URLs, pick mobile, desktop or both, and download a CSV, Excel or JSON file. It calls Google's PageSpeed Insights API with its own key, so you don't need one (you can add yours). It costs $0.003 per successful audit, which is one URL on one device. Failed, invalid and timed-out audits aren't charged. We re-read that price on the Apify listing at 8:40 PM Central tonight. One honest limit: it takes a URL list. It doesn't read a sitemap by itself, so do step 0 first.

Want to try one page before you sign up for anything? The free site check on our site runs the same Google test on a single URL, mobile and desktop, with no account.

The real run

At 8:42 PM Central on November 12, 2026, we fed it all 31 URLs from our own sitemap, mobile only (Apify run VgKKuyNy6xheWrRjm). It finished in 80 seconds: 31 audited, 0 failed. At the list price that's 31 × $0.003 = $0.093. The same night, the Lighthouse CLI loop above took 5 minutes 45 seconds for the same 31 pages on our machine.

What we measured (mobile)Bulk PageSpeed Checker (Google's servers)Lighthouse CLI (our machine)
Performance score97 to 10099 to 100
Lab LCP759 to 1,556 ms787 to 1,620 ms
Lab CLS0 on 30 pages, 0.005 on /contact0 on all 31
Real-user (CrUX) dataNone for any page, or for the whole originNot available locally
Time for 31 URLs80 seconds5 min 45 s

Our pages are small static HTML, so the scores are boringly high. A big WordPress theme with a pile of plugins will look different. The useful parts were the misses:

Here's one row as the tool returned it, shortened:

{
  "url": "https://hillcountrybots.com/tools/",
  "finalUrl": "https://hillcountrybots.com/tools/",
  "strategy": "mobile",
  "status": "ok",
  "scores.performance": 100,
  "lab.lcpMs": 766,
  "fieldData": null,
  "originFieldData": null,
  "opportunities": [
    { "title": "Avoid multiple page redirects", "displayValue": "Est savings of 750 ms" }
  ]
}

What to do with the results

Questions

Can I check Core Web Vitals for my whole site for free?

Yes. If you own the site, the Core Web Vitals report in Google Search Console groups your indexed pages by real-user status. For a page by page table, loop over your sitemap with the PageSpeed Insights API (free key) or run Lighthouse CLI on your own machine.

Why does my site show no Core Web Vitals data?

Real-user data comes from the Chrome UX Report, which only covers pages and origins with enough Chrome visitors. New or quiet sites often have none. Ours had none tonight. You still get lab numbers from Lighthouse for every page that loads.

Do I need an API key for the PageSpeed Insights API?

In practice, yes. When we called the API without a key on November 12, 2026, Google answered 429 (quota exceeded) on the first request. Get a free key in Google Cloud, then check your project's quota page for your real limit.

How many URLs can I check at once?

With your own script, as many as your API quota allows. Free web tools we looked at cap a run at 10 to 500 URLs. Lighthouse CLI has no cap, but it runs one page at a time on your machine.

Related

Google Search Console, PageSpeed Insights, Lighthouse and Chrome are trademarks of Google LLC. Hill Country Bots is not affiliated with, endorsed by or sponsored by Google LLC or any other company named here.