Open on Apify Examples on GitHub
What it does
Give it a homepage or a direct sitemap URL. It discovers sitemap files from robots.txt and common paths, follows sitemap indexes, and unpacks compressed files. Optional include and exclude filters let you keep only the paths you care about.
Each kept URL is one row, with the sitemap fields when the file provides them. If a start URL is invalid or a sitemap cannot be read, you get a clear error row. Those rows are not charged.
This tool reads sitemap files only. It does not crawl pages or check the HTTP status of each URL. A page missing from the sitemap will not be invented.
Who it is for
It is the first step for a lot of bulk work: get the URL list, then audit or test speed.
- SEO agencies and consultants exporting a site's URL list for audits, migrations or index coverage checks.
- Developers who need a clean CSV or JSON list before a PageSpeed run or a content pass.
- Anyone automating this through the Apify API or MCP, then feeding URLs into another tool.
Price
$0.50 per 1,000 extracted URLs ($0.0005 per unique URL kept after filters). That is the live Apify Store price as of November 11, 2026.
Invalid start URLs, failed sitemap fetches and filtered-out URLs are free. 2,000 URLs from a large index is $1.00. Use maxUrls if you want a hard cap.
Example input and output
Input
A homepage is enough. Pass a direct sitemap.xml if you already have it, and turn discovery off.
{
"startUrls": [
"https://www.sitemaps.org/"
],
"discoverFromRobots": true,
"tryCommonPaths": true,
"maxUrls": 1000
}Output
A shortened row from a real run starting at sitemaps.org on November 10, 2026.
{
"url": "https://www.sitemaps.org/protocol.html",
"lastmod": "2016-11-21",
"changefreq": "monthly",
"priority": "0.8",
"sourceSitemap": "https://www.sitemaps.org/sitemap.xml",
"status": "ok"
}Questions
Do I need the sitemap URL, or will a homepage do?
A homepage is enough. The extractor reads robots.txt and also tries common sitemap paths.
Does it support gzip sitemaps and sitemap indexes?
Yes. It follows nested indexes and unpacks .gz files, up to the limits you set.
Does it crawl the website?
No. It only fetches robots.txt and sitemap XML. A URL appears in the output only if a sitemap lists it.
What can I do with the URL list?
A typical next step is to feed the url column into the SEO Audit Tool or the bulk PageSpeed checker.
Are failed fetches free?
Yes. Invalid start URLs, failed sitemap fetches and URLs dropped by include or exclude filters are not charged.