How to extract URLs from a sitemap to CSV
Published November 12, 2026 by Hill Country Bots
A sitemap is the site's own list of the pages it wants search engines to find. Turning that list into a CSV gives you a page inventory you can sort, filter or feed into another check. This guide starts with methods that work with no account and no special tool. We ran every command below against real public sitemaps on the evening of November 12, 2026, Central Time, and we quote the output we actually got. Near the end we point to the free converter on this site, and to the Sitemap URL Extractor we built on Apify for the cases a one-file paste cannot cover.
The short answer
- Find the sitemap first. Open
/robots.txtand look for lines that start withSitemap:. If none are listed, try/sitemap.xmlor/sitemap_index.xml. - If it's a small, single file, you can finish in the browser, in Google Sheets, or with a one-line
curlcommand. - If it's a sitemap index, a
.gzfile, or several files, use the short Python script below, or a tool that follows child sitemaps for you. - Collect only the page addresses. That means the
locunder eachurlentry, not everylocin the file. Image sitemaps also list image files.
Step 0: find the sitemap
The most reliable pointer is robots.txt. Our own file is short and clear:
User-agent: *
Allow: /
Sitemap: https://hillcountrybots.com/sitemap.xml
A site can list more than one. On the same evening, wordpress.org/robots.txt had nine Sitemap: lines: the main index, a news sitemap, then separate ones for themes, plugins, news, showcase, documentation, patterns and photos. If you only open /sitemap.xml, you miss the rest.
Common fallback paths, if robots.txt has no Sitemap line, are /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml (WordPress 5.5 and later) and the same paths with .xml.gz. Guessing still fails often enough that we hit two examples tonight:
- en.wikipedia.org/sitemap.xml returned HTTP 404. The site's robots.txt points at a different address, under
/w/rest.php/.... https://wordpress.org/sitemap.xml.gzredirected to/wp-sitemap.xml, and that URL also returned 404. The file that does exist is/sitemap.xml, which is the one robots.txt already named.
Once you have the file, look at the root tag. There are two shapes. A urlset lists pages. A sitemapindex lists other sitemap files, and the page URLs are one step further on (sometimes more than one). The sitemaps.org protocol caps each file at 50,000 URLs or 50 MB uncompressed, which is why large sites split the list. You may also compress a file with gzip; the cap applies after you unpack it.
Method 1: the browser
Open the sitemap URL. For a short urlset, you can copy the addresses by hand or save the file and go through it. Our sitemap has 32 URLs, and that is about the size where this still feels reasonable.
It stops being reasonable fast. When we opened wordpress.org/sitemap.xml, the file was a sitemap index with three children:
https://wordpress.org/sitemap-1.xml
https://wordpress.org/image-sitemap-1.xml
https://wordpress.org/video-sitemap-1.xml
Those are not page URLs. And a gzip sitemap will not show up as readable XML until something unpacks it.
If you'd rather not copy tags by hand, paste the file (or upload it) into our free sitemap to CSV converter. It runs in your browser, so the file is not sent to us. You get a table and a CSV download with url, lastmod, changefreq and priority. If you upload a .gz file, modern browsers can unpack it on the page. If you paste a sitemap index, the tool lists the child sitemap files and says so; it does not fetch those children itself. That's the honest limit of a converter that never leaves your machine.
Method 2: Google Sheets IMPORTXML
Google Sheets can pull a sitemap into a column with IMPORTXML. The usual first try is //loc or //url/loc, and that often returns nothing. The reason is the XML namespace on the root tag. XPath treats a bare name as "no namespace," and a valid sitemap has one, so the query is legal and matches zero nodes.
We ran the same kind of lookup in Python on our own sitemap, on sitemaps.org, and on the wordpress.org index. In all three cases, findall('.//loc') returned 0. The files actually contained 32, 84 and 3 loc elements. The fix is to match the element by its local name:
=IMPORTXML("https://hillcountrybots.com/sitemap.xml", "//*[local-name()='url']/*[local-name()='loc']")
That expression asks only for loc children of url entries, so you don't pick up child sitemap addresses or image file URLs. Then use File, Download, Comma-separated values (.csv). We did not run this formula in Google Sheets tonight; the function and its arguments come from Google's help page, and the namespace result comes from the Python checks above.
One formula reads one URL. It will not follow a sitemap index for you. If you get an index back, put each child address in its own cell and run IMPORTXML on those cells. We also did not test gzip files or very large sitemaps in Sheets, so we won't guess how those behave.
Method 3: a command-line one-liner
For a flat, uncompressed urlset, this is enough:
curl -s https://hillcountrybots.com/sitemap.xml \
| grep -oE '<loc>[^<]+</loc>' \
| sed -E 's#</?loc>##g' > urls.csv
We ran that pipeline at 10:03 PM Central on November 12, 2026. It printed 32 URLs. The first five were:
https://hillcountrybots.com/
https://hillcountrybots.com/bulk-pagespeed-checker
https://hillcountrybots.com/domain-availability-checker
https://hillcountrybots.com/guides/
https://hillcountrybots.com/n8n-nodes
Pointed at wordpress.org/sitemap.xml, the same command printed exactly three lines, and they were the child sitemap files, not pages. The one-liner also misses lastmod and does not unpack gzip. Use it for a quick look at a small file, then stop.
Method 4: a short Python script
This script uses only the Python standard library. It follows nested sitemap indexes, unpacks gzip when the file starts with the gzip magic bytes (so the filename can be wrong and it still works), and writes one CSV row per url entry with the page address, lastmod and the sitemap it came from. It matches tags by their local name, so it still works when a file uses an older namespace than the usual 0.9 one.
#!/usr/bin/env python3
import csv, gzip, sys, urllib.request
import xml.etree.ElementTree as ET
def local(tag):
return tag.rsplit("}", 1)[-1]
def fetch(url):
req = urllib.request.Request(
url,
headers={"User-Agent": "HillCountryBots-sitemap-to-csv/1.0 (+https://hillcountrybots.com/contact)"},
)
with urllib.request.urlopen(req, timeout=30) as response:
data = response.read()
if data[:2] == b"\x1f\x8b":
data = gzip.decompress(data)
return ET.fromstring(data)
def walk(url, rows, seen=None, depth=0):
if seen is None:
seen = set()
if url in seen or depth > 5:
return
seen.add(url)
root = fetch(url)
kind = local(root.tag)
if kind == "sitemapindex":
for child in root:
if local(child.tag) != "sitemap":
continue
loc = next(((c.text or "").strip() for c in child if local(c.tag) == "loc"), "")
if loc:
walk(loc, rows, seen, depth + 1)
return
for entry in root:
if local(entry.tag) != "url":
continue
loc = next(((c.text or "").strip() for c in entry if local(c.tag) == "loc"), "")
lastmod = next(((c.text or "").strip() for c in entry if local(c.tag) == "lastmod"), "")
if loc:
rows.append({"url": loc, "lastmod": lastmod, "sitemap": url})
if __name__ == "__main__":
if len(sys.argv) < 2:
sys.exit("Usage: python3 sitemap_to_csv.py https://example.com/sitemap.xml")
rows = []
walk(sys.argv[1], rows)
writer = csv.DictWriter(sys.stdout, fieldnames=["url", "lastmod", "sitemap"])
writer.writeheader()
writer.writerows(rows)
print(f"# {len(rows)} rows", file=sys.stderr)
Run it like this (we used Python 3.13.5):
python3 sitemap_to_csv.py https://hillcountrybots.com/sitemap.xml > urls.csv
On our sitemap, stderr printed # 32 rows. The CSV started:
url,lastmod,sitemap
https://hillcountrybots.com/,2026-11-12,https://hillcountrybots.com/sitemap.xml
https://hillcountrybots.com/bulk-pagespeed-checker,2026-11-12,https://hillcountrybots.com/sitemap.xml
https://hillcountrybots.com/domain-availability-checker,2026-11-12,https://hillcountrybots.com/sitemap.xml
The same script against https://www.sitemaps.org/sitemap.xml wrote 84 rows. The first three were https://www.sitemaps.org/ (lastmod 2016-11-21), https://www.sitemaps.org/protocol.html (2022-12-15) and https://www.sitemaps.org/faq.html (2016-11-21).
Pointed at https://wordpress.org/sitemap.xml, it followed all three child files and wrote 845 rows: 54 from sitemap-1.xml, 727 from the image sitemap and 64 from the video sitemap. Those 845 rows contain 222 unique page URLs, because a page can appear in more than one of those files, and the image and video sitemaps list the same page once per extra asset. The first row was:
https://wordpress.org/about/privacy/cookies/,2026-11-06T16:16:33Z,https://wordpress.org/sitemap-1.xml
The script does not read robots.txt, so it only sees the tree you point it at. On wordpress.org that means the three children of the main index, not the eight other sitemaps listed in robots.txt. It also has no retries. If a single child file fails, that walk stops. Those are fair reasons to switch to a tool when the tree is large or you want this on a schedule.
Sitemap indexes, and what robots.txt can add
If you take nothing else from this guide, take this: open the file and check whether the root tag is urlset or sitemapindex before you celebrate a three-line CSV.
Indexes can nest. Tonight, several of the extra sitemaps in the wordpress.org robots.txt file were indexes of indexes. We opened all nine Sitemap addresses (structure only, we did not download every child): eight were sitemap indexes, and the remaining one, /news-sitemap.xml, was a urlset with a single URL. The themes, plugins, documentation, patterns and photos trees go deeper than the main /sitemap.xml index.
Gzip sitemaps
Don't trust the file extension alone. Look at the first two bytes. Gzip files start with 1f 8b. If you try to parse the compressed bytes as XML, you get a mess. We fetched https://www.nytimes.com/sitemaps/www.nytimes.com/sitemap.xml.gz (13,957 bytes, magic 1f8b0800). Feeding those bytes straight to Python's XML parser raised:
ParseError: not well-formed (invalid token): line 1, column 0
After unpacking, the file was 327,502 bytes, and it was a sitemap index, not a list of pages. It named 2,034 child sitemaps, and every child URL also ended in .xml.gz. We opened one child, sitemap_2020_04.xml.gz (122,572 bytes compressed, 901,939 uncompressed). That one was a urlset with 5,547 page URLs. The first three were:
https://www.nytimes.com/2020/04/30/opinion/coronavirus-vaccine-covid.html
https://www.nytimes.com/2020/04/22/nyregion/coronavirus-quarantine-nyc-bronx-diary.html
https://www.nytimes.com/2020/04/29/magazine/racial-disparities-covid-19.html
We did not walk all 2,034 children. That would be a long, heavy run for a demonstration, and the script above would hold every row in memory. If you need a full export of a site that size, use a tool that streams the files, or raise the memory and be patient.
Common errors
- Zero URLs from a file you can see in the browser. That's usually the namespace. Match local names, as in the Sheets formula and the Python script above.
- Your list is other sitemap files, not pages. You opened an index. Follow the children, or use a script that does.
- Your list is about twice as long as the number of pages. You collected every
loc, including image files. The wordpress.org image sitemap had 727urlentries and 727 nestedimage:locvalues, for 1,454locelements in total. The first entry looks like this:
<url>
<loc>https://wordpress.org/40-percent-of-web/</loc>
<lastmod>2022-10-04T14:46:18Z</lastmod>
<image:image>
<image:loc>https://wordpress.org/files/2021/05/wp-40-milestones.png</image:loc>
</image:image>
</url>
If you only want pages, take the loc under url. If you want the image files, take image:loc on purpose, in its own column.
- The usual path 404s. Read robots.txt. Wikipedia's
/sitemap.xmlwas 404 tonight. WordPress.org's guessed/wp-sitemap.xml(the core default) was also 404; Jetpack is serving/sitemap.xmlinstead. - A script that hard-codes the 0.9 namespace finds nothing. Most sitemaps use
http://www.sitemaps.org/schemas/sitemap/0.9. The index at https://www.google.com/sitemap.xml uses the olderhttp://www.google.com/schemas/sitemap/0.84namespace. A lookup locked to 0.9 found 0 of its 23 child sitemaps. Matching local names found all 23. The first washttps://www.google.com/gmail/sitemap.xml. - A
.gzfile parses as garbage. Decompress first, then check whether you now have a urlset or another index.
Our free converter, and the tool we built on Apify
If you want the CSV without running any of the commands above, use the free sitemap to CSV converter on this site. Paste or upload one file, including a .gz upload, and download the CSV. Nothing is sent to us. As noted above, an index comes back as a list of child files, which you can open one by one.
When the tree is bigger than that, we built the Sitemap URL Extractor, which runs on Apify. It reads robots.txt, follows nested indexes, unpacks gzip, and can run on a schedule or through the API. It costs $0.50 per 1,000 URLs kept after filters. Failed fetches, invalid inputs and URLs you filter out are free. We re-read that price from the live Apify API at 10:01 PM Central on November 12, 2026.
For one small sitemap, the free methods are the right place to start, ours included. The Apify tool is for many sites at once, include and exclude filters, or a list that needs to refresh every week and feed something else, such as our bulk PageSpeed checker.
Questions
How do I extract URLs from a sitemap to CSV for free?
For a small, single sitemap, paste it into our free converter, use a Google Sheets IMPORTXML formula, or run the curl one-liner in this guide. For a sitemap index or a gzip file, use the short Python script. None of those need a paid account.
Why does my formula or script return no URLs?
Most sitemaps put every tag in an XML namespace. A query or find call for loc, with no namespace handling, matches nothing. We saw this on our own sitemap on November 12, 2026: Python's findall('.//loc') returned 0 rows, while there were 32 loc elements. Use local-name() in XPath, or compare the local part of each tag in Python.
What is a sitemap index?
An index is a sitemap whose loc values point at other sitemap files, not at pages. The protocol allows up to 50,000 URLs or 50 MB per file (uncompressed), so large sites split the list. Opening the index alone will not give you the pages.
Can I read a gzip .xml.gz sitemap?
Yes. Check the first two bytes for the gzip marker, then decompress before you parse the XML. A New York Times sitemap we fetched on November 12, 2026 was 13,957 bytes compressed and 327,502 bytes after unpacking. It was an index of 2,034 child files, each of those also gzipped.
Does a sitemap list every page on a website?
No. It lists what the site chose to publish. Pages can be missing, and some sitemaps include image or video URLs as well as pages. If you need pages a sitemap left out, you have to crawl the site or check Search Console, which we cover in our guide on finding every page on a website.