Loading tutorials…
Loading tutorials…
Yoast generates a multi-file sitemap_index.xml automatically — but the default output usually includes content you do not want indexed and excludes content you do. This is the audit + submit workflow that produces a sitemap Google actually trusts.
Who this is forWordPress site owners with Yoast SEO active who need to verify their sitemap is correct, fix sitemap 404s, exclude unwanted URLs, and submit cleanly to GSC and Bing. Especially relevant if Google shows "Submitted URL not found" errors or your sitemap returns inconsistent counts.
What you'll need
Step 1
Open yoursite.com/sitemap_index.xml in a private window. You should see an index file linking to sub-sitemaps. Any 404 here blocks everything else.
Open a private/incognito window. Navigate to yoursite.com/sitemap_index.xml.
You should see an XML document with a list of sub-sitemaps: post-sitemap.xml, page-sitemap.xml, category-sitemap.xml, etc. The exact list depends on which content types you marked indexable.
If you get a 404 or "page not found": Yoast SEO → Settings → APIs → REST API → confirm it is enabled. Most sitemap 404s trace to the REST API being blocked by a security plugin (Wordfence Premium, iThemes Security) or a server-level rewrite rule.
If you see the index but sub-sitemaps return 404, your permalink structure is broken. WordPress Admin → Settings → Permalinks → click Save (without changing anything). This flushes the rewrite rules. Re-test sitemap.
If sub-sitemaps load but show 0 URLs each, Yoast indexables table is empty. Yoast SEO → Tools → Optimize SEO data → run the indexable rebuild.
Step 2
Open every sub-sitemap link from the index. Verify each URL listed should actually be in Google. Remove unwanted ones via noindex or exclusions.
Click each sub-sitemap link from sitemap_index.xml: post-sitemap.xml, page-sitemap.xml, category-sitemap.xml, product-sitemap.xml (WooCommerce), etc.
Scan each list. Common things you want excluded: staging pages, thank-you pages, "coming soon" pages, draft duplicates, test posts, sandbox content.
For one-off exclusions: edit the specific post in WordPress → Yoast SEO sidebar → Advanced tab → "Allow search engines to show this page in search results?" → No. Save. The URL drops from the sitemap on the next regeneration.
For category-wide exclusions: Yoast SEO → Settings → Content Types or Categories & Tags → toggle Show in search results → NO. All items of that type drop from the sitemap.
For page-builder template URLs (Elementor, Divi, Bricks libraries): these usually live under /?elementor_library= or similar. Yoast should exclude them automatically since they are not public_query_var. If they appear, contact your page builder support.
Step 3
Yoast Premium has a per-post visibility toggle. Without Premium, exclude URLs by setting noindex on the post or via a code filter in functions.php.
Per-post (any Yoast): Open the WordPress post editor → Yoast SEO sidebar → Advanced → Allow search engines to show this page in search results? → No → Save. Yoast both noindexes the page AND removes it from the sitemap.
Per-content-type (any Yoast): Yoast SEO → Settings → Content Types → toggle Show in search results → NO for the whole content type.
Custom filter (developer): add to your theme functions.php or a child theme: add_filter('wpseo_sitemap_exclude_post_ids', function($ids) { return array_merge($ids, [123, 456]); }); — replaces 123, 456 with the post IDs to exclude.
For bulk exclusion via taxonomy term: add_filter('wpseo_sitemap_exclude_taxonomy', function($value, $taxonomy) { if ($taxonomy === 'product_tag') return true; return $value; }, 10, 2);
Test after every change: re-open the relevant sub-sitemap in a private window. The excluded URLs should be gone.
Step 4
GSC → Sitemaps → paste sitemap_index.xml → Submit. Status moves from Pending to Success within 24-72 hours.
Open Google Search Console → select your property → Sitemaps in the left sidebar.
In the "Add a new sitemap" field, paste: sitemap_index.xml (just the filename, GSC prepends your domain).
Click Submit. The status row appears: Pending → Couldn't fetch → Success (or Has errors).
If "Couldn't fetch" persists for >2 hours, GSC could not reach the sitemap. Causes: robots.txt blocks the file, the URL is wrong, or your site returns a 5xx on robot requests. Fix the underlying issue, then click the row → Resubmit.
Wait 24-72 hours for Google to process. The "Discovered URLs" count should roughly match your sub-sitemap totals. Mismatch by more than 20% usually means robots.txt blocks or canonical tag issues.
Step 5
Bing Webmaster Tools → Sitemaps → submit sitemap_index.xml. Also enable IndexNow for instant URL pinging.
Bing Webmaster Tools → bing.com/webmasters → your site → Sitemaps → Submit Sitemap. Paste sitemap_index.xml.
Bing also operates IndexNow — a real-time URL ping protocol Yoast supports natively. Yoast SEO → Settings → Site features → IndexNow → toggle ON. Yoast pings Bing automatically every time you publish or update a post.
IndexNow also pings Yandex and other supported search engines. No GSC equivalent — Google does not participate in IndexNow.
Bing's sitemap fetcher is faster than Google's — usually processes within 4-12 hours. Useful as a secondary signal that your sitemap is correctly formatted.
Step 6
GSC → Indexing → Sitemaps → click your sitemap row. Watch for "Has errors" status and click through to specific URL issues.
Google Search Console → Indexing → Sitemaps. Click sitemap_index.xml row to see per-sub-sitemap status.
Each sub-sitemap shows its own status (Success / Has errors). Click into errors to see specific URL issues.
Common sitemap errors: "Submitted URL not found (404)" → the URL was deleted but is still in the sitemap (waits for next Yoast regeneration), "Submitted URL blocked by robots.txt" → robots.txt disallows a URL the sitemap submits (fix robots.txt or remove from sitemap), "Submitted URL has noindex" → meta noindex conflicts with sitemap inclusion (usually a Yoast bug — force indexable rebuild).
Set a calendar reminder to check GSC sitemap status every Monday. Most weeks it is a 60-second check. Catching errors early prevents them piling up.
Step 7
Yoast adds a Sitemap: line to the virtual robots.txt automatically. Verify yoursite.com/robots.txt includes the line.
In a private window, visit yoursite.com/robots.txt.
Look for a line like: Sitemap: https://www.yoursite.com/sitemap_index.xml.
Yoast adds this automatically to the virtual robots.txt WordPress generates. If you have a physical robots.txt file (uploaded to your server root), it overrides the virtual one — and Yoast cannot inject the line.
If you have a physical robots.txt: edit it manually to add the Sitemap line. Or delete the physical file to let WordPress + Yoast generate the virtual one.
The Sitemap directive helps crawlers other than Google (Yandex, Baidu, DuckDuckGo) discover your sitemap without manual submission.
Common mistakes
Submitting sitemap.xml instead of sitemap_index.xml
What goes wrong: GSC shows 'Couldn't fetch' or 404. You see error notifications and assume something is broken with Yoast when the submitted URL is just wrong. Time wasted debugging the wrong problem.
How to avoid: Always submit sitemap_index.xml — the index file. GSC follows the index links to the sub-sitemaps automatically. Never submit individual sub-sitemaps.
Leaving thank-you pages and other conversion-completion URLs indexable
What goes wrong: /thank-you, /order-complete, /demo-confirmed URLs appear in the sitemap and get indexed. Users find them via Google, skip the form, contaminate conversion analytics. CRO numbers degrade silently.
How to avoid: Open each thank-you / completion page → Yoast SEO sidebar → Advanced → Allow search engines to show this page → No. Repeat for every funnel-completion URL.
Not flushing the indexables table after major URL changes
What goes wrong: You change permalinks, restructure categories, or bulk-delete posts. The Yoast indexables table still references old URLs. Sitemap submits 404s. GSC logs hundreds of 'URL not found' errors.
How to avoid: Yoast SEO → Tools → Optimize SEO data → Start SEO data optimization. Rebuilds the indexables table from current WordPress data. Re-submit sitemap.
Conflicting robots.txt and sitemap noindex flags
What goes wrong: Robots.txt disallows a path that is also marked noindex in Yoast. Google can't crawl to see the noindex, so the page can still appear in the index with no snippet. Worse: the sitemap includes the URL because Yoast respects noindex but the robots block prevents crawl.
How to avoid: Choose ONE method per URL: either noindex (let Google crawl) OR robots.txt disallow (block crawl). Never both. For sitemap inclusion, use noindex and let the URL drop from the sitemap on next rebuild.
Submitting the wrong protocol or subdomain version of the sitemap
What goes wrong: Site serves at https://www.yoursite.com but you submitted http://yoursite.com/sitemap_index.xml. GSC processes a redirect chain on every URL fetch — wastes crawl budget and logs warnings.
How to avoid: Match the canonical site URL exactly. https + www if that is your canonical. Re-submit in GSC under the correct property if needed.
Ignoring GSC sitemap error logs for weeks
What goes wrong: 10 'URL not found' errors quickly become 200+ as deleted content piles up. Real coverage issues (canonical conflicts, soft 404s) drown in the noise. You stop checking GSC because it always shows errors.
How to avoid: Weekly Monday check: GSC → Sitemaps → click sitemap_index.xml row → review errors. Fix or accept each one within 7 days of appearance.
Recap
Done — what's next
How to configure Yoast SEO with the first-time configuration wizard
Read the next tutorial
Hand it off
Sitemap submission looks like a 5-minute task and works that way on a clean install. On a site with 5+ years of history, page builders, and conversion-funnel completion URLs, sitemap cleanup is closer to 3-4 hours of focused work. A vetted technical SEO specialist audits the sitemap, excludes noise, fixes errors, and submits cleanly — typically $40-80 total at $14-16/hr.
See specialist rates
Yes. Yoast SEO → Settings → APIs → XML sitemaps → toggle OFF. Yoast stops generating /sitemap_index.xml. You can then submit a custom sitemap from another plugin (Google XML Sitemaps) or generated by a static site generator. Only do this if you have a specific reason — Yoast sitemaps are well-maintained.
Most common: pages set to noindex via the Yoast Advanced tab. Yoast respects noindex and excludes those URLs. Second most common: content types toggled off in Yoast SEO → Settings → Content Types. Third: paginated URLs (page/2/, page/3/) are not in the sitemap by default — only canonical first pages.
Yes — Yoast regenerates the sitemap on save_post hook. The new URL appears in the relevant sub-sitemap within seconds. If you use object caching (Redis, Memcached), there can be a 1-2 minute cache delay before the public sitemap reflects the change.
Yoast automatically splits sub-sitemaps at 1,000 URLs each (the limit Google honors for fast processing). The sitemap_index.xml can reference up to 50,000 sub-sitemaps with 50,000 URLs each = 2.5B URLs theoretical. Practically, sites under 1M URLs never hit limits.
Yoast does not generate dedicated video or image sitemaps in the free version. Yoast Premium includes images inline in the post sitemap (Image XML extension). For dedicated video sitemaps (large video libraries), use a specialized plugin like XML Sitemap & Google News, or build a custom sitemap.
Yoast SEO
Yoast's first-time configuration wizard is six clicks long and quietly makes 30+ decisions about how your site appears in search. Most owners rush it and ship the wrong indexing defaults, wrong schema type, and wrong social profile links. This is the slow version.
Yoast SEO
Yoast outputs JSON-LD schema by default — Organization, WebSite, BreadcrumbList, Article, WebPage. But the defaults assume a one-author blog. Multi-author sites, e-commerce, local businesses, and publishers need configuration to get the schema Google actually rewards with rich results.
Google Search Console
Submitting a sitemap is one click. Submitting the right sitemap, in the right format, with the right URLs is what separates indexed sites from invisible ones. Here's how to get this right and how to diagnose the most common errors.
Google Search Console
GSC's Indexing report shows you what's broken — in language that often hides what to actually do about it. This is the field-tested decoder: every error type, what causes it, and the specific fix that works.
Yoast SEO
Most WordPress site owners can install and configure Yoast themselves. But some scenarios — migrations, schema deep-debugging, 5+ year old sites with technical debt — burn more DIY time than the specialist would cost. This is the honest signals list.