Loading tutorials…
Loading tutorials…
If your site uses React, Vue, Next.js without SSR, or any client-side framework, the default Screaming Frog crawl shows you a skeleton — not what Googlebot sees. JS rendering fixes that, but only if configured correctly.
Who this is forTechnical SEOs auditing JS-heavy sites — SPAs, Next.js with ISR misconfigured, Vue/Nuxt without SSR, headless commerce setups. If your default crawl returns near-empty meta tags or zero word count, you need this.
What you'll need
Step 1
Spot-check 5 representative URLs in "View Source." If content is in the HTML, you don't need it. If View Source is empty, you do.
Open 5 representative pages on your site in a browser. Right-click → View Source (NOT 'Inspect' — Inspect shows the post-JS DOM, View Source shows the pre-JS HTML).
Search for the page's main heading and a chunk of body copy. If they're in the View Source HTML, you have server-rendered content and DON'T need JS rendering.
If View Source shows only `<div id="root"></div>` and a bundle of JavaScript, content is client-rendered and you DO need JS rendering.
Mixed cases: some pages SSR'd, others CSR'd. Common in Next.js apps where /blog is statically generated but /app is client-rendered. Use Screaming Frog's JS rendering only on the CSR paths via include rules, not site-wide.
If unsure, run a small JS-rendered crawl (50 URLs) and compare to the same URLs without JS rendering. If word counts, titles, and meta differ significantly, you need JS rendering.
Step 2
Configuration → Spider → Rendering → JavaScript. Default is Chromium headless (the right choice for most sites).
Configuration → Spider → Rendering. Change the dropdown from 'Text Only' (default) to 'JavaScript.'
Once enabled, Screaming Frog loads each URL in a headless Chromium instance, waits for JS to execute, then extracts the rendered HTML.
Chromium is the default renderer and matches Googlebot's current rendering engine. Don't change this unless you have a specific reason.
Note: enabling JS rendering doubles or triples crawl time per URL. A 5K-URL site that crawls in 30 minutes without JS rendering will take 90-180 minutes with it.
Step 3
Configuration → Spider → Rendering. Set AJAX Timeout to 5s default, 10s for slow sites. Window size to 1366×768.
AJAX Timeout: how long Screaming Frog waits after page load before extracting. Default is 5 seconds. Raise to 8-10 for slow sites (multiple API calls, heavy hydration).
Setting this too low means SF extracts before client-side content loads → empty word counts, missing meta. Setting too high means each URL takes 10+ seconds even on fast sites → crawl time balloons.
Window Size: 1366×768 is the default and matches Googlebot's mobile crawler since 2019. Larger window sizes can affect responsive layouts and trigger different content rendering.
User-Agent should match the rendering mode: 'Googlebot (Smartphone)' for mobile-first indexing, which is now the default for Google.
Enable 'Flatten iframes' if your site uses iframes for embedded content you want crawled — otherwise iframe content is ignored.
Step 4
JS rendering = triple memory load. Allocate at least 8 GB heap; switch to Database Storage for any site over 25K URLs.
Configuration → System → Memory Allocation. JS rendering loads a full Chromium instance per concurrent thread, plus the rendered DOM for each crawled URL. 2 GB heap will crash within 500 URLs.
Bump to 8 GB minimum for sites under 25K URLs. 12-16 GB for larger sites. Match this to your machine's available RAM (don't allocate more than 50% of physical RAM).
Switch to Database Storage (Configuration → System → Storage Mode → Database Storage) for any JS-rendered crawl over 25K URLs. Memory mode WILL crash; database mode handles million-URL JS crawls reliably (slowly).
Restart the app after every memory or storage change. The crawl will use the new settings only after restart.
Step 5
Don't run a full JS-rendered crawl until you've validated on 50-100 URLs. Compare word counts, titles, meta against the live site.
Before launching a full crawl, run a List-mode crawl of 50-100 representative URLs with JS rendering enabled. This tests config without burning hours.
After completion, spot-check 10 URLs: open the URL in a browser, compare the rendered DOM (Inspect) against what Screaming Frog extracted.
Look for: word count, title tag, meta description, H1, canonical, structured data. All should match the rendered browser state.
If word counts are 10-20% lower than browser, your AJAX timeout is too short — raise it.
If titles or meta are missing in SF but present in the browser, your renderer isn't waiting for the JS that injects them. Raise AJAX timeout and re-test.
Only after validation succeeds on the sample should you launch the full crawl. JS-rendered crawls of large sites take hours; you don't want to discover a config issue 8 hours in.
Step 6
Run both modes on the same site. Compare URL counts, word counts, link counts. The deltas reveal what Googlebot sees vs. doesn't.
Run a text-only crawl (JS rendering OFF) on the same URL set.
Run the same crawl with JS rendering ON.
Export both Internal HTML tabs. Compare side-by-side: URL count, word count per URL, internal link count, title presence.
URLs that appear in the JS crawl but not the text-only crawl are JS-discovered — they're reachable only via client-side links. Google's render queue eventually finds these, but the delay (sometimes weeks) can hurt new content.
Pages with significantly higher word count in JS mode are pages where main content is client-rendered. These are at risk if Google's rendering ever fails — consider SSR for SEO-critical content.
This comparison is the diagnostic for whether your site has a real JS rendering risk in production SEO. Run it quarterly.
Common mistakes
Enabling JS rendering on a fully SSR'd site
What goes wrong: Crawl time triples. A 1-hour crawl becomes 3 hours. You burn machine resources and audit deadlines while getting identical data to what text-only would have produced. Lost productivity: 4-12 hours per crawl cycle, compounded across audits.
How to avoid: Always check 'View Source' on 5 representative pages first. If content is in the HTML, leave JS rendering OFF.
Setting AJAX timeout to 2 seconds 'to speed things up'
What goes wrong: Pages with API calls slower than 2 seconds get extracted before content loads. Your crawl reports empty meta descriptions, missing H1s, and zero word counts on perfectly valid pages. You waste a week 'fixing' issues that don't exist.
How to avoid: Default to 5 seconds. Raise to 8-10 for sites with multiple async calls or heavy hydration. Never drop below 4 seconds unless you've validated all URLs render that fast.
Running JS rendering with 2 GB heap on a 50K-URL site
What goes wrong: Chromium instances exhaust memory around URL 800-1,500. The crawl silently slows then stops processing URLs. You end up with a partial crawl marked 'complete' and miss thousands of pages. Lost audit value: a full repeat cycle, often 12-30 hours.
How to avoid: Always allocate at least 8 GB heap for JS rendering. Switch to Database Storage for over 25K URLs. Restart the app before crawling.
Not validating with a small crawl first
What goes wrong: You launch a 100K-URL JS-rendered crawl overnight. In the morning, the crawl is 60% complete but the AJAX timeout was too short — every page extracted is missing meta and word counts. You start over, losing 8 hours and risking a missed deadline that can cost $1,500-5,000 in agency credibility.
How to avoid: Always validate with a List-mode crawl of 50-100 URLs before launching the full crawl. Spot-check the output against the live site.
Comparing JS-rendered crawl to GSC without understanding the delay
What goes wrong: Google's render queue can lag indexing by days or weeks. Your JS-rendered SF crawl might find 5K URLs that GSC still shows as 'discovered but not indexed' — you panic-fix things that are actually fine because the data isn't reconciled yet.
How to avoid: When comparing SF to GSC on a JS-heavy site, allow for 2-4 week render lag. Treat new pages discovered only via JS as 'eventually indexed' rather than 'broken now.'
Using JS rendering for a site where only some sections need it
What goes wrong: A Next.js site has /blog SSR'd and /app client-rendered. You enable JS rendering site-wide. The /blog section crawls 3x slower for no reason while /app benefits. Wasted crawl time + wasted credits.
How to avoid: Configuration → Spider → Include. Limit JS rendering to URL patterns that need it. Or run two crawls: text-only for SSR'd sections, JS-rendered for the rest.
Recap
Done — what's next
How to set up Screaming Frog and run your first crawl
Read the next tutorial
Hand it off
JS rendering audits are the most diagnostic-heavy SEO work — they require systems knowledge (what's SSR'd vs CSR'd in your stack), config tuning, and back-and-forth with engineering. A vetted technical SEO specialist on EverestX will own the audit and the coordination — typically $600-1,200/mo at $14-16/hr.
See specialist rates
Quick test: right-click any page → View Source (not Inspect). If your title, body copy, and main headings are in the HTML, you don't need JS rendering. If View Source is mostly empty and content fills in via JavaScript, you do. Mixed sites need JS rendering only on the JS-heavy paths.
Roughly 2-4x slower than text-only. A 5K-URL crawl that takes 30 minutes text-only takes 60-120 minutes with JS rendering. Database mode adds another 20-30% on top. Plan accordingly.
Some sites have client-side links that don't appear in the initial HTML. JS rendering follows those links and discovers URLs the text crawl missed. This is also why Google's render queue is important for JS-heavy sites — it surfaces URLs that aren't reachable via static crawl.
Yes — Googlebot's render queue can delay indexing of JS-only content by days or weeks. For SEO-critical content (commercial pages, hub pages), prefer SSR or static generation over client-side rendering. SF's JS-rendered crawl shows you what Google will eventually see, not what it sees immediately.
Yes, if the SPA has client-side routing that follows browser URL changes (e.g., React Router with push state). Screaming Frog follows these links during JS rendering. SPAs without proper URL handling (hash-only routes) are uncrawlable by any tool and need SSR/SSG to be indexable.
Screaming Frog SEO Spider
Screaming Frog only earns its keep when the crawl matches how Googlebot actually sees your site. This walks through the install, license activation, memory tuning, and configuration choices that 90% of first-time users get wrong.
Screaming Frog SEO Spider
Custom Extraction is the most under-used Screaming Frog feature and the closest thing to a superpower it offers. Pull prices, schema fields, review counts, hreflang variants — anything on the page, at crawl scale.
Screaming Frog SEO Spider
70% of site migrations lose traffic. Almost all of those are preventable with one disciplined pre-launch Screaming Frog crawl. This is the exact checklist that separates a clean migration from a six-month recovery.
Screaming Frog SEO Spider
You've crawled the site. You have 6,000 issues. You're not sure which 30 actually matter. This is the honest decision framework for when self-managed technical SEO becomes false economy.