AI crawlers do not execute JavaScript. If a link only exists after your framework hydrates, GPTBot, ClaudeBot and Bingbot will never follow it, and the page behind it stays out of AI answers. Server-rendered HTML links are the fix, and the cost of waiting compounds quietly.
This has been true for a while and mostly went unmeasured. Two datasets published since December 2024 now put numbers on it, and the numbers are worse than the industry assumed.
AI crawlers fetch JavaScript files without ever running them
The crawlers behind the major AI assistants download your JavaScript and treat it as text. Vercel and MERJ measured this across their network and found that none of the major AI crawlers render JavaScript, covering OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, Anthropic's ClaudeBot, Meta-ExternalAgent, ByteDance's Bytespider and PerplexityBot (Vercel and MERJ). ChatGPT's crawlers spent 11.50 percent of requests on JavaScript files and Claude's spent 23.84 percent, without executing any of it.
Google's own Gemini is the exception, because it rides on Googlebot infrastructure and inherits full rendering (Vercel and MERJ). That single exception is why so many teams think they are fine. They check Gemini, see their content, and conclude the whole category is handled. We walk through the wider version of this problem in optimizing your website for AI agents.
A 41-day experiment measured what JavaScript links cost in AI crawler discovery
Someone finally ran the controlled version. Vinicius Stanula built a site where half the internal links were hard-coded in HTML and half were injected by JavaScript, then logged every bot request for 41 days (Search Engine Land). The design isolates exactly one variable: how the link is delivered.
The result on the AI side was absolute. GPTBot, ClaudeBot, Bingbot, PerplexityBot, Meta's and Amazon's crawlers, and every other bot tracked found exactly zero of the JavaScript-linked pages (Search Engine Land). Not a reduced sample, not a slower crawl. Zero.
Googlebot renders JavaScript and still barely crawled the JavaScript-linked pages
Rendering capability and rendering budget are different things. In the same experiment, Googlebot reached 5 percent of the 748 HTML-linked pages and 2 percent of the 293 JavaScript-linked ones, while GoogleOther covered 66 percent and 48 percent respectively (Search Engine Land). GoogleOther does not build the search index. Googlebot does.
Depth made it worse: GoogleOther's JavaScript coverage fell from 67 percent at the shallowest level to 42 percent at the deepest (Search Engine Land). Google documents that Googlebot processes JavaScript in a separate rendering phase after crawling (Google Search Central), and a separate phase is a phase that can be deprioritized on a site without strong signals.
Client-side rendering hides your navigation structure from AI search
The content is rarely the problem. Most teams already server-render article bodies and product copy, then leave faceted navigation, category trees, pagination and mega-menus to the client. Those are the links that build the map, and without the map an AI crawler never learns the deeper pages exist (Vercel and MERJ).
That is why the failure is invisible in analytics. Nothing 404s, nothing errors, and the pages render perfectly in a browser. The only place it shows is an absence: sections of the site that never appear in AI answers about your category. If you are tracking answer engine visibility at all, this is the first thing to rule out before touching content.
Server-side rendering for AI search is a delivery decision
You do not need to abandon your framework. Every modern stack supports server-side rendering, static generation or incremental regeneration for the routes that matter, and Vercel's recommendation is explicitly to server-render critical content while keeping client-side rendering for enhancements like view counters and chat widgets (Vercel and MERJ).
The practical rule is narrow enough to hand to an engineer this week. Main content, meta information and navigation structure ship in the initial HTML response. Anchor tags carry a real href attribute. Anything decorative can hydrate later. The same principle governs how machine-readable your pages are for agents generally, which is the ground WebMCP covers from the other direction.
Recovery after fixing JavaScript links runs at different speeds per crawler
Fixing it works, and the crawlers do not respond in unison. After the experiment converted its JavaScript links to HTML, GPTBot re-swept within 48 hours and took 250 pages in a single pass, Bingbot caught up within a week, and ClaudeBot kept hitting the site for weeks while only recrawling pages it already knew (Search Engine Land). Google was slowest of all: by day 41, Googlebot had visited exactly one page in the converted sections.
Two things follow. The upside from a fix arrives fastest in the AI engines, which is where the discovery gap was widest to begin with. And the longer a section stays behind a JavaScript-only link, the longer some crawlers keep working from a map that never included it.
The JavaScript SEO checks worth running this week
Three checks will tell you where you stand in under an hour. Disable JavaScript in your browser and click through your primary navigation, then fetch a category page with curl and search the raw HTML for the anchor tags you expect to find. Anything missing from that raw response is missing from most AI crawlers (Vercel and MERJ).
Then read your server logs by user agent. Analytics never records the crawler that failed to arrive, so the discovery gap only shows in the raw request log. Structured data belongs in the same audit, which is why we pair this with schema markup for AI search. If you want a technical read on your own stack before the next release, we can take a look.
