Crawl Failed Test

This documents the crawl_failed issue type, which is triggered when a page returns a valid HTTP 2xx response but cannot be crawled.

Test Endpoint

The /api/204 endpoint returns HTTP 204 No Content:

• /api/204 - Returns HTTP 204 with no body

What Happens

1. Crawler attempts to navigate to the URL
2. Server responds with HTTP 204 (valid 2xx status)
3. Playwright cannot render the page (no content)
4. Navigation fails with net::ERR_ABORTED
5. Crawler captures the HTTP status from network context and creates a crawl_failed issue

Expected Detection

Checklist item: All pages can be crawled and indexed (all-pages-crawlable)

Issue type: crawl_failed

Category: Visibility

What's NOT Expected

This should NOT trigger:

• page_error issue (204 is not a server error)
• "No 5xx Errors" checklist failure (204 is a 2xx status)

Code Location

Detection logic: apps/worker/src/crawler.ts:evaluateAllPagesCrawlableCheck()

Issue type definition: packages/issuetypes/src/registry.ts:crawl_failed