Non-HTTP Protocol Links Test Page

This page contains various non-HTTP(S) protocol links that should NOT be crawled as regular URLs. These links use special URI schemes like mailto:, tel:, javascript:, and data:.

Email Links (mailto:)

• Email to info@example.com
• Email with subject line
• Email with subject and body

Telephone Links (tel:)

• Call +1-555-123-4567
• Call UK number
• Emergency number

Other Protocol Links

• JavaScript void link
• JavaScript alert link
• Data URI link
• FTP link
• SMS link
• WhatsApp link
• Skype call link
• Slack channel link

What should happen?

• These links should NOT be crawled as relative URLs
• The crawler should ignore all non-HTTP(S) protocol links
• No broken link or 404 errors should be reported for these links
• The fix prevents mailto:info@example.com from being incorrectly resolved as /issues/non-http-protocols/info@example.com

Expected Detection:

No issues should be detected - all these protocol links should be filtered out during the link extraction phase.

Code location: apps/worker/src/crawler.ts:1604-1643 (extractLinks method)

The fix uses getAttribute('href') to get raw attribute values and filters out any protocols that aren't http: or https: before URL normalization.

Test Verification:

To verify the fix is working correctly:

1. Run a crawl on http://localhost:3030
2. Check that this page is crawled successfully
3. Verify that NO link_targets or link_relationships are created for the non-HTTP(S) protocol links above
4. Check that no skipped_urls entries exist with reasons like "invalid URL" or "relative path resolution failed" for these protocol links

Normal HTTP Links (control group):

These should be crawled normally:

• Home page (relative link)
• External HTTPS link
• Test matrix (relative link)