SEO audit at scale.

What the audit checks

• Orphan pages: pages with no incoming internal links.
• Broken links: URLs returning 4xx or 5xx status codes.
• Crawler policy: robots.txt and meta tag directives.
• Weekly diff: changes between two crawls.

Endpoints

GET https://api.crawlcrawl.com/v1/crawls/{id}/orphans
GET https://api.crawlcrawl.com/v1/crawls/{id}/links
GET https://api.crawlcrawl.com/v1/crawls/{old}/diff/{new}
GET https://api.crawlcrawl.com/v1/robots-policy?url=https://example.com

{
  "orphans": [
    {
      "url": "https://example.com/orphan-page",
      "title": "Orphan Page"
    }
  ]
}

Automate on cron

Schedule weekly crawls and receive audit results via webhook.

POST https://api.crawlcrawl.com/v1/crawls
Authorization: Bearer crk_...
{
  "url": "https://example.com",
  "cron": "0 6 * * 1",
  "webhook_url": "https://your-stack.com/webhook"
}

Compare results against the previous run using /diff.

For SEO agencies

One API key per client project.

• Usage is isolated per project.
• Each client gets their own monitors and audit reports.