llms.txt audit
Validate a site's /llms.txt against the AEO-readiness convention. Score completeness and detect link rot.
/v1/aeo/llms-txt-audit Fetches https://<domain>/llms.txt, parses the loose-markdown convention from llmstxt.org, and scores the file 0–100 / A–F across five criteria: file parses, H1 title present, blockquote tagline present, 10+ links, and coverage of the canonical sections (about, product, pricing, API, blog). HEAD-checks up to 50 of the linked URLs to detect link rot. Returns concrete recommendations for what to add. The llms.txt standard is genuinely new (2024+) and no incumbent SEO tool ships this audit yet.
Convention validation, section coverage scoring, link-rot detection (first 50 links), structural recommendations.
Does not crawl the site for missing pages — it only audits what's in llms.txt. If llms.txt doesn't exist (HTTP 404) the endpoint returns a 0/F score with a recommendation to create one.
Parameters
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
| domain | string | yes | — | Domain to inspect — e.g. qcrawl.com. |
Request
curl -X POST https://api.qcrawl.com/v1/aeo/llms-txt-audit \
-H "Authorization: Bearer osk_..." \
-d '{"domain": "qcrawl.com"}'Response
{
"status": "success",
"domain": "qcrawl.com",
"llms_txt_url": "https://qcrawl.com/llms.txt",
"exists": true,
"score": 90,
"grade": "A",
"title": "Qcrawl",
"tagline": "Clean structured data from any URL, in one API call.",
"section_count": 5,
"sections_present": ["About", "Product", "Pricing", "API reference", "Blog posts"],
"expected_sections_covered": ["about", "product", "pricing", "api", "blog"],
"expected_sections_missing": [],
"link_count": 47,
"link_check": {"checked": 47, "broken_count": 0, "broken_sample": [], "skipped_over_cap": 0},
"recommendations": []
}Errors
| Code | Meaning |
|---|---|
| 400 | Domain blocked by SSRF guard. |
| 500 | llms.txt fetch failed (network error or non-200/404). |