Technical SEO audit across 9 categories: crawlability, indexability, security, URL structure, mobile, Core Web Vitals, structured data, JavaScript rendering, and IndexNow protocol. Use when user says "technical SEO", "crawl issues", "robots.txt", "Core Web Vitals", "site speed", or "security headers".
Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.
What the file says
# Technical SEO Audit
## Categories
### 1. Crawlability
- robots.txt: exists, valid, not blocking important resources
- XML sitemap: exists, referenced in robots.txt, valid format
- Noindex tags: intentional vs accidental
- Crawl depth: important pages within 3 clicks of homepage
- JavaScript rendering: check if critical content requires JS execution
- Crawl budget: for large sites (>10k pages), efficiency matters
#### AI Crawler Management
As of 2025-2026, AI companies actively crawl the web to train models and power AI search. Managing these crawlers via robots.txt is a critical technical SEO consideration.
**Known AI crawlers:**
| Crawler | Company | robots.txt token | Purpose |
|---------|---------|-----------------|---------|
| GPTBot | OpenAI | `GPTBot` | Model training |
| ChatGPT-User | OpenAI | `ChatGPT-User` | Real-time browsing |
| ClaudeBot | Anthropic | `ClaudeBot` | Model training |
| PerplexityBot | Perplexity | `PerplexityBot` | Search index + training |
| Bytespider | ByteDance | `Bytespider` | Model training |
| Google-Extended | Google | `Google-Extended` | Gemini training (NOT search) |
| CCBot | Common Crawl | `CCBot` | Open dataset |
…
The line at the top of this page pins the label v1.7.0, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.
GET https://markdownregistry.com/api/v1/artifacts/art_cun6lwowdhsbrkho
GET https://markdownregistry.com/api/v1/resolve?ref=hashgraph-online/awesome-codex-plugins/seo-technical
GET https://markdownregistry.com/api/v1/blob/d347f9093801fc9539ee90d79454218ffd8c47d5731a8ec44b755fd1c7e8f843
Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
hashgraph-online/awesome-codex-plugins · .agents/skills/list-my-plugin/SKILL.md · Add a Codex plugin to the awesome-codex-plugins README as one correctly placed line, check it locally, and finish with…
hashgraph-online/awesome-codex-plugins · plugins/321zz321/computefirst-skill/skills/computefirst/SKILL.md · Use ComputeFirst for exact or deterministic computation when model-side arithmetic or manual transformation risks being…
hashgraph-online/awesome-codex-plugins · plugins/6Delta9/task-scheduler-codex-plugin/skills/task-planner/SKILL.md · Turn a list of goals, tasks, deadlines, and working constraints into a practical schedule with next actions and…
hashgraph-online/awesome-codex-plugins · plugins/6jeffr3y/burpsuite-mcp-bridge/skills/use-burpsuite-mcp-bridge/SKILL.md · Operate BurpSuite MCP Bridge for professional, authorized web testing. Use when Codex needs to inspect Burp…
rampstackco/claude-skills · skills/seo-technical/SKILL.md · Run a comprehensive technical SEO audit covering crawlability, indexability, rendering, site architecture, structured…
fusengine/agents · plugins/seo/skills/seo-technical/SKILL.md · Use when auditing technical SEO — robots.txt, sitemap.xml, Core Web Vitals, crawlability, indexability.
maudeunfledged834/startup-founder-skills · skills/seo-technical/SKILL.md · When the user needs a technical SEO audit, on-page optimization guidance, or help resolving crawlability, indexation…