crawlability

Here are 4 public repositories matching this topic...

naphiertech / search-engine-launch

Universal Agent Skill for safely preparing, auditing, launching, and troubleshooting any public website for search-engine discovery, crawlability, indexing, Schema.org site identity, Google Search Console, and Bing Webmaster Tools.

search-engine sitemap seo indexing schema-org robots-txt json-ld structured-data google-search-console agent-skills bing-webmaster-tools crawlability

Updated Aug 26, 2026
Python

Wolfnicos / openseo-lens

Star

AI Search Readiness & Web Discoverability Toolkit — Analyze how well websites are prepared for AI-powered search engines

python cli open-source seo geo schema-org ai-search nlnet generative-engine-optimization tdm-reservation crawlability eu-dsm-directive

Updated Apr 26, 2026
Python

nimajafari / faceted-nav-seo

Star

A framework-agnostic reference for handling faceted navigation without wrecking crawl budget: a five-tier classifier (index / canonicalize / noindex / robots-block) driven by one facet registry, with sitemap, robots.txt, and parameter normalization.

python sitemap seo indexing robots-txt url-parameters noindex canonical-urls technical-seo ecommerce-seo crawlability crawl-budget faceted-navigation

Updated May 24, 2026
Python

justhandledlabs / robots-sitemap-validator

Star

Local-only robots.txt and sitemap.xml validation for AI coding agents

sitemap seo robots-txt skill-md ai-agent-skills openclaw-skill crawlability

Updated Jul 28, 2026
Python

Add this topic to your repo

To associate your repository with the crawlability topic, visit your repo's landing page and select "manage topics."

Learn more