{"product_id":"master-docs-with-ai-complete-site-crawling-markdown-conversion","title":"Master Docs with AI: Complete Site Crawling \u0026 Markdown Conversion","description":"\u003ch3\u003eMaster Docs with AI: Complete Site Crawling \u0026amp; Markdown Conversion\u003c\/h3\u003e\n\n\u003cp\u003eThis skill is designed to crawl an entire documentation or design-system website, converting it into a single, LLM-ready Markdown corpus. It discovers every page from the site's sitemap.xml, or follows links if no sitemap is available, extracts the main content in Markdown format, and downloads images locally. It also renders JavaScript-heavy pages using a headless browser.\u003c\/p\u003e\n\n\u003ch3\u003eWhat this skill does\u003c\/h3\u003e\n\u003cul\u003e\n    \u003cli\u003eCrawls entire websites, such as documentation, design-system, API reference, component library, or knowledge base sites.\u003c\/li\u003e\n    \u003cli\u003eUtilizes the site's sitemap.xml for page discovery, with a same-origin link-following fallback.\u003c\/li\u003e\n    \u003cli\u003eExtracts each page's main content into clean Markdown.\u003c\/li\u003e\n    \u003cli\u003eDownloads associated images, including inline base64 images, to a local folder.\u003c\/li\u003e\n    \u003cli\u003eHandles JavaScript-heavy pages using a headless browser for accurate content capture.\u003c\/li\u003e\n\u003c\/ul\u003e\n\n\u003ch3\u003eWho it is for\u003c\/h3\u003e\n\u003cul\u003e\n    \u003cli\u003eDevelopers and teams using AI coding agents such as Claude Code, Cursor, and Codex.\u003c\/li\u003e\n    \u003cli\u003eProfessionals involved in documentation archiving, research, or conversion for AI model training.\u003c\/li\u003e\n    \u003cli\u003eResearchers analyzing design systems or knowledge bases for contextual insights.\u003c\/li\u003e\n\u003c\/ul\u003e\n\n\u003ch3\u003eUse cases\u003c\/h3\u003e\n\u003cul\u003e\n    \u003cli\u003eCrawling a multi-page documentation site to create a comprehensive Markdown repository.\u003c\/li\u003e\n    \u003cli\u003eConverting a design-system or API reference site into a format suitable for ingestion by large language models (LLMs).\u003c\/li\u003e\n    \u003cli\u003eArchiving an entire component library or knowledge base for offline use or research.\u003c\/li\u003e\n\u003c\/ul\u003e\n\n\u003ch3\u003eTechnical details\u003c\/h3\u003e\n\u003cul\u003e\n    \u003cli\u003eEmploys \u003ccode\u003eagent-skills\u003c\/code\u003e for page crawling and Markdown conversion.\u003c\/li\u003e\n    \u003cli\u003eCapable of rendering complex sites using a headless browser to ensure accurate page content capture.\u003c\/li\u003e\n    \u003cli\u003eOutputs clean, LLM-compatible Markdown with relative image paths for local referencing.\u003c\/li\u003e\n\u003c\/ul\u003e\n\n\u003cp\u003eNote: This skill targets publicly available information only. Users must comply with the target site's terms of service and it does not bypass any authentication mechanisms or CAPTCHA.\u003c\/p\u003e\n\u003c!-- mcpcart:static-blocks:start --\u003e\n\u003chr\u003e\n\u003ch3\u003eSource \u0026amp; Licence\u003c\/h3\u003e\n\u003cp\u003eThis package is built on open-source work published by \u003cstrong\u003eCaesiumY\u003c\/strong\u003e (\u003ca href=\"https:\/\/github.com\/CaesiumY\/ko-design-md\" rel=\"nofollow noopener\" target=\"_blank\"\u003eCaesiumY\/ko-design-md\u003c\/a\u003e) and distributed under \u003cstrong\u003eMIT\u003c\/strong\u003e. The original licence text and copyright notice are included in your download.\u003c\/p\u003e\n\u003cp\u003ePersonal and commercial use, modification and redistribution are permitted, provided the original copyright and licence notice are retained.\u003c\/p\u003e\n\u003cp\u003eYour purchase covers curation, licence verification, packaging, documentation and instant delivery. It does not grant exclusive rights to the underlying open-source code, which remains available under its original licence.\u003c\/p\u003e\n\u003ch3\u003eDelivery \u0026amp; Support\u003c\/h3\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cstrong\u003eDelivery:\u003c\/strong\u003e instant — a secure download link is emailed to you as soon as payment is confirmed.\u003c\/li\u003e\n\u003cli\u003e\n\u003cstrong\u003eFormat:\u003c\/strong\u003e ZIP archive containing the skill files, documentation and the original licence.\u003c\/li\u003e\n\u003cli\u003e\n\u003cstrong\u003eSupport:\u003c\/strong\u003e \u003ca href=\"mailto:support@mcpcart.com\"\u003esupport@mcpcart.com\u003c\/a\u003e — we aim to reply within 2 business days.\u003c\/li\u003e\n\u003cli\u003e\n\u003cstrong\u003eUpdates:\u003c\/strong\u003e updates are included only where stated on this page.\u003c\/li\u003e\n\u003c\/ul\u003e\n\u003ch3\u003eRefunds\u003c\/h3\u003e\n\u003cp\u003eThis is a digital product delivered immediately after purchase. By completing your order you request immediate delivery and acknowledge that, once the download has been accessed, the statutory right to cancel no longer applies to the extent permitted by law. Refund requests are handled in accordance with our published Refund Policy.\u003c\/p\u003e\n\u003cp style=\"font-size:0.85em;color:#666;\"\u003eClaude, Codex, Gemini and Cursor are trademarks of their respective owners. MCP Cart is an independent marketplace and is not affiliated with, endorsed by, or sponsored by any of them. Compatibility references describe interoperability only.\u003c\/p\u003e\n\u003c!-- mcpcart:static-blocks:end --\u003e","brand":"MCP Cart","offers":[{"title":"Default Title","offer_id":52960451461431,"sku":"MCP-CAESIUMY-KO-DESIGN-MD-DOCS-CRAWLER","price":23.99,"currency_code":"GBP","in_stock":true}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/0981\/3950\/4951\/files\/oVm7Gb08oTtuWt212V6bR_8c80b3ab823840119fee6d469e15df54.jpg?v=1788869278","url":"https:\/\/mcpcart.com\/products\/master-docs-with-ai-complete-site-crawling-markdown-conversion","provider":"SPF PRO","version":"1.0","type":"link"}