doc-scraper MCP Server
Sriram-PR/doc-scraper
[](https://agentrank-ai.com/tool/Sriram-PR--doc-scraper/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge) Go web crawler to scrape documentation sites and convert content to clean Markdown for LLM ingestion (RAG, training data)
claude mcp add agentrank -- npx -y agentrank-mcp-server Overview
Sriram-PR/doc-scraper is a Go tool licensed under Apache-2.0. Go web crawler to scrape documentation sites and convert content to clean Markdown for LLM ingestion (RAG, training data) Topics: data-preparation, llm, web-scraper, documentation, golang-cli, mcp, mcp-server, web-crawler.
Ranked #6266 out of 24846 indexed tools.
Ecosystem
Score Breakdown
105 stars → growing community
Last commit 1mo ago → moderate activity
1/7 issues closed → many open issues
3 contributors → small team
No dependents → no downstream usage
Weights: Freshness 20% · Issue Health 20% · Dependents 22% · Stars 10% · Contributors 8% · How we score →
How to Improve
Badge all embed codes →
[](https://agentrank-ai.com/tool/Sriram-PR--doc-scraper/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge) <a href="https://agentrank-ai.com/tool/Sriram-PR--doc-scraper/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge"><img src="https://agentrank-ai.com/api/v1/badge/Sriram-PR--doc-scraper" alt="AgentRank"></a> Matched Queries
From the README
# LLM Documentation Scraper (`doc-scraper`) > A configurable, concurrent, and resumable web crawler written in Go. Specifically designed to scrape technical documentation websites, extract core content, convert it cleanly to Markdown format suitable for ingestion by Large Language Models (LLMs), and save the results locally. doc-scraper crawls documentation sites into clean Markdown and serves them to coding agents over MCP. Your agent searches and reads the docs you chose, offline, instead of guessing from training data or a live web search. Point `doc-scraper add` at any docs page and it detects the framework (30+ docs generators, including Docusaurus, Starlight, Sphinx, VitePress, Material for MkDocs, and mdBook), validates the content selector, and drafts the config for you. **Documentation: https://sriram-pr.github.io/doc-scraper/** ## Features - **Offline search for agents:** BM25 full-text search (`search_docs`) and page reads (`read_page`) over the stored corpus, with no nRead full README on GitHub →
Get the weekly AgentRank digest
Top movers, new tools, ecosystem insights — straight to your inbox.