Crawl4AI is the leading open-source LLM-friendly web crawler and scraper — async, blazing fast, and outputs clean markdown ready for RAG and agent pipelines.