by jitsmaster
Configurable web crawler that extracts structured content from websites while respecting robots.txt rules and offering customizable settings for depth, delay, and concurrency.
Get the fastest-growing projects, useful MCP servers, and technical reads in one weekly email.