Web scraping speed benchmark

How fast is scraping a page, really? Curl is fastest because it does nothing — raw HTML, no render. Fastcrawl does the full job (fetch, render, convert to markdown) and still lands in a similar range on static pages, with the real win on anything JavaScript-heavy. This is a transparent snapshot, method below, so you can judge it yourself.

Median scrape times (seconds)

Reading the numbers

curl is fastest because it returns raw HTML with the JavaScript never run — so on a React SPA it gives you an empty shell. crawl4ai is a fast local scraper (you run the browser and the proxies). fastcrawl is a hosted round trip — edge, render, markdown — with zero infrastructure on your side.

The real differentiator shows on JavaScript-heavy sites: Fastcrawl boots an actual browser for pages that need it, so a React app returns real content rather than an empty shell. And on repeat checks of the same URL, the Redis cache serves the result in ~0.1s.

Methodology

Three runs per tool per URL, median reported. Fastcrawl figures use maxAge: 0 to force a fresh render (never a cached result); curl uses the standard time_totaltiming variable; crawl4ai runs in a local Chromium via a single browser session. Captured 2026-08-16.