Documentation

One key. REST and MCP.

Base URL https://fastcrawl.net (or your custom domain). Auth headerAuthorization: Bearer YOUR_KEY. Failed calls are never charged.

1. Get a key

Sign up at /app. The dashboard shows your key once.

2. Scrape

curl -X POST https://fastcrawl.net/api/v1/scrape/ \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","formats":["markdown","json"],"timeout":30,"fetchMode":"auto"}'

Returns markdown, json (structured data: title, headings, paragraphs, links, images, word count), metadata, duration_ms. Add "html" to formats for raw HTML, "links" for outbound links, "changeTracking" to get a diff vs your previous scrape of the same URL (metadata.change_information: first / unchanged / added / removed / different + line diff). timeout (5-30s, default 30) fails fast on blocking pages. PDF URLs are detected automatically and parsed via document extraction — no separate call needed.

Failures are business results: 422 with an actionable error_code (target_dns_error, target_not_found, target_blocked, target_rate_limited, target_timeout, scrape_empty_content, …) and a retryable flag — retry only when it's true. Failed requests are never charged.

fetchMode: auto (default) tries HTTP first (~100ms) and escalates to a browser render only when the page needs JavaScript; http never opens a browser (~100ms, for static/SSR pages). There is no forced-render mode — auto already escalates, and forcing a render spends seconds and browser-hours on pages plain HTTP fetches in milliseconds.

3. Search + content

Search, and optionally the first results' pages, in one call — the agent loop of "search → read top N" without a second round trip. include_content (1-5) fetches that many results in order through the same pipeline as /scrape, so a bundled page is served from the same cache a direct scrape of it would use.

curl -X POST https://fastcrawl.net/api/v1/search/ \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{"query":"model context protocol server registry","limit":5,"include_content":2}'

Response (real call, markdown trimmed):

{
  "success": true,
  "query": "model context protocol server registry",
  "results": [
    { "title": "MCP Server Registry | Discover & Govern AI Agent Tools | Kong I…",
      "url": "https://konghq.com/products/mcp-registry",
      "snippet": "The Model Context Protocol (MCP) has simplified how agents connect to tools. …",
      "content": "# Kong AI Registry: Govern What AI Agents Can Discover  | Kong Inc.… (20,893 chars)",
      "content_error": null },
    { "title": "Awesome MCP Servers",
      "url": "https://mcpservers.org/",
      "snippet": "A lightweight MCP (Model Context Protocol) server for Blender. …",
      "content": "# Awesome MCP Servers … (15,537 chars)",
      "content_error": null },
    { "title": "GitHub - mapbox/mcp-server: Mapbox Model Context Protocol (MCP) …",
      "url": "https://github.com/mapbox/mcp-server",
      "snippet": "…",
      "content": null, "content_error": null }
  ]
}

Fields: content is the page as markdown for the first include_content results (rows past N keep it null), content_error is null on success. Results 1 credit; each content page 1 credit, cache hits free — a repeat of a page we already scraped inside the 48h window costs nothing. A page that fails (404, empty body, anti-bot) never fails the request: that row gets content: null and a short content_error (e.g. target_not_found, scrape_empty_content) while the call still returns 200. include_content above 5 is a 400, and the whole request has a 60s budget — rows that miss it come back with content_error: "budget exceeded". REST only: the MCP search tool stays one credit per call.

4. Screenshot & PDF

Both return a data: URI you can write straight to a file. Set viewport for mobile widths or tall full-page captures.

# PNG screenshot
curl -X POST https://fastcrawl.net/api/v1/screenshot/ \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","viewport":{"width":1280,"height":800}}'

# Rendered PDF
curl -X POST https://fastcrawl.net/api/v1/pdf/ \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'

Each returns { "success": true, "data": "data:image/png;base64,…" } (screenshot) or data:application/pdf;base64,… (pdf). 1 credit per call.

5. HTML to PNG image

Markup in, hosted PNG out — render a chart, invoice, social card or OG image from HTML+CSS with no browser on your side. This route takes html only; for a URL capture use /screenshot.

curl -X POST https://fastcrawl.net/api/v1/image/ \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{"html":"<html><body><h1>Hello from fastcrawl</h1></body></html>","css":"body{font-family:sans-serif;padding:40px} h1{color:#2563eb}","viewport_width":800,"viewport_height":600}'

Returns { "success": true, "url": "https://…/v1/image/…" } and that URL serves content-type: image/png. Fields: html (required, 1MB max), css, viewport_width (default 800), viewport_height (default 600), google_fonts. PNG only — no PDF/JPEG, and no width/height in the response because the renderer does not report them. 1 credit per image; an identical repeat within 30 minutes is a cache hit, returns "cached": true and is free. Render capacity saturation answers 429 — back off and retry, failed calls are never charged.

6. MCP

{
  "mcpServers": {
    "fastcrawl": {
      "url": "https://mcp.fastcrawl.net/mcp",
      "headers": { "Authorization": "Bearer YOUR_KEY" }
    }
  }
}

14 tools: scrape, crawl, map, search, extract, screenshot, pdf, image, parse, verify_email, monitor_create, monitor_list, monitor_run, monitor_delete.

Autonomous agents can discover this server via the A2A agent card at /.well-known/agent-card.json (no auth required).

7. Verify emails

Is this address worth sending to? Syntax and gibberish, disposable and webmail domains, MX records over DNS-over-HTTPS, then a real SMTP handshake at the mailbox (RCPT TO — no message is ever sent). One credit per address, up to 10 addresses per call, and the same key you scrape with. Failed calls are never charged.

# one address
curl -X POST https://fastcrawl.net/api/v1/email/verify \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{"email":"[email protected]"}'

# or up to 10 in one call
  -d '{"emails":["[email protected]","[email protected]"]}'

Every result carries a status plus the evidence behind it — score, mx_records, smtp_check, accept_all, disposable, webmail — so your own threshold can decide, rather than ours. valid send it · invalid remove it · risky a catch-all domain, so the mailbox may or may not exist · unknown unproven, because the large providers rate-limit the probe IP. Retry unknown, never treat it as invalid — that one rule is what keeps a list clean. A batch row is not always a verdict: skipped means the 80-second budget ran out before that address was probed and error means the probe threw — retry both, they say nothing about the address.

Checking a single address by hand? The free email checker runs the syntax, typo, disposable and MX layers in your browser with no key. The mailbox-level handshake is what the API adds.

Endpoints

Limits

Past 5,000 credits on Go: $1.00 per 1,000 extra, metered — requests are never blocked. Disable overage anytime in the dashboard (Overview) to hard-cap at 5,000. Extract is 1 credit/page. Failed requests are free.

CLI · Free tools · llms.txt · openapi.json · agent skill