What is a crawl endpoint?
A crawl endpoint is an API operation that starts from one URL, discovers linked pages, and fetches multiple pages as one managed job. Unlike a single-page scrape endpoint, its billable volume can grow after submission because the crawler decides which in-scope links to visit. (Firecrawl pricing, Diffbot crawl documentation, and ScrapeGraphAI pricing, checked 2026-07-26.)
Crawl endpoint across the APIs we track
| API | Per crawled page | Per single page | Checked |
|---|---|---|---|
| Desearch | $0.50 per 1,000 pages | not published | Oct 9, 2026 |
| Firecrawl | 1 credit per page | 1 credit per page | Oct 7, 2026 |
| ScrapeGraphAI | 1 credit per page | not published | Oct 9, 2026 |
In more detail
The vendor examples below were read on the dates given in the text; the table above holds each vendor's current figure.
A crawl changes a bill because the starting URL is not the final unit. Vendors can charge for each received page, each successfully extracted page, a crawl-start fee, the selected output mode per page, or some combination. Filters, page limits, duplicate handling, and stealth options determine how far the job expands and which meters apply. (Firecrawl pricing, Diffbot credit rules, ScrapeGraphAI pricing, and Search1API pricing, checked 2026-07-26.)
Firecrawl prices a crawl page at one credit, the same recorded meter as a basic page scrape. Firecrawl-side timeouts or server errors are not charged, but a received page that returns a target-side 4xx or 5xx response is billable. Plan credits generally do not roll over. (Firecrawl pricing, checked 2026-07-26.)
Diffbot says spidering a site to discover links costs zero credits, while each successfully extracted crawl page consumes one Extract credit. Crawl access is limited to Plus and higher plans, and Plus supports up to twenty-five active crawls. Its monthly credits do not roll over. (Diffbot credit rules, Diffbot crawl documentation, and Diffbot rate limits, checked 2026-07-26.)
ScrapeGraphAI charges two credits to start a crawl, then applies the selected scrape-format cost to every page. Optional stealth adds credits per page, and plan limits distinguish concurrent crawls from requests per minute. The public page does not state whether failed or blocked pages consume credits. (ScrapeGraphAI pricing, checked 2026-07-26.)
Search1API charges one credit per crawl page. Its deep-search workflow adds one search credit plus one credit for every result page crawled successfully, while DeepCrawl has a separate fixed per-run credit cost. Failed requests do not deduct credits under its published rule. (Search1API pricing and Search1API free API guide, checked 2026-07-26.)
Before signing up, define allowed hosts, paths, depth, page cap, query-parameter treatment, duplicate rules, output format, and stop conditions. Check whether discovery, startup, extraction, rendering, and stealth are separate meters; whether target error pages count; and how cancelled jobs are billed. Forecast the maximum in-scope pages, not merely the number of seed URLs. (Firecrawl pricing, Diffbot crawl documentation, ScrapeGraphAI pricing, and Search1API pricing, checked 2026-07-26.)