Success rates, retries, and which scraping API vendors bill failures

The dataset flags 9 product records: Apify, Browserbase, Browserless, Firecrawl, Kernel, Oxylabs Web Scraper API, Oxylabs SERP Scraper API, ScrapeOwl, and Scrapfly, with all policies retrieved 2026-07-26. That strict flag is not the full answer: several “pay only for success” APIs bill target 4xx results, and Bright Data’s custom features bill 100% of requests, so the HTTP status, configuration, and whose system produced the error matter. Bright Data Web Unlocker documentation, retrieved 2026-07-26.

What success rate can this dataset support?

No measured cross-vendor success-rate figure exists in the current dataset. Across all 74 vendor records checked 2026-07-26, the dataset does not contain an observed success-rate field. That omission is intentional: this site has not run every service against the same targets, protection states, locations, request modes, and acceptance test. This is not a benchmark.

The published billing definitions also use different billing units. Firecrawl can bill a fetched target 4xx or 5xx, while Browserless meters resource use that can remain billable when the final task fails. A percentage based on billable target responses cannot be compared honestly with one based on browser sessions or application-accepted pages. The cloud-browser billing guide owns the Browserless meter arithmetic. Firecrawl pricing and Browserless unit documentation, retrieved 2026-07-26.

The worked example below therefore starts from named events in a hypothetical request log, not an assumed success percentage. For a real workload, apply the vendor’s published billing rule to your own target-specific log. Until that measurement exists, the success-rate answer is unknown rather than an advertised percentage.

The nine records that charge at least some failures

The exact policy excerpts below are deliberately short and verbatim. “Charges failures” covers two different commercial models: result APIs that classify some error responses as successful, and browser or compute platforms that bill resources consumed before an unsuccessful outcome.

The nine records that charge at least some failures table

This is why one boolean cannot tell the whole story. Apify, Browserbase, Browserless, and Kernel meter work performed or time held, not a clean-page outcome. Firecrawl, Oxylabs, and Scrapfly instead draw a boundary around which returned results count as a billable success. The operational question is not only “does it charge failures?” It is “what event closes the meter?”

Vendor-side failure versus a target returning an error

A vendor-side failure means the scraping provider did not complete its service: its proxy pool failed, its browser timed out, its internal retry limit was exhausted, or its own server errored. A target-side response means the provider reached the requested site and delivered what that site returned, even when the response was 404 Not Found, 410 Gone, 403 Forbidden, or 500 Internal Server Error.

That distinction produces policies that look contradictory until both HTTP layers are separated:

Vendor-side failure versus a target returning an error table

Do not calculate retry waste from the status code your application ultimately sees until you know whether it is the vendor’s status or the target’s status. A provider may retry three times internally, bill only the final delivered result, and hide those three attempts from your quota. Another provider may meter every second of the browser session regardless of which navigation eventually failed.

Worked example: 1,000 billable target errors add $1

This is a pricing scenario built from explicit log outcomes, not a benchmark or a claim about Oxylabs SERP Scraper API’s observed reliability. The example uses Google results on the Micro plan. These are the stored inputs, shown with the record field that holds each figure:

Worked example: 1,000 billable target errors add $1 table

Assume the application needs 49,000 accepted Google results. Its log contains 48,000 accepted original results, 1,000 target 404 results rejected by the application, and 1,000 accepted retries. Those are scenario inputs, not measured vendor results. Applying the sourced billing rule gives:

accepted original results        48,000
target 404 results                 1,000 billable
accepted retries                   1,000 billable
                                  ------
accepted application results      49,000
billable vendor results           50,000

The accepted-output count fits the included 49,000 results, but the billed-result count exceeds it by 1,000. The record supplies a fixed overage rate, so the arithmetic does not invent a tier upgrade:

overage results = 50,000 - 49,000
                = 1,000

overage cost = 1,000 × $0.001
             = $1.00

monthly total = $49 + $1.00
              = $50.00

increase versus the plan fee = ($50.00 - $49) / $49 × 100
                             = 2.040816...%
                             ≈ 2.04%

Each rejected target 404 plus its retry creates 2 billable results, but only the rejected result is extra relative to the 49,000 accepted outputs the job required. In this scenario, 1,000 such results add $1.00 to the $49 plan fee. The meter, billing boundary, allowance, and rate come from Oxylabs SERP Scraper API pricing and the Oxylabs billing guide, retrieved 2026-07-26.

Bright Data provides a contrasting success-only rule, with an important exception: enabling a listed Custom Web Unlocker feature changes billing to 100% of requests, successful and failed. The blocked-scrape billing guide owns the mixed-status invoice example. Bright Data Web Unlocker pricing and billing documentation, retrieved 2026-07-26.

Worked example: a 5% failure rate crosses a monthly plan

Assume the application needs 100,000 usable basic pages and its own target log shows that 5% of billable attempts are rejected. The 5% is a scenario input, not a Firecrawl success-rate claim. Firecrawl’s current record stores 1 credit per basic page, Standard at $99 for 100,000 credits, and Growth at $399 for 500,000 credits. Firecrawl does not publish a fixed Standard overage rate; its page describes separately billed credit packs without a public fixed pack price. Firecrawl pricing, retrieved 2026-07-26.

usable share = 1 - 0.05 = 0.95
attempts = 100,000 usable basic pages / 0.95
         = 105,263.16 attempts
whole attempts required = ceil(105,263.16) = 105,264

credits required = 105,264 attempts x 1 credit = 105,264 credits
Standard shortfall = 105,264 - 100,000 = 5,264 credits

Standard does not cover the worked attempt count. If the buyer compares only the published monthly plans because the separate pack price is not public, Growth is the first listed allowance that covers it. The monthly plan choice moves from $99 to $399, a $300 increase or ($399 - $99) / $99 x 100 = 303.03%. This is a plan-cliff result, not a smooth retry price and not proof that Growth is the only purchase route offered in an account. Firecrawl pricing, retrieved 2026-07-26.

What to ask before buying

Get written answers to these questions for the exact endpoint and configuration you will use:

1. Which outer API status codes consume quota or money?
2. Which target status codes can be carried inside an API success?
3. Are internal retries included, and does a client retry create a second billable unit?
4. Do custom headers, cookies, premium domains, browser actions, or CAPTCHA solving switch the billing model?
5. Is there a published overage rate, an automatic top-up, an early renewal, or only a plan upgrade?
6. Can a cost ceiling stop work before the next tier or pay-as-you-go balance is used?

If the vendor does not publish one of those rules, treat it as unknown. Do not turn “not published” into “free,” and do not use a vendor’s advertised success rate as though it were a measured result for your targets.