How many requests does pagination add to a catalogue scrape?

A 500-product catalogue is not automatically 500 billable requests: with 24 products per listing page and one detail fetch per product, it is 521 requests before retries or extra endpoints. Estimate collection pages or batches + detail pages + extra endpoint calls, then apply the vendor’s billing units per request and billable-response rule.

The count can also be lower than 500. If every required field is present on 24-product listing pages, the same catalogue needs only ceil(500 / 24) = 21 requests. Exactly 500 requests is the narrow case where 500 detail URLs are already known, every product needs one fetch, and the job makes no discovery, variant, review, stock, image, or retry calls.

The request-count formula

Use a separate count for each layer:

numbered listing pages = ceil(products to discover / products per listing page)

infinite-scroll collection calls =
  initial document call
  + background data batches needed after the initial batch

detail calls = distinct products whose required fields are absent from the listing

extra calls =
  variant calls + stock calls + review calls + other required endpoints

requests per run =
  numbered listing pages OR infinite-scroll collection calls
  + detail calls
  + extra calls

Do not add numbered pages and infinite-scroll batches unless the target actually uses both. They are usually alternative collection paths. Count the initial document once if it supplies the first product batch.

Here is how the same assumed 500-product job changes with scope:

The request-count formula table

The figures in this table are scenario assumptions, not observations about a named store. Replace 24 or 20 with the target’s observed page or batch size.

Why listing pages, detail pages, and infinite scroll multiply differently

Listing pages multiply discovery. A catalogue split into numbered pages needs one collection request per page. The final partial page still consumes a request, which is why the calculation rounds up.

Detail pages multiply enrichment. A listing card may be enough for a title and displayed price but not for the fields in your specification. Count one detail request only for each distinct product that needs detail-only data. If category pages overlap, deduplicate product identifiers or official URLs before assigning detail work.

Infinite scroll hides pagination in the Network panel. Record the initial document request, then record each response that adds another product batch. A cursor, offset, or page parameter may replace the visible page number, but the batch still belongs in the request count. If the target does not publish a total, write that the total is not published and keep a range until a terminating response or repeated cursor establishes the end.

Product subresources create another multiplier. A separate variant, availability, seller, review, or media endpoint can turn one product into several calls. Add only endpoints required by the output specification. Do not count every browser asset if the scraping API returns the needed document without separately billing those assets.

Worked example with current vendor rates

Assume one run with 500 distinct products, 24 products per numbered listing page, 500 required detail pages, and no extra endpoints:

listing requests = ceil(500 / 24) = 21
detail requests = 500
total requests = 21 + 500 = 521

Pagination adds 21 collection calls to the 500 detail calls. That is a 521 / 500 = 1.042× request multiplier, or a 4.2% increase over fetching the known detail URLs alone. Four refreshes make the monthly shape explicit:

listing requests = 21 × 4 = 84
detail requests = 500 × 4 = 2,000
monthly requests = 521 × 4 = 2,084

The next table maps every published input to the exact dataset field that holds it. For monthly credit plans, “allowance value” is only plan price × share of allowance consumed. For packs, the purchase floor rounds up to a whole pack. Neither calculation claims that a vendor sells fractional plans or packs.

Worked example with current vendor rates table

ScrapeLens makes the pagination boundary visible. If the 500 detail URLs were already known, 500 × 20 = 10,000 credits would fit exactly one $1 pack. Discovering those URLs through 21 listing pages adds 21 × 20 = 420 credits, taking the run to 10,420 credits and two whole packs. Pagination adds only $0.042 of credit value here, but it adds $1 to the purchase floor because the detail-only case lands exactly on a pack boundary. This is a boundary effect, not a claim that pagination usually doubles the purchase floor. (Pack and request pricing, retrieved July 26, 2026.)

The comparison therefore answers two different buying questions. The request or allowance value shows how much capacity the run consumes. The purchase floor shows what must actually be paid to start the plan or buy enough whole packs. The example is plan arithmetic, not a benchmark or a forecast.

Request modes can multiply every paginated call

After counting calls, apply the exact request-mode meter. ScrapeOwl publishes mutually exclusive totals of 1 credit for a basic request, 5 for JavaScript rendering, 10 for a residential-proxy request, and 25 for a rendered premium-proxy request. (Request costs, retrieved July 26, 2026.) ScrapingBot publishes 1 credit for basic, 5 for JavaScript, and 75 for stealth; AI extraction adds 5 credits to the selected request mode. (Request costs, retrieved July 26, 2026.)

Applied to the same 521-call shape:

Request modes can multiply every paginated call table

ScrapeOwl’s four values are replacement bundles, so they are not added together. ScrapingBot’s AI extraction meter is additive, which is why the last row adds 5 to the 1-credit basic request. (ScrapeOwl request costs and ScrapingBot request costs, retrieved July 26, 2026.) Credits should not be compared as if one credit equalled one request or one dollar across vendors.

How to estimate page count before choosing a plan

1. Write the output fields first. Mark each field as available on the listing, available only on the detail page, or available through another endpoint. This determines whether 500 products mean 21 listing requests, 521 listing-plus-detail requests, or more.

2. Observe the collection unit. For numbered pagination, record the product count on a full page and use the last page or displayed result total when either is present. For infinite scroll, capture the initial batch size and the size of successive product-bearing responses.

3. Deduplicate before counting details. Collect stable product identifiers or official URLs from the listing layer. The detail count should follow the distinct set, not the sum of products shown across overlapping categories or filters.

4. State unknown totals as unknown. If the target publishes neither a total nor a terminal page, do not turn a short sample into a precise catalogue size. Keep a lower and upper case, or run a bounded discovery pass and record the stopping condition.

5. Separate attempts from vendor-billable responses. ScrapingFish charges HTTP 200 and 404 responses. ScrapeOwl also charges 200 and 404, while its documented 400, 401, 403, 429, and 500 responses do not consume credits. Oxylabs counts 2xx and 4xx responses as successful, while its 5xx and 6xx system errors are not charged. (ScrapingFish request rules, ScrapeOwl request rules, and Oxylabs pricing, retrieved July 26, 2026.) Use each vendor’s definition instead of adding an invented retry percentage.

6. Multiply by refresh frequency. A 521-request run repeated 4 times is 2,084 outbound requests before target or feature meters. Apply the relevant credits, results, or dollar rate to each request class, then compare the monthly total with the plan allowance and purchase floor.

7. Keep a high case for mode changes. If some pages may require rendering, residential routing, or a target-specific endpoint, calculate that class separately with the published rate. When the vendor does not publish how features combine, label the combined cost unknown and verify it before committing money.

The final worksheet can stay small:

collection requests per run:
detail requests per run:
extra endpoint requests per run:
total outbound requests per run:
refreshes per month:
vendor units for each request class:
monthly vendor units:
smallest plan or pack that covers those units:
unpublished combination or failure-billing rules:

Pagination is hidden only when the buying calculation begins with “number of products.” Begin with the collection path, add the pages needed to enrich each distinct product, and only then translate the resulting calls into the vendor’s own meter.