Scraping API Prices
MenuClose

Search the site

82 providers, their alternatives and head-to-heads, targets, guides and answers

Home / Cost tools / cURL converter / Python

cURL to Python converter

Paste a cURL command and get the same request in Python, using requests. It converts in your browser; nothing you paste is sent anywhere.

Converted in your browser. Nothing you paste is stored or sent, so a command with your API key in it stays on your machine.

Python, with requests
import requests

headers = {
    "Authorization": "Bearer YOUR_API_KEY",
}

response = requests.get("https://api.scraper.example/v1/scrape?url=https%3A%2F%2Fshop.example.com%2Fproducts%3Fpage%3D2&render_js=true&country=us", headers=headers)
print(response.status_code)
print(response.text)

Install first: pip install requests

Running the code

Save it as a .py file and run it with Python 3 after installing requests. requests builds the query string, sends a JSON body with the right Content-Type when it is passed as json=, and takes basic authentication as auth=(user, password).

Where requests differs from curl

requests follows redirects on its own, as curl does only with -L; pass allow_redirects=False to stop it. It waits without limit unless you give a timeout, and a scraping API that renders JavaScript can take tens of seconds, so set one in seconds.

What calls like this one cost

The example is a scraping API call with JavaScript rendering on and a country chosen. 100,000 of them a month cost from $49.00 on ScrapeBadger, and the same pages without rendering from $19.00 on BrowserCloud, the cheapest complete bills across 27 web scraping APIs this site prices from their own pages. What rendering adds on each API

The same command in another language