How to scrape Google Search with Python
This guide shows how to scrape Google Search with Python: call one endpoint, get the organic results as clean JSON, then save them to a CSV file. No browser automation, no parsing HTML.
One request, real response
The hard part of scraping Google Search yourself is driving a browser, dodging blocks, and parsing a results page that keeps changing. Instead you send one HTTP request to a search endpoint and get structured data back. First a quick test with curl:
Then the same call in Python with the requests library. Install it with pip install requests, drop in a key, and run:
The organic list holds the ranked results in order. Here are the first two that came back, shown as CSV so they drop straight into Sheets or a spreadsheet:
Change q to any search you would type into Google, like project management tools or plumbers in miami.
The full record, ready to use
Every result carries title, link, snippet, and position, plus a date when Google shows one. The snippet is the summary text, so it feeds a RAG pipeline or a report with no extra fetch. One result as CSV, ready for a spreadsheet or a database:
Or the same result as JSON, exactly as the API returns it:
The top level of the response wraps that list with the query you sent, the People Also Ask questions, and the related searches, so you capture the whole page in one call:
vs writing your own scraper
You can build a Google Search scraper with a headless browser and a parser, and plenty of tutorials show how. The trade is maintenance. Here is the difference in practice.
| Writing it yourself | CrustAPI | |
|---|---|---|
| Setup | Install a headless browser, handle blocks and page changes | One GET request, no browser |
| Output | Parse the results page yourself; selectors break when the layout changes | Clean JSON from the API, or the same rows as CSV |
| Maintenance | Fix the parser each time Google changes the page | We keep it working |
| Free tier | Free, but you pay in time and blocked requests | 3,000 results / month, no card |
| Cost model | Server, proxy, and developer time | One credit per successful search, however many results; empty searches free |
| Storing the data | Yours to manage | No storage limits. Export to CSV or a database |
One credit per search
Prepaid packs, and credits never expire: 25k for $49 · 100k for $149 · 500k for $549 · 2.5M for $1,999 · up to 250M, all self-serve, ex-tax. Each successful search costs one credit, however many results come back; a search that returns none costs nothing.
Save the results to a CSV file
To go past the first page, add a page parameter and loop it: page 1 is the default, so set it to 2, 3, and up to walk deeper. Because the response is plain JSON, Python's built-in csv module writes every result to a file with no extra libraries:
Open results.csv in Sheets or Excel and you have a finished list of ranked results. Swap the query, run it again, and append to build a bigger dataset.
Related: Google Search API · Scrape Google search results · Serper alternative · Scrape Google Maps with Python · API docs
When a different tool fits better: a formal contract with an SLA, guaranteed uptime, or Google's own ranking APIs are Google Cloud products, not scraping. This endpoint returns the public results anyone sees on the page, which is what most SEO research, rank tracking, and RAG work needs, but it is not the official Custom Search API.
Common questions
Clay
Try it on a query you know
3,000 free credits covers a real search. Run one query and look at the JSON that comes back.
Get 3,000 free credits