How to scrape Google search results
Two ways to scrape Google search results: build a headless browser stack yourself, or call an API that already did that work. This guide covers both, with working code in curl, Python, and Node, plus the real data that comes back.
DIY headless browser, or one API call
Google stopped serving results to plain HTTP clients. The page needs JavaScript to render, bot checks guard it, and suspicious IPs get a block page instead of results. That leaves two real routes, and the right one depends on how much plumbing you want to own.
| Build it yourself | Call an API | |
|---|---|---|
| Setup | Headless Chrome, stealth patches, a proxy account, parser code. Days before the first clean result | One HTTPS GET. Minutes |
| IPs and blocks | Your problem. Datacenter IPs hit the block page fast, so you buy and rotate residential or ISP proxies | Our problem. You never see a block page |
| Parsing | You write selectors against scrambled class names that change without notice | Stable JSON, same field names every time |
| Cost shape | Proxy and server bills arrive whether your queries worked or not | 1 credit per query that returns results. Failures and empties cost nothing |
| Maintenance | Ongoing. When Google changes markup, your pipeline goes dark until you patch it | Ours |
| Best for | Scraping is your core product and you want control of every layer | You want the data, without owning the plumbing |
What building it yourself takes
If you go DIY, here are the four layers you end up building. We know because we built all four, and they now run behind our API.
- A real browser. Google requires JavaScript to render results, so plain requests get a blank shell or a bot check. You run headless Chrome through Playwright or Puppeteer, with stealth patches so it does not announce itself.
- Clean IPs. Datacenter addresses get blocked fast. You buy residential or ISP proxies, then write rotation logic for the day an IP gets throttled anyway.
- A parser. Result markup uses scrambled class names that change without notice. Your selectors work today and return empty arrays some Tuesday next month.
- Babysitting. Each layer above breaks on its own schedule. The maintenance is the real product.
For a handful of queries a day from your own machine, a small script genuinely works. The short version looks like this:
Run that a few hundred times in an hour and you will meet the block page. At that point you start buying proxies and writing rotation logic, and the project stops being a script and starts being infrastructure.
One call in curl, Python, or Node
The other route is one HTTPS GET. Auth is your key in the x-api-key header. The browser pool, the clean IPs, and the parsing run on our side, and you are only charged when a query returns results.
Useful parameters: gl for country, hl for language, page for deeper pages, location for city-level results. The full list is in the docs, and you can run queries in the playground without writing code.
The data that comes back
Every result comes back in a stable, serper-compatible shape: ranked organic results with positions, plus knowledge graph, people-also-ask, and related searches when Google shows them. Here are the top organic rows as CSV, then the same response as JSON. Trimmed for space:
Published median for web search is about 1.2 seconds. The shape stays put, so the parser you write today keeps working.
12 search types, one endpoint
Swap type=web for any other Google surface on the same endpoint, same key, same billing. Here is a real autocomplete call we ran while writing this page:
One note on emails: Google results and Maps records do not include email addresses. Maps gives you name, address, phone, website, rating, reviews, categories, and hours. Verified emails come from our LinkedIn people search endpoints, which are separate.
Deeper dives: Google Maps scraper API · Google Reviews API · the official Google Search API, compared
One credit per query that works
Free tier is 3,000 queries a month with no card. Prepaid packs after that, and credits never expire: 25k for $49 · 100k for $149 · 500k for $549 · 2.5M for $1,999, all self-serve, ex-tax. A query that returns results costs 1 credit. A query that fails or comes back empty costs nothing.
The legality, stated plainly
Most guides either dodge this or oversell it, so here is the neutral version. US courts have repeatedly sided with scraping public, logged-out pages. Google's Terms of Service separately prohibit automated access, and a terms violation is a contract matter for whoever does the collecting. The results themselves are mostly facts: links, titles, and rankings.
What you do with the data carries its own rules. Privacy laws like GDPR and CCPA apply if you store personal data about people they cover, no matter how it was collected. Copyright applies if you republish full page content wholesale. Rankings, links, and your own analysis of them are the normal territory rank trackers and SEO tools have worked in for two decades.
Our own line: we stay on public, logged-out pages, with no fake accounts and no login walls. None of this is legal advice. For anything load-bearing, ask a lawyer.
When DIY is the better pick: scraping is your core product and you want control of every layer, you only need a handful of queries a day from your own machine, or you are doing it to learn how the plumbing works. Those are real cases. And if you are shopping hosted APIs, Serper and SerpApi are both solid products. We keep side-by-side comparisons: CrustAPI vs Serper and CrustAPI vs SerpApi.
Common questions
Clay
Run one query and read the JSON
3,000 free queries a month covers a real project. Skip the proxy shopping and see what comes back.
Get 3,000 free credits