What do I need to scrape Google News with Python?

Python 3, the requests library, and a free CrustAPI key. Install requests with pip install requests, sign up for the key, then call the news endpoint. There is no browser to drive and no HTML to parse on your side.

Do I have to parse HTML to scrape Google News with Python?

No. The endpoint returns clean JSON, so you read data['news'] and each article already carries the title, source, link, date, snippet, image, and position. Nothing breaks when Google changes its page layout.

Is there a free way to scrape Google News with Python?

CrustAPI includes 3,000 free credits every month with no credit card, and credits never expire. That covers a real news feed you can call from a Python script for monitoring or research.

How to scrape Google News with Python

Here is how to scrape Google News with Python: send one request to the API and get clean JSON back with the headline, source, date, link, and hero image for every article. Want a spreadsheet instead? The same rows are one click away as CSV.

3,000 free / month, no card Pure Python, requests only ~1 second responses

STEP 1 - MAKE THE REQUEST

One request, real response

Install the requests library with pip install requests, grab a free API key from the dashboard, then call the news endpoint. There is no browser to drive and no HTML to parse on your side. Here is the whole script:

# pip install requests import requests resp = requests.get( "https://crustapi.com/v1/search", params={"type": "news", "q": "openai"}, headers={"x-api-key": "YOUR_KEY"}, ) articles = resp.json()["news"] for a in articles: print(a["position"], a["source"], a["title"])

Prefer the command line to check it first? The same call as curl:

# one credit per successful search, however many articles come back; empty searches cost nothing curl "https://crustapi.com/v1/search?type=news&q=openai" -H "x-api-key: YOUR_KEY"

The articles come back as clean JSON, and the same rows are available as CSV. Here are the first two, so they drop straight into Sheets or a CRM:

title,source,date,link "China delivers a one-two punch to America's AI dominance","The Verge","8 hours ago","https://www.theverge.com/ai-artificial-intelligence/967781/..." "An OpenAI exec's comments on China's Kimi K3 kicked off a big US tech debate","Business Insider","14 hours ago","https://www.businessinsider.com/open-source-ai-china-kimi..."

And the same response as JSON, which is what resp.json() hands your script:

{ "searchParameters": { "q": "openai", "gl": "us", "hl": "en", "type": "news" }, "news": [ { "title": "China delivers a one-two punch to America's AI dominance", "source": "The Verge", "date": "8 hours ago", "position": 1 }, { "title": "An OpenAI exec's comments on China's Kimi K3 kicked off a big US tech debate", "source": "Business Insider", "date": "14 hours ago", "position": 2 } ] }

Add gl and hl for country and language, and page with num for more results.

STEP 2 - READ EACH ARTICLE

The full record, ready to use

Each item in data["news"] is a plain dictionary, so you read the fields directly, no HTML parsing and nothing that breaks when Google changes its page. Here is the top article, shown as CSV, ready for Sheets or a database:

title,source,link,date,imageUrl,position "China delivers a one-two punch to America's AI dominance","The Verge","https://www.theverge.com/ai-artificial-intelligence/967781/...","8 hours ago","https://platform.theverge.com/.../gettyimages-2286280034.jpg",1

Or the same record as JSON, exactly the shape your Python code loops over:

// the top article in full { "title": "China delivers a one-two punch to America's AI dominance", "link": "https://www.theverge.com/ai-artificial-intelligence/967781/chinese-ai-models-open-source-moonshot-kimi-k3-alibaba-qwen", "snippet": "", "date": "8 hours ago", "source": "The Verge", "imageUrl": "https://platform.theverge.com/wp-content/uploads/sites/2/2026/07/gettyimages-2286280034.jpg?...", "position": 1 }

To move it into pandas for analysis, one line does it: pandas.DataFrame(data["news"]) gives you a table with a column for every field above.

HOW IT COMPARES

vs serper and serpapi

All three return Google News as JSON you can call from Python. The difference is what each result carries and how you pay for it.

	Other News APIs	CrustAPI News
Billing unit	One credit per search, even when it returns nothing	One credit per successful search; empty or failed searches cost nothing
What it returns	JSON	Clean JSON from the API, or the same rows as CSV
Article image	Small gstatic thumbnail	The real publisher hero image (og:image)
Python setup	requests, or a vendor SDK	requests and one header, no SDK
Free tier	Trial credits that expire	3,000 credits / month, no card
Credits expiry	Expire monthly or in 6 months	Never expire
Storing the data	Varies by provider	No storage limits. Keep it, export it, build on it

PRICING

One credit per search

Start free with 3,000 credits a month, no card. After that, prepaid packs, and credits never expire: 25k for $49 · 100k for $149 · 500k for $549 · 2.5M for $1,999 · up to 250M, all self-serve, ex-tax. One credit per successful search, however many articles come back, and a search that returns none costs nothing.

Run your first script free

3,000 credits a month. No card, no contract.

Get a key →

FAIR PLAY

When a licensed news feed is the better pick: you need full article bodies with a publisher license, guaranteed historical archives going back years, or a contract that indemnifies redistribution. Google News gives you the headline, source, snippet, and link, which is what most monitoring and research scripts need, but it is not a licensed full-text archive.

BEFORE YOU ASK

Common questions

WORKS WITH YOUR STACK

LangChain n8n MCP

Clay

Try it on a topic you track

3,000 free credits covers a real news feed. Paste the script above, swap in your key, and look at the JSON that comes back.

Get 3,000 free credits

No credit card required