How to Scrape Google Trends in Python (API, Scraper, PyTrends)

Google Trends shows how interest in a search query moves over time and across regions, and pulling that data out at scale means automating it. There’s no official Google Trends API, so this guide covers the four working routes in Python. A no-code Google Trends scraper, a third-party Trends API, the PyTrends library, and your own Selenium scraper, ordered from zero code to full control.

How Google Trends Data Works

Google Trends never shows search counts. Every number is an index from 0 to 100. Google’s FAQ describes the normalization behind it. Each data point is divided by the total searches of its geography and time range, and the results are scaled onto that 0 to 100 range. On any one chart the peak reads 100 and everything else is relative to it.

These are the consequences that matter for scraping.

1. Numbers are relative to the request. A 50 for “coffee” queried alone and a 50 for “coffee” queried against “tea” are different numbers. To compare terms, they must travel in the same request, and the comparison caps at five terms.
2. Values below the threshold read as 0. Low-volume regions and terms show zero even when real searches exist, so an empty-looking region is “below the cutoff”, which is different from “nobody searches this”.
3. The data is sampled. Two identical requests can return slightly different curves, and Google’s own FAQ says the statistical noise behind that is most noticeable on queries with little or no search interest, which is worth remembering before treating a two-point difference as a trend.

So a scraped Trends dataset answers “what is rising against what” rather than “how many people searched”, and every method below inherits that.

Practical Applications of Google Trends Data

Google Trends offers a wealth of data with diverse applications across various fields. Before delving into automated data collection methods, let’s explore the potential uses and applications of Google Trends data.

Market Research

One of the most common ways to use Google Trends data is for market research. Google Trends offers valuable insights into search query volumes across different regions and time periods.

Analyzing search trends over time can reveal valuable information about the fluctuating interest in a product or service. This information can be used to strategically plan marketing campaigns during periods of peak interest. Additionally, Google Trends allows for comparative analysis of brand popularity, enabling businesses to identify industry leaders and potential areas of growth.

Regional search data also shows geographical variations in demand. This can be influenced by factors beyond product awareness and seasonality, such as regional preferences and cultural trends. By understanding these variations, businesses can optimize their marketing strategies to effectively target specific regions.

SEO and Content Strategy

Scraping Google Trends data can be an invaluable addition to your SEO strategy. Here are some ways you can use it to improve your website’s search rankings and organic traffic:

• Identify trending keywords. By analyzing Google Trends data, you can uncover keywords that are gaining popularity in your niche. Incorporating these keywords into your content can help you attract more relevant traffic to your site.
• Optimize content timing. Google Trends can reveal when certain topics are spiking in interest. Use this information to schedule the publication of your blog posts, articles, and other content to coincide with these peaks in search demand.
• Research competitors. Conduct competitive analysis using Google Trends to identify gaps in your content strategy. See which keywords and topics your competitors are ranking for that you’re not, and create content to target those areas.

So, by using Google Trends data you can attract more organic traffic to your website.

Other Use Cases

The applications of Google Trends data scraped from the web are virtually limitless. Regardless of your industry, you can find the most suitable application for the data you collect, from financial research and sentiment analysis to competitor tracking and forecasting future trends based on the data you collect.

In any case, Google Trends data provides a wide range of opportunities for analysis and finding ideas and solutions in various fields, making it a valuable tool for business, research, and planning. And scraping this data allows you to get the most up-to-date data automatically.

Choosing a Web Scraping Method

Four routes reach this data, ordered here by how much code each one asks you to own, from none at all to a scraper you build yourself.

Option 1. Using a Google Trends no-code Scraper

Even without coding skills, you can get Google Trends data through services that run the collection for you and hand back ready datasets.

This method takes minutes to set up, and it offers the least flexibility. You cannot customize the scraper or modify the data format. Since these services provide a complete and packaged solution, you are limited to the features they offer.

Therefore, this approach is recommended if you need data quickly, do not require ongoing data collection, and are satisfied with the filter options and data format provided by the chosen Google Trends no-code scraper.

Option 2. Using a Web Scraping API

In addition to no-code scrapers and web scraping libraries, another option for accessing Google Trends data is to use a Google Trends API. While Google Trends doesn’t offer an official API, there are several third-party APIs available that can provide you with Google Trends data.

A Google Trends API brings four things:

1. Simplifies the data scraping process. APIs provide a structured and organized way to access Google Trends data, eliminating the need for complex web scraping techniques.
2. Enhances flexibility. APIs offer more flexibility compared to no-code scrapers, allowing you to customize your data requests and retrieve specific data points.
3. Reduces development time. The scraping and parsing side already exists, so you write only the requests and the handling of the results.
4. Carries the infrastructure. The proxy pool and the page rendering sit on the vendor’s side rather than in your code.

Overall, a Google Trends API fits developers who want the data without owning a scraper.

Option 3. Using a Web Scraping Library

The next most difficult method is to use specialized libraries to scrape Google Trends data. The most popular among them is PyTrends, and its state matters before you build on it. The last release is 4.9.2 from April 2023, so nothing in the library has been updated for anything Google has changed since. It still returns data, and a request for a year of weekly interest values comes back populated, but the maintenance is on you.

Among the unconditional advantages of this library is the possibility of obtaining data quite simply (compared to the last option), but you will have to solve the problems yourself in case of blocking or captcha. In addition, for its normal use, you will definitely need proxies.

This option is suitable for those who are already quite good at programming and are ready to face difficulties, but do not want to write their scraper from scratch and are looking for a library that could help in the process of obtaining data from the Google Trends page.

Option 4. Using a Headless Browser

Using headless browser libraries to create your Google Trends scraper is the most complex of the four. It assumes you’re comfortable handling all the steps involved in gathering and processing the necessary data, requiring strong programming skills.

If you’re unfamiliar with such libraries but interested in learning, check out our article on headless browser scraping. If you’re confident in your skills and want to build your Google Trends scraping tool, this article will also provide guidance.

In summary, this is the most challenging approach but offers the most customization, allowing you to extract any data you need.

The four options in one view:

Each route gets its own section below, end to end.

Prerequisites

Node.js and Python are the two most popular programming languages for web scraping. However, Python is generally considered easier to learn and use, even for beginners. Therefore, we will be using Python in this tutorial.

To follow along with the examples in this tutorial, you will need the following:

• Python 3.10 or higher
• A Python IDE or any code editor with syntax highlighting.

If you are new to Python programming, you can refer to our introductory article, which covers the installation process and how to create your first web scraper.

In addition to Python, we will also need to install the following libraries:

pip install requests pytrends selenium

For full functionality with Selenium, you may also need additional files, such as a web driver (only necessary for older versions of Selenium). You can find more information in our guide to web scraping with Selenium.

Using no-code Google Trends Scrapers

As promised, we’ll start with the simplest data scraping methods that don’t even require coding skills, and then move on to more complex and advanced ones. Therefore, let’s begin with the easiest method and show you how to use ready-made tools to get Google Trends data.

For this example, we’ll use HasData’s Google Trends no-code scraper. To begin, you’ll need to sign up on our website. Once registered, log in and navigate to the no-code scrapers marketplace.

Find and navigate to the page of a Google Trends no-code scraper.

Let’s break down the elements on this page:

1. Keywords. Enter your target keywords here, one per line.
2. Geo. Specify the region you want to target.
3. Timeframe. Set the date range for your search results.
4. Run Scraper. Once the parameters are set, click this button to start the scraping process. Your progress will be displayed on the right side of the screen.
5. Scraping Results. This section shows you all the results from your no-code scraping runs. The same pane downloads the data as JSON, CSV or XLSX.

As you can see, setting up and running this tool is quite simple. Here’s an example of the data you can expect to get:

As you can see, with Google Trends no-code scraper, you can access search data for any time period that interests you, even without any programming skills.

Get the Data with Google Trends API

To build your own script instead, take HasData’s Google Trends API as the data source. To do this, you’ll need to sign up on our website to obtain an API key, just like in the previous method. You can find it on the main page of your account.

Next, you can use the API Playground to set your parameters and generate code in any programming language you like, or you can use the documentation and write the code yourself.

Unlike a no-code scraper, the Google Trends API provides a wider range of parameters that allow you to customize your query. Using the API, you can specify the following parameters:

1. Search Query. The heart of your search, defining the keyword or phrase you want to investigate.
2. Location. Specify the geographical area for your query, restricting results to a specific country, region, or city.
3. Region. The “region search” feature targets specific areas like cities, metropolitan areas, countries, or subregions.
4. Data Type. Interest over time, interest by region, related topics, or related queries.
5. Time Zone. Define the relevant time zone to accurately interpret search patterns across different regions.
6. Category. Narrow down your search by selecting a specific category, similar to the functionality on the Google Trends website.
7. Google Property. Filter results based on the search source, such as google search, news, images, or YouTube search.
8. Date Range. The time period the results cover.

These parameters cover the same filters the Trends page shows. The API Playground can generate the request in any language from them, and the example below writes the same code from scratch.

You can also view and run the ready-made version of the considered script on Google Collaboratory.

To begin, create a new file with the extension .py and import the necessary libraries. Since most of the work is done by the API, we will only need a library to make requests and work with JSON data, as the API returns data in JSON format.

import requests
import json

Next, we’ll define variables with the parameters we want to set. You can find a full list of parameters in our official documentation.

query = "Coffee"
geo = "US-NY"
region = "dma"
data_type = "geoMap"
category = "65"
date_range = "now 7-d".replace(" ", "+")

Then compose the link:

url = f"https://api.hasdata.com/scrape/google-trends/search?q={query}&geo={geo}&region={region}&dataType={data_type}&cat={category}&date={date_range}"

Put your API key to the request headers:

headers = {
  'Content-Type': 'application/json',
  'x-api-key': 'PUT-YOUR-API-KEY'
}

Make the request and print the response on the screen:

response = requests.request("GET", url, headers=headers)
print(response.text)

As a result you will get the data like at this example:

As you can see, in addition to the data itself, the API also returns a request URL and a screenshot of the query page that was executed.

With this information, you can either process the retrieved data or save it for later use. For example, let’s save the data to a JSON file:

data = response.json()
with open("google_trends_data.json", "w") as json_file:
    json.dump(data, json_file, indent=4)

You can also save the data to a file in any other format that is convenient for you, such as CSV.

Scrape Google Trends Using PyTrends Library

Another method is the PyTrends library. It is built on plain HTTP queries and employs the Requests and BeautifulSoup libraries for scraping, thus having certain limitations.

Scrape the Data with PyTrends

Let’s create a simple web scraper to extract data about interests by region. To do this, create a new Python script and import the necessary libraries into the project:

from pytrends.request import TrendReq

Then create a session pyTrend object:

pytrend = TrendReq()

And make a request to get a TOKEN:

pytrend.build_payload(kw_list=['coffee', 'green tea'])

After that, you can access the keyword data you need. For example, to get data by region and display it, use the following code:

interest_by_region_df = pytrend.interest_by_region()
print(interest_by_region_df.head())

As a result you will get the next data:

Unfortunately, as we mentioned before, using the PyTrends library has its drawbacks and limitations. For full functionality, you will need to use proxies. If you choose not to use them, your script will fail with a 429 error.

429 Error for PyTrends

The complete error message is “pytrends.exceptions.TooManyRequestsError: The request failed: Google returned a response with code 429”. This typically indicates that Google has flagged your request as suspicious and is refusing to return data. To resolve this issue, you’ll need to use proxies within your script.

We previously discussed how to use proxies in Python, and PyTrends has its own parameter for them. Simply specify the required parameters during object creation:

pytrend = TrendReq(hl='en-US', tz=360, proxies=['https://128.3.21.11:8080',])

It’s important to highlight that PyTrends only supports HTTPS proxies. You can either find free ones on our free proxy list page to test this functionality or purchase proxies from a reliable provider.

Use Selenium to Scrape Google Trends

The most complex yet flexible way to scrape data from Google Trends is to create your tool using a library like Selenium or any other library that supports headless browsers.

Here, we will discuss two options for obtaining the necessary data:

1. Standard method. This method involves navigating to the page and parsing its content. This is a more complex method, as it can be a bit difficult to find the necessary selectors.
2. A little trick to get the data you need. This will allow you to get all the data you need even if you don’t want to deal with selectors.

In any case, whichever method you choose, you will be able to get all the data you need from Google Trends.

Standard Scraping Method

This method involves using a headless browser, such as Selenium, to simulate user interactions and extract data from web pages. While this approach is relatively simple, it can be time-consuming. To use it, just follow this algorithm:

1. Formulating the URL. Construct the URL based on the specified parameters. This involves understanding the URL structure and incorporating the desired search terms, time range, and location filters.
2. Navigating to the Page. Open the constructed URL in a headless browser such as Selenium, which renders the dynamic page without a visible window.
3. Scraping the Data. Employ web scraping techniques to extract the relevant data from the HTML content of the page. This may involve identifying and parsing specific elements using XPath or CSS selectors.
4. Processing and Saving the Data. Clean, organize, and format the extracted data into a structured format, such as CSV or JSON. Save the processed data for further analysis or visualization.

Now, let’s implement the algorithm we discussed. First, we need to identify the patterns that govern the formation of the URL. To do this, we’ll visit the Google Trends page and examine all the available filters. Then, we’ll analyze how the URL changes based on the selected parameters.

For instance, with country US, category food and drinks, time last 7 days, and the keyword coffee, the link comes out as:

https://trends.google.com/trends/explore?cat=71&date=now%207-d&geo=US&q=coffee

Let’s start by creating a script and importing the necessary libraries and modules:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time

Next, we’ll configure the settings and generate a link:

category = "71"
date_range = "now 7-d"
geo = "US"
query = "coffee"

url = f"https://trends.google.com/trends/explore?cat={category}&date={date_range}&geo={geo}&q={query}"

Then, create a WebDriver object:

chrome_options = Options()

driver = webdriver.Chrome(options=chrome_options)

Now you can proceed to the page. However, there is a small nuance here. If you follow a direct link directly to the page with the required parameters, you will receive the following error:

This error can be easily avoided by accessing the page from the main Google Trends page or by going by the link twice:

driver.get(url)
driver.get(url)

One block on the page illustrates the whole extraction, so the example below gathers the “Related queries” section:

Inspecting the page’s HTML using DevTools (press F12 or right-click and select Inspect), reveals that the ‘Related Searches’ section is contained within a div element with the class ‘.fe-related-queries’. The individual related search items, within this section, have the class ‘.item’.

Let’s proceed with scraping this data:

related_queries_div = driver.find_element(By.CSS_SELECTOR, '.fe-related-queries')
items = related_queries_div.find_elements(By.CSS_SELECTOR, '.item')

To keep the rows usable, collect them into a list:

related_queries = []
for item in items:
    parts = item.text.split('\n')
    if len(parts) == 4:
        rank, title_category, score, more = parts
        title, category = title_category.split(' - ')

        related_queries.append({
            'rank': rank,
            'title': title,
            'category': category,
            'score': score
        })

Print the Google Trends data on the screen and close the webdriver:

print(related_queries)
driver.quit()

As a result you will get the following data:

The resulting list

Selenium also handles pagination when a block has more pages, and the other blocks extract the same way, with only the selectors differing.

Download Google Trends Data with the Export Buttons

As we mentioned at the beginning, there is a much simpler way to get Google Trends data, and that is to simply download it. Each block has a download button next to it that allows you to download the data:

Instead of manually collecting this data, we can simply download it all at once. The script structure, up to the page navigation part, will remain the same:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options

category = "71"
date_range = "now 7-d"
geo = "US"
query = "coffee"

url = f"https://trends.google.com/trends/explore?cat={category}&date={date_range}&geo={geo}&q={query}"

chrome_options = Options()

driver = webdriver.Chrome(options=chrome_options)
driver.get(url)
driver.get(url)

Then download the available Google Trends data:

export_buttons = driver.find_elements(By.CSS_SELECTOR, '.widget-actions-item.export')
for button in export_buttons:
    button.click()
    time.sleep(1)

As a result, we get four files with all the data from the page:

This approach allows you to obtain all the data quickly and easily. Moreover, the data is well-structured and can be used for further processing.

Conclusion

This article covered the four working routes to Google Trends data, from the no-code scraper to your own Selenium build, plus how the 0-100 numbers are made.

Which route fits depends on how much of the pipeline you want to own. The no-code scraper needs nothing but a query, the API needs a key and a request, and the Selenium build needs a browser you maintain in exchange for control over every step.