How to Get Data from Google Maps in Three Simple Steps Using the Web Scraping API

Scraping in the modern world is an essential part of any business. It allows you not to waste employees’ time manually collecting the necessary information, whether it is data on competitors or the dynamics of price changes on the trading floor.

Often scrapers are used to collect leads. They allow you to manage current databases or update old ones quickly.

Ways to Get Data

To use such an advanced tool as scraping, you must decide what you are ready to pay for: time, money, or programming knowledge.

Try Our Ready-Made Solutions for Your Needs

Google Maps Scraper

Effortlessly extract Google Maps data – business types, phone numbers, addresses, websites, emails, ratings, review counts, and more. No coding needed! Download results in convenient JSON, CSV, and Excel formats.

Google Maps Reviews Scraper

Effortlessly scrape and analyze customer reviews from Google Maps to gain actionable insights into customer sentiment, identify areas for improvement, and enhance your brand reputation. No coding expertise required! Easily download your data in Excel, CSV, or JSON formats for simple analysis and use.

Here we will consider precisely the option of using the Web Scraping API. And you can even try it for free - at least the first 1000 requests.

If you think programming is too complex, there is an alternative. For example, you can use Zapier for visual customization and integration.

Scraping Google Maps Using Web Scraping API

To use the web scraping API, sign up for HasData. To do this, click “Try for Free” and fill in the login information. Now you can create a simple request to scrape some data from Google Maps. To do this, go to the “Google Maps Builder” tab and enter the keyword for crawling in the keyword field:

Run the request from the site by clicking “Run Script.” It will return the first 20 rows of results from Google Maps.

Algorithm of Scraper Creating

We will write the entire script in the Python programming language, utilizing a Google Maps Search API. To execute the script, you’ll need to install a Python interpreter. For more information on how to get started with this technique, you can refer to an introductory article about web scraping with Python.

Now, to more accurately imagine the final result, let’s describe the algorithm of what we will consider next.

1. Try an example of a simple Google Maps scraper.

2. Get a list of keywords from an Excel document.

3. Get the start value and several positions from the command line.

4. Save data to a new document.

5. Creating a Python file with a simple Google Maps scraper is the easiest part. To do this, go to Google Maps scraper API builder and select the code type - Python. Then copy the resulting code into a file with the *.py extension.

However, then the HTTP.client library will be used. But for further convenience, we should use the requests library:

import requests
import json

url = "https://api.hasdata.com/scrape/google/locals"

payload = json.dumps({
  "country": "US",
  "domain": "com",
  "keyword": "Pizza",
  "ll": "@40.7455096,-74.0083012,14z"
})
headers = {
  'x-api-key': 'YOUR-API-KEY',
  'Content-Type': 'application/json'
}
response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

It is an example of the simplest scraper. So we can go to the next step.

Get a list of Keywords from a Document

Create a file that will store the keywords. Let’s name it “keywords.csv.” Therefore, every time the program goes to a new line and sees a new keyword, it should re-execute the query.

Add a line that will open the keywords.csv file for reading and specify a line-by-line cycle:

with open("keywords.csv", "r+") as f:
  for keyword in f:

Place here the request body, where instead of the given keyword, there will be a variable:

 temp = """{
      "country": "US",
      "domain": "com",
      "keyword": """+"\""+keyword.strip()+"\""+""",
      "ll": "@40.7455096,-74.0083012,14z"
    }
    """
    payload = json.loads(json.dumps(temp))

To ensure everything is fine and working, you can run the file on the command line.

Set Rows Amount

Let’s ask on the command line for the scraping start position and the number of pages from that position. We will understand 20 lines by one page since the API returns 20 rows, starting from the specified.

To do this, add a request to enter the starting position and enter it into the variable start:

start = int(input("Start position: "))

And also, add a request for the number of pages:

pages = int(input("Number of pages: "))

And with this in mind, let’s change the request body and add a loop for scraping the specified number of pages starting from the position start:

    for i in range(start, 20*pages, 20):
      temp = """{
        "country": "US",
        "domain": "com",
        "keyword": """+"\""+keyword.strip()+"\""+""",
        "ll": "@40.7455096,-74.0083012,14z",
        "start": """+str(i)+"""
      }
      """

It is important to remember that we can use only string variables in the request body, but we need numeric variables for the loop.

Save results in an Excel Document

For the data to be conveniently processed, let’s save it to an excel file. Every time the program starts, it shouldn’t add the data in an existing file to the end. In this case, we will delete the old data. We add a file overwrite at the beginning of the script. We will also name the columns in which we will save the data:

with open("result.csv", "w") as f:
  f.write("position; title; phone; address; website; rating; reviews; type\n")

We will place data saving in the try..except block so that if the request returns an empty response, the script doesn’t stop its work completely:

      try:
        for item in data["scrapingResult"]["locals"]:
          with open("result.csv", "a") as f:
            f.write(str(item["position"])+"; "+str(item["title"])+"; "+str(item["phone"])+"; "+str(item["address"])+"; "+str(item["website"])+"; "+str(item["rating"])+"; "+str(item["reviews"])+"; "+str(item["type"])+"\n")
      except Exception as e:
        print("There are no locals")

Let’s try to run the resulting script and check the generated result.csv file:

Try Our Ready-Made Solutions for Your Needs

Email Scraper

Find and extract emails from any website with ease. Build targeted email lists for lead generation, outreach campaigns, and market research. Download your extracted data in your preferred format (CSV, JSON, or Excel) for immediate use.

Yellow Pages Scraper

Yellow Pages Scraper allows you to quickly and easily extract data such as business names, addresses, categories, phone numbers, and so on from numerous listings. Download your data in JSON, CSV, and Excel formats. No coding is required.

Full Scraper Code

import requests
import json

url = "https://api.hasdata.com/scrape/google/locals"

headers = {
  'x-api-key': 'YOUR-API-KEY',
  'Content-Type': 'application/json'
}

with open("result.csv", "w") as f:
  f.write("position; title; phone; address; website; rating; reviews; type\n")

start = int(input("Start position: "))
pages = int(input("Number of pages: "))
with open("keywords.csv", "r+") as f:
  for keyword in f:
    for i in range(start, 20*pages, 20):
      temp = """{
        "country": "US",
        "domain": "com",
        "keyword": """+"\""+keyword.strip()+"\""+""",
        "ll": "@40.7455096,-74.0083012,14z",
        "start": """+str(i)+"""
      }
      """
      payload = json.loads(json.dumps(temp))
      response = requests.request("POST", url, headers=headers, data=payload)
      data = json.loads(response.text)
      try:
        for item in data["scrapingResult"]["locals"]:
          with open("result.csv", "a") as f:
            f.write(str(item["position"])+"; "+str(item["title"])+"; "+str(item["phone"])+"; "+str(item["address"])+"; "+str(item["website"])+"; "+str(item["rating"])+"; "+str(item["reviews"])+"; "+str(item["type"])+"\n")
      except Exception as e:
        print("There are no locals")

Conclusion and Takeaways

The web scraping API allows the developer to not worry about proxies and captcha solving and to focus on creating the main product. And using the Google Maps Search API allows you to save time when collecting leads or suppliers.

Using Python, the task of creating a full-fledged scraper is quite simple. Similarly to the option discussed in the article, you can also change coordinates or the country for which you will perform scraping.