HasData
Back to all posts

The Complete Guide to Scraping Amazon Product Data using Python

Amazon is one of the largest and fastest-growing online marketplaces, attracting millions of monthly visitors worldwide . For researchers, business analysts, and sellers, it’s a valuable source of data, but extracting that data isn’t as simple as it might seem.

There are Amazon scrapers built with Python below, the same job through the Amazon API, and ready-to-use tools for anyone who would rather not write code at all.

What Amazon Checks Before It Serves You a Page

Amazon fronts its pages with AWS WAF, and a request that looks automated gets a bot-check page instead of the product. The page still returns HTTP 200, which is the part that catches people out. Your parser runs, finds none of its selectors, and writes empty fields without a single error in the log. Any scraper that cannot tell those two pages apart will silently produce nothing.

Three things decide which page you get.

The request headers. A bare requests.get() sends a python-requests/2.x user agent and almost nothing else, which is the easiest possible signal. Adding a browser user agent is the first thing to try, and on a quiet product page it is sometimes enough on its own. It stops being enough as soon as the rest of the headers disagree with it. A Chrome user agent arriving without Accept-Language, without Sec-Fetch-* and without the Accept string Chrome actually sends is a combination no real browser produces.

What runs the page. Parts of an Amazon product page are rendered after the initial HTML arrives, so even a request that is served correctly can come back missing the fields you wanted. That is why the scraper below uses Selenium rather than requests. A real browser executes what the page asks for and the selectors find their elements.

The address it came from. One datacenter IP asking for a few hundred product pages looks like exactly what it is. Residential exits and a slower request rate are the usual answer, and the cost of both is the reason the API section at the end of this article exists.

The order matters. Try headers first, because it is free and sometimes sufficient. Move to a browser when fields come back empty rather than wrong. Reach for proxies when the volume, not the page, is the problem.

Scraping Amazon Product Page

Amazon is one of the biggest marketplaces out there, and while manually gathering data from it could take forever, the good news is we can automate the whole process.

Full Amazon Product Scraper

If you’re in a rush, here’s the complete script right off the bat:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import pandas as pd
import time

chrome_options = Options()
driver = webdriver.Chrome(options=chrome_options)

url = "https://www.amazon.com/weBoost-472120-Signal-Booster-Carriers/dp/B081BM99M9"
driver.get(url)

time.sleep(5)


def find_or_none(by, value, element=None):
    target = element if element is not None else driver
    try:
        return target.find_element(by, value)
    except NoSuchElementException:
        return None


def find_all(by, value, element=None):
    target = element if element is not None else driver
    try:
        return target.find_elements(by, value)
    except NoSuchElementException:
        return []


def text_of(by, value, default, element=None):
    found = find_or_none(by, value, element)
    return found.text.strip() if found else default


title = text_of(By.ID, "productTitle", "Title not found")
price = text_of(By.CLASS_NAME, "priceToPay", "Price not found")
delivery_info = text_of(By.ID, "amazonGlobal_feature_div", "Delivery info not found")
about = text_of(By.ID, "featurebullets_feature_div", "About section not found")
review_link = find_or_none(By.ID, "acrCustomerReviewLink")
if review_link:
    rating = text_of(By.CLASS_NAME, "a-size-small.cm-cr-review-stars-rating-spacing", "No rating")
    review_count = text_of(By.CLASS_NAME, "a-size-small.a-color-base.cm-cr-review-stars-text-sm", "No reviews")
else:
    rating = "Rating not found"
    review_count = "Review count not found"


features_table = None
try:
    features_section = find_or_none(By.ID, "productOverview_feature_div")
    if features_section:
        features = features_section.find_element(By.TAG_NAME, "table")
        rows = features.find_elements(By.TAG_NAME, "tr")
        feature_data = []
        for row in rows:
            cols = row.find_elements(By.TAG_NAME, "td")
            if len(cols) == 2:
                feature_data.append([cols[0].text.strip(), cols[1].text.strip()])
        features_table = pd.DataFrame(feature_data, columns=["Feature", "Value"])
    else:
        features_table = pd.DataFrame(columns=["Feature", "Value"])
except Exception as e:
    features_table = pd.DataFrame(columns=["Feature", "Value"])
    print(f"Error extracting features table: {e}")

high_quality_images = set()
try:
    li_elements = find_all(By.CSS_SELECTOR, "li")
    for li in li_elements:
        img = find_or_none(By.TAG_NAME, "img", li)
        if img:
            img_url = img.get_attribute("data-a-hires") or img.get_attribute("src")
            if img_url and img_url.startswith("https://m.media-amazon.com"):
                if "AC_UF100" in img_url or "AC_UL100" in img_url:
                    continue
                high_quality_images.add(img_url)
except Exception as e:
    print(f"Error extracting images: {e}")

data = {
    "Title": title,
    "Price": price,
    "Delivery Info": delivery_info,
    "About": about,
    "Rating": rating,
    "Review Count": review_count
}

main_data_df = pd.DataFrame([data])
main_data_df.to_csv("main_data.csv", index=False)

images_df = pd.DataFrame(list(high_quality_images), columns=["Image URL"])
images_df.to_csv("images.csv", index=False)

features_table.to_csv("features_table.csv", index=False)

print("Extracted Data:")
for key, value in data.items():
    print(f"{key}: {value}")

print("\nFeatures Table:")
print(features_table)

print("\nImages:")
print(f"{len(high_quality_images)} images found")
for img in high_quality_images:
    print(f" - {img}")

driver.quit()

A while back, scraping Amazon was a bit easier. You could get away with using simpler libraries like Beautiful Soup. But as Amazon’s tracking methods have gotten smarter, dealing with CAPTCHAs has become a real headache.
The fix was a library that drives a real browser instead of sending bare requests. It takes more setup than the old approach and it survives what Amazon does to the page.

Step 1. Prerequisites

Product pages have a structure worth knowing before any code gets written, because the selectors come straight out of it. Each Amazon product page contains a lot of information, some of which is specific to a particular category.

A product page carries these elements worth scraping.

Amazon product page showing title, price, rating, reviews, description, and features sections to scrape

Here we can highlight the following CSS selectors for main elements:

Data FieldCSS Selector
Title#productTitle
Price.priceToPay
Delivery Info#amazonGlobal_feature_div
Rating and Reviews#acrCustomerReviewLink
Description (About)#featurebullets_feature_div
Features Table#productOverview_feature_div table
High-Quality Imagesli img[data-a-hires^="https://m.media-amazon.com"]

These elements are common to all product categories and can be extracted using the scraper. If there are additional elements on the page you need, feel free to update the resulting code to add them.

Step 2. Retrieve the page HTML

The first step is grabbing the whole page. Create a new .py file and import what the scraper needs.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import pandas as pd
import time

Pandas rather than the csv module, because it gives more flexibility when it comes to processing and saving data later on.

Configuring the driver, opening the page and waiting for it to render is four lines.

chrome_options = Options()
driver = webdriver.Chrome(options=chrome_options)

url = "https://www.amazon.com/weBoost-472120-Signal-Booster-Carriers/dp/B081BM99M9"
driver.get(url)

time.sleep(5)

Chrome is here because it works well with Selenium. Swap in another browser if you prefer one.

That sleep(5) is the weak line. A fixed wait is either too short on a slow connection or wasted time on a fast one, and WebDriverWait with a condition on the element you actually need is what a production scraper uses. It stays here because it keeps the example to one idea at a time.

Step 3. Scrape the product details

Two utility functions come first, because an element that isn’t on the page crashes the script when you reach for it. Wrapping every single search in try/except works and reads terribly, so the alternative is to write a single helper function to handle this for us. Spoiler alert: we’re going with the second option.

def find_or_none(by, value, element=None):
    target = element if element is not None else driver
    try:
        return target.find_element(by, value)
    except NoSuchElementException:
        return None


def find_all(by, value, element=None):
    target = element if element is not None else driver
    try:
        return target.find_elements(by, value)
    except NoSuchElementException:
        return []


def text_of(by, value, default, element=None):
    found = find_or_none(by, value, element)
    return found.text.strip() if found else default

Now, you might think, “Why not just use perfect selectors from the start and skip all this code?” Good question! But the reality is, missing data isn’t always a selector problem. There are plenty of reasons why data might not show up as expected. Maybe there’s a hiccup in the page load, and some elements just don’t render. Or maybe the page itself loads incorrectly due to some server-side glitch.

From my experience, even with spot-on selectors, data issues can and will happen. That is what the fallback functions are for. They keep the script running and they save the debugging session that a bare NoneType error would have cost.

The One-Call Fields

Title, price and description are each one call against the selectors from the table above. Rating and review count share a parent element, so that one grabs the parent first and reads both out of it.

title = text_of(By.ID, "productTitle", "Title not found")
price = text_of(By.CLASS_NAME, "priceToPay", "Price not found")
about = text_of(By.ID, "featurebullets_feature_div", "About section not found")

review_link = find_or_none(By.ID, "acrCustomerReviewLink")
if review_link:
    rating = text_of(By.CLASS_NAME, "a-size-small.cm-cr-review-stars-rating-spacing", "No rating")
    review_count = text_of(By.CLASS_NAME, "a-size-small.a-color-base.cm-cr-review-stars-text-sm", "No reviews")
else:
    rating = "Rating not found"
    review_count = "Review count not found"

Every call carries its own default, which is what keeps a missing element from becoming a NoneType error three functions downstream. The review parent is the one that goes missing most often, on products nobody has reviewed yet.

.priceToPay is the current price. Sale prices, subscribe-and-save prices and delivery-inclusive prices each live under their own class, so change the selector rather than the code if you want a different one.

The description is the weakest of the four. Most of what a buyer actually compares on is in the features table, which is next.

Product Features

Different product categories may have additional descriptions and features, but they are transient, dynamic, and different from category to category. So, let’s extract full table with product features and make a dataframe using pandas:

features_table = None
try:
    features_section = find_or_none(By.ID, "productOverview_feature_div")
    if features_section:
        features = features_section.find_element(By.TAG_NAME, "table")
        rows = features.find_elements(By.TAG_NAME, "tr")
        feature_data = []
        for row in rows:
            cols = row.find_elements(By.TAG_NAME, "td")
            if len(cols) == 2:
                feature_data.append([cols[0].text.strip(), cols[1].text.strip()])
        features_table = pd.DataFrame(feature_data, columns=["Feature", "Value"])
    else:
        features_table = pd.DataFrame(columns=["Feature", "Value"])
except Exception as e:
    features_table = pd.DataFrame(columns=["Feature", "Value"])
    print(f"Error extracting features table: {e}")

This approach will allow you to extract product features, no matter how many there are or which specific ones you’re looking for.

Delivery Infomation

The another important thing to scrape is delivery information:

delivery_info = text_of(By.ID, "amazonGlobal_feature_div", "Delivery info not found")

However, this data can vary depending on your location or the region of the proxies you’re using.

Product Images

Finally, let’s extract the image data:

high_quality_images = set()
try:
    li_elements = find_all(By.CSS_SELECTOR, "li")
    for li in li_elements:
        img = find_or_none(By.TAG_NAME, "img", li)
        if img:
            img_url = img.get_attribute("data-a-hires") or img.get_attribute("src")
            if img_url and img_url.startswith("https://m.media-amazon.com"):
                if "AC_UF100" in img_url or "AC_UL100" in img_url:
                    continue
                high_quality_images.add(img_url)
except Exception as e:
    print(f"Error extracting images: {e}")

That filter keeps the full-size images and drops the thumbnails, which share the same host but carry AC_UF100 or AC_UL100 in the path.

Step 4. Export to CSV

The extracted fields go into one dictionary before they are written out.

data = {
    "Title": title,
    "Price": price,
    "Delivery Info": delivery_info,
    "About": about,
    "Rating": rating,
    "Review Count": review_count
}

Three files come out of one product page, because the fields, the images and the features table have different shapes and forcing them into one CSV loses all of them.

main_data_df = pd.DataFrame([data])
main_data_df.to_csv("main_data.csv", index=False)

images_df = pd.DataFrame(list(high_quality_images), columns=["Image URL"])
images_df.to_csv("images.csv", index=False)

features_table.to_csv("features_table.csv", index=False)

driver.quit()

This step is crucial to avoid overloading your PC.

Scraping Amazon Product Listings

Building a scraper for Amazon’s search results or product listings works much the same way, using Python. Like in the previous example, we’ll rely on Selenium to get the page’s source code and use selectors to extract the data we need.

Full Amazon Product Listings Scraper

So, if you’re just looking for the ready-made code, feel free to copy it right away::

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time
import csv

chrome_options = Options()
driver = webdriver.Chrome(options=chrome_options)

url = 'https://www.amazon.com/s?k=pen'
driver.get(url)

time.sleep(10)

cards = driver.find_elements(By.CSS_SELECTOR, '.s-main-slot .s-result-item')

data = []

for card in cards:
    title_elem = card.find_elements(By.CSS_SELECTOR, 'h2 .a-link-normal span')
    title = title_elem[0].text if title_elem else 'Title not found'

    price_whole_elem = card.find_elements(By.CSS_SELECTOR, '.a-price-whole')
    price_fraction_elem = card.find_elements(By.CSS_SELECTOR, '.a-price-fraction')
    if price_whole_elem and price_fraction_elem:
        price = f"{price_whole_elem[0].text}.{price_fraction_elem[0].text}"
    else:
        price = 'Price not found'

    rating_elem = card.find_elements(By.CSS_SELECTOR, 'div[data-cy="reviews-block"] .a-icon-alt')
    rating = rating_elem[0].get_attribute('aria-label') if rating_elem else 'Rating not found'

    reviews_elem = card.find_elements(By.CSS_SELECTOR, 'div[data-csa-c-content-id="alf-customer-ratings-count-component"]')
    reviews = reviews_elem[0].text if reviews_elem else 'Reviews not found'

    image_elem = card.find_elements(By.CSS_SELECTOR, '.s-image')
    image_url = image_elem[0].get_attribute('src') if image_elem else 'Image not found'
    if title == "Title not found":
        continue
    else:
        data.append({
            'title': title,
            'price': price,
            'rating': rating,
            'reviews': reviews,
            'image_url': image_url
        })

keys = data[0].keys()
with open('amazon_data.csv', 'w', newline='', encoding='utf-8') as output_file:
    dict_writer = csv.DictWriter(output_file, fieldnames=keys)
    dict_writer.writeheader()
    dict_writer.writerows(data)

driver.quit()

Now, if you’re interested in a bit more explanation, this time around, we’re skipping pandas and going with the good old csv library instead. Honestly, it’s more than enough for what we need.

Step 1. Analyze a listings page

An Amazon search results page with relevant products looks like:

Amazon Search Page

Open DevTools with F12, or right-click the page and pick Inspect, and read the selectors off the elements you want. These are the bits of HTML that correspond to the product titles, prices, images, and so on. Here’s a quick table of the selectors we’re focusing on:

Data FieldCSS Selector
Titleh2 .a-link-normal span
Price.a-price-whole и .a-price-fraction
Ratingdiv[data-cy=“reviews-block”] .a-icon-alt
Reviewsdiv[data-csa-c-content-id=“alf-customer-ratings-count-component”]
Image URL.s-image

On to the script.

Step 2. Scrape search results with Python

The first part of the code will stay mostly the same, with a few changes. Instead of using a product page URL, we’ll use a search results URL. Also, we’ll need to import the csv library to save our data later.

Here’s the updated part:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time
import csv

chrome_options = Options()
driver = webdriver.Chrome(options=chrome_options)

url = 'https://www.amazon.com/s?k=pen'
driver.get(url)

time.sleep(10)

Now, we’ll gather all the product cards on the search results page and save the data into a variable.

cards = driver.find_elements(By.CSS_SELECTOR, '.s-main-slot .s-result-item')

data = []

Next, we’ll loop through each product card and extract the relevant details like title, price, rating, reviews, and image URL. Here’s how to do it:

for card in cards:
    title_elem = card.find_elements(By.CSS_SELECTOR, 'h2 .a-link-normal span')
    title = title_elem[0].text if title_elem else 'Title not found'

    price_whole_elem = card.find_elements(By.CSS_SELECTOR, '.a-price-whole')
    price_fraction_elem = card.find_elements(By.CSS_SELECTOR, '.a-price-fraction')
    if price_whole_elem and price_fraction_elem:
        price = f"{price_whole_elem[0].text}.{price_fraction_elem[0].text}"
    else:
        price = 'Price not found'

    rating_elem = card.find_elements(By.CSS_SELECTOR, 'div[data-cy="reviews-block"] .a-icon-alt')
    rating = rating_elem[0].get_attribute('aria-label') if rating_elem else 'Rating not found'

    reviews_elem = card.find_elements(By.CSS_SELECTOR, 'div[data-csa-c-content-id="alf-customer-ratings-count-component"]')
    reviews = reviews_elem[0].text if reviews_elem else 'Reviews not found'

    image_elem = card.find_elements(By.CSS_SELECTOR, '.s-image')
    image_url = image_elem[0].get_attribute('src') if image_elem else 'Image not found'
    if title == "Title not found":
        continue
    else:
        data.append({
            'title': title,
            'price': price,
            'rating': rating,
            'reviews': reviews,
            'image_url': image_url
        })

driver.quit()

At this point, we have a data list that holds all the information we need. You can either add pagination handling to scrape more pages, or, if you’re happy with the data on the current page, you can save it to a CSV.

Step 3. Handle pagination

Instead of manually linking to separate search result pages, we can take advantage of pagination. Extracting the last page number gives you an array of links to visit, which covers every result for the query.

def get_pagination_links():
    page_links = []
    pagination_items = driver.find_elements(By.CSS_SELECTOR, '.s-pagination-item')

    for item in pagination_items:
        if item.tag_name == 'a' and 'href' in item.get_attribute('outerHTML'):
            page_link = item.get_attribute('href')
            page_links.append('https://www.amazon.com' + page_link)

    return page_links

page_links = get_pagination_links()

for url in page_links:
    pass  # the per-page scraping code goes here

Wrap the earlier code in a loop over the array of links, and change the file save mode from w (which overwrites the file) to a (which appends data to the file if it already exists).

Step 4. Export to CSV

The product scraper above saved with pandas. This one uses the csv library instead, which avoids the dependency when all you need is a flat file.

keys = data[0].keys()
with open('amazon_data.csv', 'w', newline='', encoding='utf-8') as output_file:
    dict_writer = csv.DictWriter(output_file, fieldnames=keys)
    dict_writer.writeheader()
    dict_writer.writerows(data)

As you can see, it’s pretty simple too. In the past, you could also scrape product reviews the same way. However, Amazon has tightened its policy, and now, in order to view more than 8 reviews for a product, you’ll need to sign in.

Scraping Data Using Amazon API

As already mentioned, we can use a web scraping API to solve the tasks at hand. It takes care of performing the requests for you. This means you don’t have to manage proxy servers or worry about request reliability. In addition, our web scraping API allows you to quickly collect data using CSS selectors, using extraction rules.

Getting API key

To get started, you need to get an API key. You can find it in your account after signing up on HasData. In addition, you will receive 1,000 free credits when you register to test our features.

HasData dashboard on the API Keys page, with the Default key row highlighted

Save it, as you will need this API Key later.

Scrape Product Details with Amazon API

Now let’s get the same data as in the previous examples, but use the web scraping API. Also, we will save all the data we get to CSV file:

import requests
import csv

api_key = 'YOUR-API-KEY'
asin = 'B0F5HPCLGB'
url = f"https://api.hasdata.com/scrape/amazon/product?asin={asin}"

headers = {
    'Content-Type': 'application/json',
    'x-api-key': api_key
}

response = requests.get(url, headers=headers)

product_data = response.json()

fields = [
    "asin", "url", "title", "price", "usedPrice", "isAvailable",
    "brand", "material", "product_dimensions", "care_instructions", "upc",
    "totalReviews", "rating", "features", "mainImage"
]

rows = [
    [
        product_data['product'].get('asin'),
        product_data['product'].get('url'),
        product_data['product'].get('title'),
        product_data['product'].get('price'),
        product_data['product'].get('usedPrice'),
        product_data['product'].get('isAvailable'),
        product.get("brand"),
        product.get("isAvailable"),
        price.get("currentPrice"),
        price.get("beforePrice"),
        price.get("discount"),
        spec.get("UPC"),
        spec.get("Manufacturer"),
        reviews.get("totalReviews"),
        reviews.get("rating"),
        len(product.get("variants", [])),
        product.get("primaryImage")
    ]
]

csv_file = "product_data.csv"

with open(csv_file, mode='w', newline='', encoding='utf-8') as file:
    writer = csv.writer(file)
    writer.writerow(fields)
    writer.writerows(rows)

print(f"Data saved to {csv_file}")

Two fields there are the reason most people scrape Amazon in the first place. price is an object rather than a number, carrying currentPrice, beforePrice and discount, so a price cut is visible without diffing two runs. variants lists every colour, size and configuration of the listing with its own ASIN, so a follow-up call per entry gets you the whole family. seller, sellerUrl and shipper name who is selling and who is shipping, which on Amazon are regularly not the same company.

One field to read before the rest: isAvailable. An unavailable listing comes back with a much smaller object and no price at all, so a script that reaches straight for the price writes blanks on every out-of-stock product it meets.

Now, let’s walk through the main stages of the script in detail. First, we start by importing the necessary libraries:

import requests
import csv

Next, we define the variables for the query parameters that we might want to tweak later. To keep things organized, I recommend placing them right at the top of the script:

api_key = 'YOUR-API-KEY'
asin = 'B0F5HPCLGB'
url = f"https://api.hasdata.com/scrape/amazon/product?asin={asin}"

Once that’s done, we set up the actual request parameters and execute the request:

headers = {
    'Content-Type': 'application/json',
    'x-api-key': api_key
}

response = requests.get(url, headers=headers)

Finally, we parse the data we get back and save it to a file.

product_data = response.json()

fields = [
    "asin", "url", "title", "price", "usedPrice", "isAvailable",
    "brand", "material", "product_dimensions", "care_instructions", "upc",
    "totalReviews", "rating", "features", "mainImage"
]

rows = [
    [
        product_data['product'].get('asin'),
        product_data['product'].get('url'),
        product_data['product'].get('title'),
        product_data['product'].get('price'),
        product_data['product'].get('usedPrice'),
        product_data['product'].get('isAvailable'),
        product.get("brand"),
        product.get("isAvailable"),
        price.get("currentPrice"),
        price.get("beforePrice"),
        price.get("discount"),
        spec.get("UPC"),
        spec.get("Manufacturer"),
        reviews.get("totalReviews"),
        reviews.get("rating"),
        len(product.get("variants", [])),
        product.get("primaryImage")
    ]
]

csv_file = "product_data.csv"

with open(csv_file, mode='w', newline='', encoding='utf-8') as file:
    writer = csv.writer(file)
    writer.writerow(fields)
    writer.writerows(rows)

print(f"Data saved to {csv_file}")

And that’s it! You now have the data ready to use however you like.

Scrape Search Results with Amazon API

For those just looking for results, here’s the code ready to go:

import requests
import csv

query = "Laptop"
page = 1
api_key = "YOUR-API-KEY"
output_file = "amazon_search_results.csv"
api_url = f"https://api.hasdata.com/scrape/amazon/search?q={query}&page={page}"


headers = {
    'Content-Type': 'application/json',
    'x-api-key': api_key
}
payload = {}

response = requests.get(api_url, headers=headers, data=payload)
response_data = response.json()

if 'productResults' in response_data:
    products = response_data['productResults']

    with open(output_file, mode='w', newline='', encoding='utf-8') as file:
        writer = csv.writer(file)
        writer.writerow(['Position', 'Title', 'URL', 'ASIN', 'Image', 'Total Reviews', 'Rating', 'Price'])

        for product in products:
            writer.writerow([
                product.get('position'),
                product.get('title'),
                product.get('url'),
                product.get('asin'),
                product.get('image'),
                product.get('reviews', {}).get('totalReviews'),
                product.get('reviews', {}).get('rating'),
                product.get('price', {}).get('currentPrice')
            ])
else:
    print("No product results found.")

This script is close to the previous one. The changes are the URL it targets and the parameters and fields it asks for.

First, update the URL and any relevant parameters. For example, if the endpoint or query options change, you’ll need to reflect that here:

import requests
import csv

query = "Laptop"
page = 1
api_key = "YOUR-API-KEY"
output_file = "amazon_search_results.csv"
api_url = f"https://api.hasdata.com/scrape/amazon/search?q={query}&page={page}"

Make a request:

headers = {
    'Content-Type': 'application/json',
    'x-api-key': api_key
}
payload = {}

response = requests.get(api_url, headers=headers, data=payload)
response_data = response.json()

Next, adjust the parsing logic to match the new data structure. If you’re saving to a file, it’s worth taking a moment to verify that everything lines up correctly.

if 'productResults' in response_data:
    products = response_data['productResults']

    with open(output_file, mode='w', newline='', encoding='utf-8') as file:
        writer = csv.writer(file)
        writer.writerow(['Position', 'Title', 'URL', 'ASIN', 'Image', 'Total Reviews', 'Rating', 'Price'])

        for product in products:
            writer.writerow([
                product.get('position'),
                product.get('title'),
                product.get('url'),
                product.get('asin'),
                product.get('image'),
                product.get('reviews', {}).get('totalReviews'),
                product.get('reviews', {}).get('rating'),
                product.get('price', {}).get('currentPrice')
            ])
else:
    print("No product results found.")

With those changes the script handles the new task.

Ask Your AI Through MCP

The API section above assumes your code does the asking. When the asker is an AI client, the same endpoints are an MCP server, and the whole integration is one config block:

{
  "mcpServers": {
    "hasdata": {
      "type": "http",
      "url": "https://mcp.hasdata.com/api/mcp",
      "headers": { "x-api-key": "YOUR_API_KEY" }
    }
  }
}

The client picks up five Amazon tools from the server. They cover product details by ASIN, search, reviews, seller details and seller products. After that, a request is a sentence:

Terminal window where a one-line question asks for the top three Amazon results for usb c hub, answered with a three-row table of title, price and rating

The model received the same parsed JSON the Python sections work with, so nothing between the question and the table touched HTML. Asking for a single product works the same way, getProductDetails with an ASIN returns the title, rating, availability and specification blocks from the API section above.

One limit worth knowing. Ask an agent for thousands of ASINs and a good one will not call this tool a thousand times. It will write the Python from the sections above and run it, because code is cheaper for it than tokens, and a script’s output does not vary. At volume, even the AI writes the script.

No-Code Amazon Scraping

The easiest route to Amazon data returns it in more than JSON or CSV format but also in a way that’s ready to be imported into your Shopify store.

If you’re like me and really don’t want to mess around with proxies or setting up a captcha-solving service, this could be a solid option for you. And even if you’re totally not into coding, don’t worry – this method is completely code-free.

Scrape Amazon Product Pages

Let’s start with the Amazon Product no-code scraper. You can find it in your account on our website under the “no-code scrapers” tab. Here’s what it looks like:

HasData Amazon Product no-code scraper interface with URL input field

To use it, simply provide the links to the product pages you want to scrape and hit the start button. Once the scraper finishes, you can download the results in your preferred format from the right side of the screen.

Example data:

Example CSV output from Amazon Product scraper showing product title, price, rating, and other fields

As you can see, we got the same data as before, but even more of it, and the best part is we didn’t have to write any code. The results were ready almost instantly.

Scrape Amazon Search Results

Let’s now take a look at the next no-code scraper – the Amazon Search Results scraper. Let’s head over to the scraper’s page and see what needs to be filled out:

HasData Amazon Search Results no-code scraper with keyword and results count input fields

There are a few more fields here, but they’re all important. You’ll need to specify the number of search results you want to get for your query, as well as a list of keywords. You don’t necessarily have to specify the Amazon domain unless it’s something that matters to you.

As a result, you’ll get data in the following format:

Example output from Amazon Search Results scraper with product position, title, ASIN, and price columns

The file actually had a lot more data, but it didn’t all fit into the screenshot.

Scrape Amazon Best Sellers

The Best Sellers no-code scraper covers a page none of the Python sections do, since Best Sellers has no stable public markup worth teaching. Its form takes a category URL:

HasData Amazon Best Sellers no-code scraper with category URL input field

To use it, all you need to do is input the category links, and you’ll get a list of top products in those categories:

Example output from Amazon Best Sellers scraper showing top products with title, price, and rating

As you can see, getting this data was also easy.

Conclusion

Collecting data from Amazon pages has several routes, and each one breaks differently. The bot check that returns HTTP 200 with no product on it is the failure worth designing against, because it is the one that produces empty rows instead of errors.

You can use the web scraping API to avoid problems such as IP blocking and dealing with dynamic content. If you don’t want to face any difficulties, you can use our ready-made no-code Amazon scraper, which will give you Amazon product details in convenient format.

Valentina Skakun
Valentina Skakun
Valentina is a software engineer who builds data extraction tools before writing about them. With a strong background in Python, she also leverages her experience in JavaScript, PHP, R, and Ruby to reverse-engineer complex web architectures.If data renders in a browser, she will find a way to script its extraction.
Articles

Might Be Interesting