READING How to Scrape Amazon Product Data & Reviews With Python 2026
On This Page

    How to Scrape Amazon Product Data & Reviews With Python 2026

    Amazon provides one of the largest collections of product information on the internet. Titles, prices, ratings, reviews, specifications, feature bullets, and high-resolution images are all valuable for research, analytics, and catalog enrichment. 

    This tutorial explains how to build a safe and structured scraper in Python that collects product details and visible reviews from publicly accessible Amazon pages. 

    The guide covers project setup, library installation, request handling, reliable parsing, and export to CSV and Excel files.

    Project Setup for an Amazon Scraper in Python

    To begin your Amazon scraping project, create a dedicated working folder. This keeps your scraper structured and avoids file clutter.

    Type the following in your terminal:

    mkdir amazon_scraper
    cd amazon_scraper
    
    • Install all required Python libraries for your Amazon scraping project.
    • These libraries support HTTP requests, HTML parsing, data processing, and file export.
    • We use requests for fetching pages, BeautifulSoup for parsing content, and pandas for organizing and exporting data.
    pip install requests beautifulsoup4 pandas openpyxl
    

    We will rely on the following tools:

    • requests for downloading HTML pages from Amazon
    • BeautifulSoup for parsing Amazon product pages
    • regular expressions for structured text extraction
    • pandas for DataFrame-based processing and Excel export
    • csv for lightweight file output
    • random and time for safe request spacing

    Step 1. Importing All Python Libraries for Amazon Scraping

    The scraper begins with clean imports. Having them at the top improves readability and helps maintain a single source of truth for dependencies. The same block supports product scraping and review scraping.

    import requests
    from bs4 import BeautifulSoup
    import re
    import time
    import random
    import pandas as pd
    

    These imports support every stage of Amazon HTML extraction and allow Python to handle network requests, parsing workflows, regex capture groups, export logic, and URL handling.

    Step 2. Setting Request Headers and Safe Delays for Amazon Scraping

    Amazon often serves incomplete pages if a request appears automated. A User Agent string gives your scraper a more realistic footprint. The Accept Language header improves the consistency of returned HTML.

    HEADERS = {
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64 "
            "AppleWebKit/537.36 (KHTML, like Gecko) "
            "Chrome/114.0.0.0 Safari/537.36"
        ),
        "Accept-Language": "en-US,en;q=0.9",
    }
    
    REQUEST_DELAY_RANGE = (2, 5)
    

    Step 3. Fetching HTML From Public Amazon Pages Safely

    Fetching HTML is central to any Amazon scraper. The code below includes retry logic, timeout settings, and checks for CAPTCHA-like patterns. This ensures the scraper does not accidentally parse incomplete or restricted pages.

    def fetch_page(url, retries=3, delay_range=REQUEST_DELAY_RANGE):
        """Fetch HTML from Amazon with retries."""
        for _ in range(retries):
            try:
                r = requests.get(url, headers=HEADERS, timeout=10)
                if r.status_code == 200 and "captcha" not in r.text.lower():
                    return r.text
            except Exception:
                pass
    
            time.sleep(random.uniform(*delay_range))
    
        print(f"[fetch_page] Failed to fetch: {url}")
        return None
    

    Step 4. Parsing Amazon Product Data With Python and BeautifulSoup

    Amazon product pages contain structured regions that hold title, prices, ratings, bullets, and technical details. These elements use specific HTML tags and attributes, but Amazon changes its layout from time to time. Because of this, it is important to know how to locate the correct HTML elements yourself.

    The easiest way is by using the browser’s Inspect Element tools by pressing F12. How to Use Inspect Element to Find the Parsers.

    To see exactly where a product detail is located in the HTML:

    1. Open the Amazon product or review page in your browse
    2. Press F12 to open Developer Tools
    3. Click the Inspect icon
    4. Hover over the element you want to scrape, and click it. The Elements panel will highlight the precise HTML structure.

    Identifying HTML Selectors for Your Parser

    Typical product elements appear in predictable locations. You can confirm them through Inspect Element:

    • Product title
      <span id=”productTitle”>
    • Price
      <span class=”a-price-whole”>
      <span class=”a-offscreen”>
    • Feature bullets
      located under
      <div id=”feature-bullets”>
    • Review blocks
      <li data-hook=”review”>
    • Review text
      <span class=”cr-original-review-content”>

     Make sure you include your regional variant Amazon site in the base URL.

    def parse_product(html, base_url="https://www.amazon.com"):
        soup = BeautifulSoup(html, "html.parser")
        result = {}
    
        # Title
        title_el = soup.find("span", {"id": "productTitle"})
        result["title"] = title_el.get_text(strip=True) if title_el else None
    
        # Price
        price_el = soup.find("span", class_="a-price-whole")
        if not price_el:
            price_el = soup.find("span", class_="a-offscreen")
        result["price"] = price_el.get_text(strip=True) if price_el else None
    
        # Rating
        rating_el = soup.find("span", {"data-hook": "rating-out-of-text"})
        result["rating_summary"] = rating_el.get_text(strip=True) if rating_el else None
    
        # Review count
        count_el = soup.find("span", {"id": "acrCustomerReviewText"})
        result["rating_count"] = count_el.get_text(strip=True) if count_el else None
    
        # Features
        bullets = []
        for li in soup.select("#feature-bullets .a-list-item"):
            txt = li.get_text(strip=True)
            if txt:
                bullets.append(txt)
        result["feature_bullets"] = bullets
    
        # Description
        desc_el = soup.find("div", {"id": "productDescription"})
        result["description"] = desc_el.get_text(strip=True) if desc_el else None
    
        # Technical details
        result["technical_details"] = {}
        table = soup.find("table", {"id": "productDetails_techSpec_section_1"})
        if table:
            for row in table.find_all("tr"):
                key = row.find("th").get_text(strip=True)
                val = row.find("td").get_text(strip=True)
                result["technical_details"][key] = val
    
        # Product images
        result["image_urls"] = re.findall(r'"hiRes":"(https?://[^"]+)"', html)
    
        return result
    

    Step 5. Scraping Amazon Reviews

    Amazon updated its review pages so that full review access often requires a login. Review pagination is also limited, and automated traffic is blocked frequently. A Python script can still extract visible reviews, but it must use headers and careful parsing. We can only scrape reviews visible on the product page.

    def parse_visible_reviews(html):
        soup = BeautifulSoup(html, "html.parser")
        reviews = []
    
        container = soup.select_one("#cm-cr-dp-review-list")
        if not container:
            print("❗ No visible review container found on product page.")
            return reviews
    
        review_blocks = container.select("[data-hook='review']")
    
        print(f"Found {len(review_blocks)} visible reviews on product page.")
    
        for li in review_blocks:
            review = {}
    
            # Reviewer name
            name_el = li.select_one(".a-profile-name")
            review["reviewer_name"] = name_el.get_text(strip=True) if name_el else None
    
            # Rating
            rating_el = li.select_one("i[data-hook='review-star-rating'] span.a-icon-alt")
            review["rating"] = rating_el.get_text(strip=True) if rating_el else None
    
            # Title
            title_el = li.select_one("a[data-hook='review-title'] span")
            review["review_title"] = title_el.get_text(strip=True) if title_el else None
    
            # Date
            date_el = li.select_one("span[data-hook='review-date']")
            review["date"] = date_el.get_text(strip=True) if date_el else None
    
            # Body text
            body_el = li.select_one("span[data-hook='review-body']")
            review["review_text"] = body_el.get_text(strip=True) if body_el else None
    
            # Images
            imgs = li.select("img[data-hook='review-image-tile']")
            review["image_urls"] = [img.get("src") for img in imgs]
    
            # Review ID
            review["review_id"] = li.get("id")
    
            reviews.append(review)
    
        return reviews
    

    Step 6. Exporting Amazon Product Data and Review Data

    Scraping often requires exporting results into stable files for indexing, transformation, or machine learning analysis. This code exports the data to both CSV and Excel.

    def save_products_to_files(products, csv_name="amazon_products.csv", xlsx_name="amazon_products.xlsx"):
        df = pd.DataFrame(products)
        df.to_csv(csv_name, index=False)
        df.to_excel(xlsx_name, index=False)
    
    
    def save_reviews_to_csv(reviews, filename="amazon_reviews.csv"):
        df = pd.DataFrame(reviews)
        df.to_csv(filename, index=False, encoding="utf-8")
    
    
    def save_reviews_to_excel(reviews, filename="amazon_reviews.xlsx"):
        df = pd.DataFrame(reviews)
        df.to_excel(filename, index=False)
    

    Step 7. Scrape Multiple Product URLs and Export the Results

    The script below scrapes several product URLs, extracts visible reviews, and exports clean datasets for further use.

    if __name__ == "__main__":
        product_urls = [
            "https://www.amazon.com/dp/B0DXMW74JZ",
            "https://www.amazon.com/dp/B0DYQF1Q3Z",
        ]
    
        products = []
        all_reviews = []
    
        for url in product_urls:
            print("\nScraping product:", url)
    
            html = fetch_page(url)
            if not html:
                continue
    
            # Product info
            product_data = parse_product(html)
            products.append(product_data)
    
            visible_reviews = parse_visible_reviews(html)
            all_reviews.extend(visible_reviews)
    
        if products:
            save_products_to_files(products)
        if all_reviews:
            save_reviews_to_csv(all_reviews)
            save_reviews_to_excel(all_reviews)
    
        print("\nDone!")
    

    This is a complete Amazon scraping workflow built entirely in Python.

    The same approach can be extended to other large online retailers as well. For example, our guide on Scraping Walmart Product Data using Python shows how to handle product listings, pricing, and structured details using a similar multi-URL scraping workflow.

    Want Amazon Product Data Without the Scraping Headache?

    This Python script is great for small-scale Amazon scraping, but keeping it running long term requires continuous maintenance. Amazon updates its HTML structure, review layouts, price blocks, and anti-bot rules frequently. Any change in these areas can break a parser overnight.

    Without proper proxy rotation, session management, and request handling, you can also run into IP blocks, throttling, or incomplete data.

    Amazon’s platform evolves often, and scraping it at scale without the right infrastructure can lead to blocked requests, missing fields, or unstable results.

    If you need reliable access to Amazon product data, including prices, variants, categories, reviews, and availability, ScrapeInsight can provide the solution.

    Get in touch and let us handle the hard parts for you.

    Keval Kothari

    Keval Kothari

    Founder and CEO