7 Python Scripts to Automate SEO in 2026

Writing Python code is no longer the primary challenge for SEOs in 2026. AI agents like Claude and Gemini generate functional scraping scripts in seconds. The real challenge for Technical SEOs has shifted from syntax to scale and access.

Most SEO scripts fail in production. They crash when processing large datasets or get blocked by anti-bot systems. Google started requiring JavaScript in Search in early 2025, which put accurate SERP data behind a rendering step. Standard libraries like requests or urllib fail to retrieve complete data due to TLS fingerprinting and client-side rendering.

This guide gives you seven Python scripts that survive that condition, and not one of them checks status codes. They solve specific engineering problems: clustering SERPs by intent, watching AI Overviews, and catching search trends while they move. 

What Happens When You Run the AI’s Python SEO Script

Point it at 200 live sites and 108 hand over the data, 84 refuse outright, and 8 return a page that looks fine while dropping fields the page actually has.

Ask Google for a Python SEO script and it’ll write one for you. The AI Overview we captured for that query returned a complete on-page auditor built on requests and BeautifulSoup, with pip install instructions and an offer to extend it to a sitemap crawl. Its own summary of the code described the browser User-Agent header as there “to guarantee high reliability against script blocks or access denial”. Google rewrites these answers constantly, so that’s one capture rather than a standing promise, and what follows is what that script actually did.

We took that script unchanged and pointed it at 200 live domains, 40 each from online stores, marketplaces, job boards, news sites, and SaaS sites. The lists are well-known sites per category, and 199 of the 200 carry a Tranco rank (abcnews.go.com is the exception, since Tranco indexes the bare domain). One request per domain. Then every URL that didn’t come back clean went through a rendering API, so we could see what the page actually contains.

The refusals are the loud half. 56 came back as plain 403s. Then 10 read timeouts, 6 rate limits, and three 202s where the site answered with a challenge page instead of content. Two 401s, and one each of 404, 405, 418, 451, an SSL failure, a connection reset and a connect timeout. A developer sees those immediately, because the script prints CRITICAL: Could not fetch page. The browser User-Agent bought nothing on the sites where money changes hands, 12 of 40 online stores and 10 of 40 marketplaces let it through.

The chart holds the loud failures. The quiet ones cost more.

Why a Silent Blank Is Worse Than a 403

Eight domains returned HTTP 200 and a page the script could parse, and it still reported fields as missing that are right there. On aliexpress.com it printed missing title, missing description, and zero H1 tags, while the rendered page has all three. Temu returned a title and description to the browser and neither to the script. Lowe’s, Business Insider, and Vivid Seats each lost their meta description. LinkedIn lost the description and the H1. Jetbrains and AP News lost the H1 alone.

That’s the failure mode you plan around. A 403 stops the run and gets fixed. A silent blank flows straight into the report, and now your deliverable says the page has no H1 when it does. On a 200-URL audit at this rate, eight rows of a spreadsheet are confidently wrong and nothing in the output marks them.

One caveat worth stating. Of the 108 clean fetches, 24 reported zero H1 tags and the rendered page had none either, and one more (nordstrom.com) came back without a title from both the script and the rendered fetch. Those are real absences, not artifacts, which is why the silent-blank count stays at eight instead of being inflated to 33.

The scripts below take the other route. Scraping work goes through an API that renders the page and rotates the IP, and the ones that need a rendered DOM ask for it. That’s the block class of failures gone, though not every failure. Script 6 still marks a page it couldn’t read as low-content, so read the skip lines rather than trusting the totals.

The Tech Stack

The dependency list stays short on purpose. Everything runs on Python 3.12 or newer. The HasData APIs handle the headless browsing, proxy rotation, and challenge pages that the measurement above priced, which is what replaces a maintained Selenium or Puppeteer install. requests and urllib3 do the HTTP work with Session objects and HTTPAdapter for connection reuse, pandas and scikit-learn cover the dataframes and the N-gram vectorization without a full NLP pipeline, trafilatura pulls clean body text out of raw HTML, and Seaborn with Matplotlib draws the similarity heatmap.

The measurement above is the argument for that shape, so rather than repeat it: enter a keyword below to see what a SERP API call returns for it.

This JSON response provides the foundation for most of the scripts in this guide. It allows you to extract AI Overviews, global position, and related queries without writing complex parsing logic.

Script 1. Recursive Google Autosuggest Scraper

Most SEOs rely on third-party databases (Ahrefs, Semrush) for keyword research. These databases rely on historical clickstream data. They often miss long-tail queries and emerging search patterns.

Google Autosuggest contains real-time intent data, but extracting it at scale is difficult. Querying Google’s suggest endpoint 500 times in a minute from a local IP to build a semantic tree will trigger an immediate block.

The Code

The script pairs the Google Suggest XML endpoint with the HasData API, which supplies the rotating proxy pool the endpoint’s rate limits require. By iterating through the alphabet (e.g., “keyword + a”, “keyword + b”), we force the expansion of the suggestion tree.

import requests
import xml.etree.ElementTree as ET
import string
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from urllib.parse import quote
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

# --- CONFIGURATION ---
API_KEY = "HASDATA_API_KEY"
BASE_KEYWORD = "coffee"
MAX_DEPTH = 2  # Depth of recursion (e.g., 1 = 'coffee a', 2 = 'coffee ab')

# Set this according to your HasData plan's concurrency limit.
# For paid plans, this is 15–1500. For the trial plan, keep it at 1.
MAX_WORKERS = 1 

# --- SESSION SETUP (PERFORMANCE OPTIMIZATION) ---
# We use a global Session object to enable TCP connection reuse (Keep-Alive).
# This significantly reduces the overhead of establishing new SSL handshakes for every request.
session = requests.Session()

# Configure the HTTP Adapter with a connection pool.
# pool_maxsize must be greater than or equal to MAX_WORKERS to prevent blocking.
adapter = HTTPAdapter(
    pool_connections=1, 
    pool_maxsize=MAX_WORKERS + 5,  # Adding a small buffer
    max_retries=Retry(total=3, backoff_factor=1, status_forcelist=[500, 502, 503, 504])
)

# Mount the adapter to both HTTP and HTTPS protocols
session.mount("https://", adapter)
session.mount("http://", adapter)

def fetch_suggestions(query):
    """
    Sends a request to Google Suggest via HasData API using a rotating proxy.
    """
    # 1. URL Encode the query to handle spaces and special characters correctly
    # 'coffee ad' becomes 'coffee%20ad'
    encoded_query = quote(query)
    target_url = f"https://suggestqueries.google.com/complete/search?output=toolbar&hl=en&q={encoded_query}"
    
    # HasData requires the API key in the headers
    headers = {
        "x-api-key": API_KEY,
        "Content-Type": "application/json"
    }
    
    # payload configuration
    payload = {
        "url": target_url,
        "jsRendering": False,      # JS is not needed for the XML endpoint, saves credits/time
        "outputFormat": ["html"]   # HasData will return the raw response body
    }

    try:
        # IMPORTANT: Use session.post() instead of requests.post() to leverage the connection pool
        response = session.post(
            "https://api.hasdata.com/scrape/web",
            headers=headers,
            json=payload,
            timeout=30
        )
        
        if response.status_code != 200:
            # Log errors for debugging
            print(f"Error {response.status_code} for '{query}': {response.text}")
            return []
        
        # 2. Parse the Response
        # Google Suggest returns XML. HasData passes this raw content in the response body.
        xml_content = response.content
        root = ET.fromstring(xml_content)
        
        suggestions = []
        for child in root:
            if len(child) > 0 and 'data' in child[0].attrib:
                suggestions.append(child[0].attrib['data'])
                
        return suggestions

    except ET.ParseError:
        return [] # Ignore parsing errors (e.g., malformed XML)
    except Exception as e:
        print(f"Exception for '{query}': {e}")
        return []

def generate_search_terms_suffix(base, current_suffix, depth, max_depth):
    """
    Recursively generates search terms by appending characters to a suffix.
    Example: 
    Depth 1: coffee a ... coffee z
    Depth 2: coffee aa ... coffee zz
    """
    terms = []
    for char in string.ascii_lowercase:
        new_suffix = current_suffix + char
        term = f"{base} {new_suffix}"
        terms.append(term)
        
        # Recursive call to go deeper if max_depth is not reached
        if depth < max_depth:
            terms.extend(generate_search_terms_suffix(base, new_suffix, depth + 1, max_depth))
            
    return terms

def run_harvest():
    print(f"Generating queries for Depth {MAX_DEPTH}...")
    queries = generate_search_terms_suffix(BASE_KEYWORD, "", 1, MAX_DEPTH)
    print(f"Total queries to process: {len(queries)}")

    results = set()
    start_time_global = time.time()

    # ThreadPoolExecutor manages concurrent execution
    with ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        future_to_query = {executor.submit(fetch_suggestions, q): q for q in queries}
        
        count = 0
        total = len(queries)
        
        print(f"\nStarting harvest with {MAX_WORKERS} workers...")
        
        for future in as_completed(future_to_query):
            query = future_to_query[future]
            count += 1
            
            # Progress logging every 20 requests to monitor speed
            if count % 20 == 0:
                elapsed = time.time() - start_time_global
                rps = count / elapsed if elapsed > 0 else 0
                print(f"Progress: {count}/{total} | Speed: {rps:.2f} req/s")

            try:
                suggestions = future.result()
                if suggestions:
                    for s in suggestions:
                        results.add(s)
            except Exception:
                pass # Skip errors during mass scraping

    total_time = time.time() - start_time_global
    print(f"\nFinished in {total_time:.2f} seconds. Average Speed: {len(queries)/total_time:.2f} req/s")
    return results

if __name__ == "__main__":
    unique_keywords = run_harvest()
    
    filename = "long_tail_keywords_hasdata.csv"
    
    # Save results to CSV
    with open(filename, "w", encoding="utf-8") as f:
        f.write("Keyword\n")
        f.write("\n".join(unique_keywords))

    print(f"Done. Collected {len(unique_keywords)} unique keywords.")
    print(f"Saved to {filename}")

The alphabet loop is what makes the query count large, so the run below is the honest cost of depth 2 on one worker.

Execution Log

Running it on the trial plan prints this:

Generating queries for Depth 2...
Total queries to process: 702

Starting harvest with 1 workers...
Progress: 20/702 | Speed: 1.40 req/s
…
Progress: 680/702 | Speed: 1.47 req/s
Progress: 700/702 | Speed: 1.47 req/s

Finished in 477.06 seconds. Average Speed: 1.47 req/s
Done. Collected 5244 unique keywords.
Saved to long_tail_keywords_hasdata.csv

That log is a trial-plan run, MAX_WORKERS = 1, which is what the code above ships with: 702 queries in 477 seconds at 1.47 requests per second, 5,244 unique keywords. ThreadPoolExecutor over a global requests.Session reuses TCP connections, so raising MAX_WORKERS to your plan’s concurrency limit multiplies that rate almost linearly, and the same harvest finishes in well under a minute at 15 workers. Set the constant to what your plan allows, not higher, since the API answers a breach of the limit with 429 Too Many Requests.

Script 2. Rising Search Trends from the Google Trends API

While standard tools provide lagging indicators, Google Trends offers leading indicators. To capture traffic before competitors, you need to identify what users are searching for right now.

The “Rising” metric in related queries identifies terms with breakout growth (e.g., +3,450%), often before they register in volume databases.

The Code

We call the HasData Google Trends endpoint to avoid the instability of libraries like pytrends and the rate limits they run into. We request relatedQueries to extract specific search terms associated with the seed keyword.

import requests

# Configuration
API_KEY = "HASDATA_API_KEY"
SEED_TOPIC = "Coffee"

def get_breakout_trends(topic):
    """
    Fetches 'Rising' and 'Top' related queries from Google Trends.
    """
    # Endpoint documentation: https://docs.hasdata.com/apis/google-trends/search
    url = "https://api.hasdata.com/scrape/google-trends/search"
    
    # HasData requires the API key in headers
    headers = { "x-api-key": API_KEY, "Content-Type": "application/json" }

    params = {
        "q": topic,
        "dataType": "relatedQueries", # Requests both 'top' and 'rising' lists
        "geo": "US",                  # Optional: Target specific region
        "date": "now 7-d"             # Past 7 days
    }

    try:
        response = requests.get(url, headers=headers, params=params)
        
        if response.status_code == 200:
            return response.json()
        else:
            print(f"Error {response.status_code}: {response.text}")
            return None
            
    except Exception as e:
        print(f"Connection Error: {e}")
        return None

if __name__ == "__main__":
    print(f"Fetching breakout trends for: '{SEED_TOPIC}'...")
    data = get_breakout_trends(SEED_TOPIC)

    if data and "relatedQueries" in data:
        # 1. Rising Queries (Breakout topics)
        # These are terms with the highest growth rate recently
        rising = data["relatedQueries"].get("rising", [])
        print(f"\n--- Rising Queries (The Opportunity) ---")
        for item in rising[:10]: # Top 10 rising
            print(f"[Growth: {item['value']}] {item['query']}")

        # 2. Top Queries (The Base)
        # These are the most popular queries overall
        top = data["relatedQueries"].get("top", [])
        print(f"\n--- Top Queries (The Volume) ---")
        for item in top[:5]:
            print(f"[Index: {item['value']}] {item['query']}")
    else:
        print("No trend data found.")

Rising queries come back with a growth index rather than a volume, which is why the output reads as multipliers.

Output Example

The run prints this:

--- Rising Queries (The Opportunity) ---
[Growth: +3,450%] javvy protein coffee
[Growth: +1,100%] anomalous coffee machine
[Growth: +400%] the coffee boy movie
[Growth: +350%] good coffee great coffee
[Growth: +350%] us coffee market

--- Top Queries (The Volume) ---
[Index: 100] the coffee
[Index: 75] coffee shop
[Index: 54] coffee machine

The script separates “Top” queries (evergreen volume like “coffee shop”) from “Rising” queries (viral intent like “javvy protein coffee”). By automating this retrieval daily, you can detect shifting market interests before they appear in mainstream SEO tools.

To filter these trends by specific regions (geoMap) or clustering concepts (relatedTopics), refer to our guide on How to Scrape Google Trends.

Script 3. The PAA Topic Graph

Standard keyword research tools show you what people type. Google’s “People Also Ask” (PAA) shows you what people want to know.

By scraping PAA questions recursively, we build a Topic Authority Tree. Answering the root question along with its 2nd and 3rd-level derivatives signals deep expertise to the ranking algorithm.

The Code

This script performs a Depth-First Search (DFS) on PAA questions. It accepts a root keyword, extracts the relatedQuestions array from the JSON response, and treats each question as a new search query to find its related questions.

import requests
import json

# Configuration
API_KEY = "HASDATA_API_KEY"
ROOT_KEYWORD = "coffee"
MAX_DEPTH = 2  # How deep to go into the rabbit hole

def get_paa_questions(query):
    """
    Fetches PAA questions for a given query using HasData's Google SERP API.
    """
    url = "https://api.hasdata.com/scrape/google/serp"
    
    # We use 'json' output to get structured 'relatedQuestions' directly
    params = {
        "q": query,
        "location": "United States",
        "deviceType": "desktop",
    }
    
    headers = {
        "x-api-key": API_KEY,
        "Content-Type": "application/json"
    }

    try:
        response = requests.get(url, params=params, headers=headers, timeout=20)
        
        if response.status_code == 200:
            data = response.json()
            # Extract list of questions from the 'relatedQuestions' key
            # Check if key exists to prevent errors on pages without PAA
            questions = [item['question'] for item in data.get("relatedQuestions", [])]
            return questions
        else:
            print(f"Error {response.status_code}: {response.text}")
            return []
            
    except Exception as e:
        print(f"Connection Failed: {e}")
        return []

def build_topic_tree(query, current_depth, max_depth, visited):
    """
    Recursively builds a tree of questions.
    """
    # Avoid infinite loops or reprocessing the same question
    if current_depth > max_depth or query in visited:
        return {}

    print(f"Scanning Level {current_depth}: {query}")
    visited.add(query)
    
    # Get questions for the current query
    paa_list = get_paa_questions(query)
    
    tree = {}
    
    for question in paa_list:
        # Recursion: Treat this question as a new query
        sub_tree = build_topic_tree(question, current_depth + 1, max_depth, visited)
        tree[question] = sub_tree
        
    return tree

def print_tree(tree, level=0):
    for question, sub_questions in tree.items():
        indent = "    " * level
        print(f"{indent}- {question}")
        print_tree(sub_questions, level + 1)

if __name__ == "__main__":
    visited_queries = set()
    print(f"Building PAA Tree for: '{ROOT_KEYWORD}'...\n")
    
    topic_tree = build_topic_tree(ROOT_KEYWORD, 1, MAX_DEPTH, visited_queries)
    
    print(f"\n--- Topic Authority Tree ---")
    print(f"- {ROOT_KEYWORD}")
    print_tree(topic_tree, 1)

The SERP API returns the questions as JSON, so nothing here parses Google’s accordion markup.

Output Example

Running this script for “coffee” generates a hierarchy ready for H2 and H3 tags:

Building PAA Tree for: 'coffee'...

Scanning Level 1: coffee
Scanning Level 2: What is coffee called in Colombia?
Scanning Level 2: Why should over 60s have four coffees a day?
Scanning Level 2: Is it healthy to drink coffee every day?
Scanning Level 2: What are the top 5 best coffees?

--- Topic Authority Tree ---
- coffee
    - What is coffee called in Colombia?
        - How do I order coffee in Colombia?
        - How do you say coffee in Colombia?
        - Are Arabica and Colombian coffee the same?
        - Why is Colombian coffee special?
    - Why should over 60s have four coffees a day?
        - How much coffee is too much for seniors?
        - Is 4 coffees a day unhealthy?
        - Is coffee good for a 70 year old?
        - What is the best coffee for senior citizens?
    …

The output is a semantic graph rather than a flat list. Four questions came back for “coffee” and each of them opened four more, which is the structure an article’s H2 and H3 levels want. The seed matters: Google’s boxes for a bare head term skew toward the odd and regional, so a more specific root (decaf coffee benefits instead of coffee) returns a tree closer to what a content plan needs.

Script 4. SERP Intent Qualifier

“Keyword Difficulty” metrics are often abstract. A keyword may show low difficulty but feature ten enterprise homepages in the top results. In this scenario, a “How-to” blog post will fail because Google has determined the intent is navigational (Branded) rather than informational.

You have to match the SERP intent. If the top results are product pages, an e-commerce page is what ranks. If they’re blogs, you need content.

The Code

This script scrapes the Top 10 organic results and analyzes the URL structure using heuristics (RegEx). It classifies every result into Informational (blogs, guides), Transactional (products, shops), Encyclopedic, Video, UGC, or Navigational (homepages).

import requests
import pandas as pd
from urllib.parse import urlparse

# Configuration
API_KEY = "HASDATA_API_KEY"
KEYWORD = "instant coffee"

def get_serp_links(query):
    url = "https://api.hasdata.com/scrape/google/serp"
    params = {
        "q": query,
        "gl": "us",
        "hl": "en",
        "deviceType": "desktop"
    }
    headers = {"x-api-key": API_KEY}

    try:
        response = requests.get(url, params=params, headers=headers, timeout=20)
        if response.status_code == 200:
            data = response.json()
            # Extract organic links
            return [result['link'] for result in data.get('organicResults', [])]
        return []
    except Exception as e:
        print(f"Error: {e}")
        return []

def classify_url(url):
    """
    Classifies a URL based on domain reputation, path keywords, and structure.
    """
    try:
        parsed = urlparse(url)
        domain = parsed.netloc.lower()
        path = parsed.path.lower()
        
        # 1. Specialized Platforms (Domain-based)
        if "wikipedia.org" in domain:
            return "Encyclopedic (Wikipedia)"
        
        if "youtube.com" in domain or "youtu.be" in domain:
            return "Video Content (YouTube)"
            
        if any(x in domain for x in ['reddit.com', 'quora.com', 'stackoverflow.com']):
            return "UGC (Forum/Discussion)"

        # 2. General Forum Detection (Path-based)
        if any(x in path for x in ['/forum', '/threads', '/community', '/board']):
            return "UGC (Forum/Discussion)"


        # 3. Homepage Detection
        if path == "" or path == "/":
            return "Homepage (Brand)"
            
        # 4. Transactional Keywords
        if any(x in path for x in ['/product', '/shop', '/item', '/collections', '/buy', '/pricing', '/store']) or "amazon.com" in domain:
            return "Transactional (Product)"
            
        # 5. Informational Keywords
        if any(x in path for x in ['/blog', '/guide', '/news', '/article', '/how-to', '/tips', '/wiki']):
            return "Informational (Blog)"
            
        # 6. Fallback (General Page)
        return "General Page"
    except:
        return "Unknown"

if __name__ == "__main__":
    print(f"Analyzing SERP for: '{KEYWORD}'...\n")
    links = get_serp_links(KEYWORD)
    
    if links:
        # Create a DataFrame for clean visualization
        df = pd.DataFrame(links, columns=['URL'])
        df['Type'] = df['URL'].apply(classify_url)
        
        # Calculate percentages
        breakdown = df['Type'].value_counts(normalize=True) * 100
        
        # Display the Data
        print("--- SERP Composition ---")
        print(df[['Type', 'URL']].to_string(index=True))
        
        print("\n--- Strategic Verdict ---")
        # Handle cases where multiple types might have the same top percentage
        if not breakdown.empty:
            dominant_type = breakdown.idxmax()
            percent = breakdown.max()
            
            print(f"Dominant Type: {dominant_type} ({percent:.1f}%)")
            
            if "Informational" in dominant_type:
                print("Action: Create a long-form Guide or Blog Post.")
            elif "Transactional" in dominant_type:
                print("Action: Create a Product Page or Collection Page.")
            elif "Homepage" in dominant_type:
                print("Action: High difficulty. Requires strong Brand Authority.")
            elif "Video" in dominant_type:
                print("Action: Text alone won't rank. Produce a high-quality Video.")
            elif "UGC" in dominant_type:
                print("Action: Engage in community discussions or create 'Real Review' style content.")
            elif "Encyclopedic" in dominant_type:
                print("Action: Definitional intent. Very hard to outrank Wikipedia directly.")
            else:
                print("Action: Mixed SERP. Manual review recommended.")
        else:
            print("Action: No valid classification data available.")
    else:
        print("No results found.")

The classifier keys off the URL shape and the SERP feature mix, so one call per query is enough to see what Google thinks the intent is.

Output Example

The output provides an instant distribution table.

--- SERP Composition ---
                       Type                                                                                 URL
0  Encyclopedic (Wikipedia)                                        https://en.wikipedia.org/wiki/Instant_coffee
1              General Page              https://www.thetakeout.com/1917415/instant-coffee-store-bought-ranked/
2   Transactional (Product)                            https://www.happyproducts.com/collections/instant-coffee
3    UGC (Forum/Discussion)   https://www.reddit.com/r/Coffee/comments/1nblr8x/what_is_the_best_instant_coffee/
4   Transactional (Product)              https://onyxcoffeelab.com/collections/instant-specialty-soluble-coffee
5   Transactional (Product)                                       https://cafely.com/collections/instant-coffee
6   Transactional (Product)          https://www.amazon.com/Best-Sellers-Instant-Coffee/zgbs/grocery/2251594011
7   Transactional (Product)                        https://www.vervecoffee.com/collections/instant-craft-coffee
8              General Page                                https://www.bonappetit.com/story/best-instant-coffee

--- Strategic Verdict ---
Dominant Type: Transactional (Product) (55.6%)
Action: Create a Product Page or Collection Page.

Google defines the rules, and this script reveals them. If 56% of results are transactional, the user wants to buy, not learn. This tool aligns strategy with algorithmic evidence rather than abstract difficulty scores.

Script 5. Keyword Similarity Analyzer by SERP Overlap

Targeting two similar keywords with one page is a risk. Guessing wrong leads to cannibalization or diluted relevance.

The solution is in the SERP itself. If Google ranks the same set of URLs for both queries, the intent is identical. If the results differ, the topics require separate pages.

The Code

This script calculates the Jaccard Index (overlap percentage) between search results for a list of keywords. It visualizes the data as a heatmap for instant clustering decisions.

import requests
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from collections import Counter
import time

# --- CONFIGURATION ---
API_KEY = "HASDATA_API_KEY"
KEYWORDS = [
    "health benefits of coffee",
    "benefits of coffee for men",
    "benefits of coffee for women",
    "health benefits of black coffee",
    "health benefits of mushroom coffee",
    "health benefits of decaf coffee",
]
# ---------------------

def get_top_urls_set(query):
    """
    Requests the API and returns a SET of organic URLs.
    Using a set is convenient for mathematical intersection operations.
    """
    url = "https://api.hasdata.com/scrape/google/serp"
    params = {
        "q": query,
        "gl": "us",
        "hl": "en",
        "deviceType": "desktop"
      }
    headers = {"x-api-key": API_KEY}

    print(f"Scanning SERP for: '{query}'...")
    try:
        response = requests.get(url, params=params, headers=headers, timeout=25)
        if response.status_code == 200:
            data = response.json()
            organic = data.get('organicResults', [])
            # Extract links only. Taking the first 10 if more are returned.
            links = [result['link'] for result in organic[:10] if 'link' in result]
            return set(links)
        else:
            print(f"Error {response.status_code} for query '{query}'")
            return set()
    except Exception as e:
        print(f"Exception for query '{query}': {e}")
        return set()

def calculate_jaccard(set1, set2):
    """Calculates Jaccard Index between two sets."""
    intersection = len(set1.intersection(set2))
    union = len(set1.union(set2))
    if union == 0:
        return 0.0
    return intersection / union

def visualize_similarity_heatmap(similarity_df):
    """Builds and displays the heatmap."""
    plt.figure(figsize=(10, 8))
    sns.set_theme(context='notebook', style='whitegrid', font_scale=1.1)
    
    # Create heatmap
    # annot=True shows numbers in cells
    # fmt=".0%" formats numbers as percentages
    # cmap="YlGnBu" - color scheme (Yellow to Blue)
    ax = sns.heatmap(
        similarity_df, 
        annot=True, 
        fmt=".0%", 
        cmap="YlGnBu", 
        vmin=0, 
        vmax=1,
        cbar_kws={'label': 'Jaccard Similarity Score'}
    )
    
    plt.title(f"SERP Similarity Heatmap", fontsize=16, pad=20)
    plt.xticks(rotation=45, ha='right')
    plt.yticks(rotation=0)
    plt.tight_layout()
    print("Visualizing heatmap...")
    plt.show()

def show_top_common_urls(results_data):
    """Counts and prints the most frequently occurring URLs."""
    all_urls_flat = []
    # Collect all URLs from all sets into one list
    for url_set in results_data.values():
        all_urls_flat.extend(list(url_set))
    
    # Count frequency
    url_counts = Counter(all_urls_flat).most_common(15)
    
    print(f"\n### Top Recurring URLs (across {len(KEYWORDS)} queries)")
    print(f"{'Frequency':<10} | URL")
    print("-" * 80)
    for url, count in url_counts:
        # Show only if it appears more than once, unless we have very few queries
        if count > 1 or len(KEYWORDS) <= 2: 
             print(f"{count:<10} | {url}")      

# --- MAIN LOGIC ---
if __name__ == "__main__":
    # 1. Data Collection
    results_data = {}
    print("--- Start SERP Collection ---\n")
    for keyword in KEYWORDS:
        results_data[keyword] = get_top_urls_set(keyword)
        # Short pause between requests
        time.sleep(1) 
    print("\n--- Collection Finished ---")

    # 2. Calculation of Similarity Matrix (Jaccard)
    n = len(KEYWORDS)
    similarity_matrix = pd.DataFrame(
        index=KEYWORDS, 
        columns=KEYWORDS, 
        dtype=float
    )

    # Double loop to compare every query against every other query
    for i in range(n):
        for j in range(n):
            kw1 = KEYWORDS[i]
            kw2 = KEYWORDS[j]
            score = calculate_jaccard(results_data[kw1], results_data[kw2])
            similarity_matrix.iloc[i, j] = score

    # 3. Output Results to Console (Text Table)
    print("\n### Similarity Matrix (Jaccard Index)")
    pd.set_option('display.max_columns', None)
    pd.set_option('display.width', 1000)
    # Format output as percentages
    print(similarity_matrix.style.format("{:.1%}").to_string())
    print("\n" + "="*50 + "\n")
    
    # 4. Common URL Analysis
    show_top_common_urls(results_data)

    # 5. Visualization (Heatmap)
    visualize_similarity_heatmap(similarity_matrix)

Scaling this script allows you to cluster thousands of keywords into groups automatically.

Output Example

The run prints this:

### Similarity Matrix (Jaccard Index)
 health benefits of coffee benefits of coffee for men benefits of coffee for women health benefits of black coffee health benefits of mushroom coffee health benefits of decaf coffee
health benefits of coffee 100.0% 38.5% 30.8% 18.8% 0.0% 0.0%
benefits of coffee for men 38.5% 100.0% 36.4% 21.4% 0.0% 0.0%
benefits of coffee for women 30.8% 36.4% 100.0% 23.1% 0.0% 0.0%
health benefits of black coffee 18.8% 21.4% 23.1% 100.0% 0.0% 0.0%
health benefits of mushroom coffee 0.0% 0.0% 0.0% 0.0% 100.0% 0.0%
health benefits of decaf coffee 0.0% 0.0% 0.0% 0.0% 0.0% 100.0%

==================================================

### Top Recurring URLs (across 6 queries)
Frequency  | URL
--------------------------------------------------------------------------------
4          | https://www.hopkinsmedicine.org/health/wellness-and-prevention/9-reasons-why-the-right-amount-of-coffee-is-good-for-you
4          | https://www.mayoclinic.org/healthy-lifestyle/nutrition-and-healthy-eating/expert-answers/coffee-and-health/faq-20058339
3          | https://www.nhlbi.nih.gov/news/2025/when-it-comes-health-benefits-coffee-timing-may-count
3          | https://www.rush.edu/news/health-benefits-coffee
3          | https://www.healthline.com/nutrition/top-evidence-based-health-benefits-of-coffee
2          | https://nutritionsource.hsph.harvard.edu/food-features/coffee/

Visualizing heatmap...

The matrix reads better as a picture:

“Health benefits of coffee” overlaps with “benefits for men” at 38.5% and “benefits for women” at 30.8%. Google is answering them with largely the same pages, which is the case for one article rather than three.

Conversely, “Mushroom coffee” and “decaf coffee” have a 0% overlap. They share no URLs with the main cluster. These topics require dedicated, separate articles.

This approach removes the guesswork from site architecture.

Script 6. Content Gap Analyzer with N-Grams

Search algorithms rely on vector space models to determine document relevance. If top-ranking results share a vocabulary your document lacks, you have a semantic distance problem.

Manual gap analysis is subjective. We solve this by treating the SERP as a training corpus. By vectorizing competitor content and comparing it against the target URL using N-Gram analysis, we mathematically define the “market vocabulary.”

The Code

This script runs Bag-of-Words vectorization and frequency analysis with scikit-learn. It sidesteps raw HTML parsing by handing the page to trafilatura, which returns the main body text with the ads and navigation already gone.

The algorithm constructs a feature matrix based on competitor consensus (Document Frequency). It then projects your content onto this matrix to identify missing n-grams. This process highlights precise semantic concepts absent from your content but present across the market leaders.

import trafilatura
import requests
import pandas as pd
import json
import re
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import ENGLISH_STOP_WORDS

# --- CONFIGURATION ---
API_KEY = "HASDATA_API_KEY"
TARGET_KEYWORD = "python web scraping"
MY_URL = "https://hasdata.com/blog/web-scraping-with-python"
TOP_N_COMPETITORS = 10

# Extend standard stop words to filter out conversational noise and contractions
CUSTOM_STOP_WORDS = list(ENGLISH_STOP_WORDS) + ['ve', 'll', 're', 'don', 'won', 't', 's', 'm', 'd']

def get_serp_links(query):
    """Retrieves top organic search results to establish the competitor baseline."""
    url = "https://api.hasdata.com/scrape/google/serp"
    try:
        response = requests.get(url, params={"q": query, "gl": "us", "hl": "en", "deviceType": "desktop", "num": TOP_N_COMPETITORS}, headers={"x-api-key": API_KEY}, timeout=20)
        return [res['link'] for res in response.json().get('organicResults', [])] if response.status_code == 200 else []
    except Exception:
        return []

def scrape_content(target_url):
    """
    Extracts main content from a URL using Trafilatura for DOM parsing.
    Routes requests through HasData's proxy pool.
    """
    api_url = "https://api.hasdata.com/scrape/web"
    # Compact payload for brevity
    payload = {"url": target_url, "proxyType": "datacenter", "proxyCountry": "US", "jsRendering": True, "outputFormat": ["html"]}
    headers = {'x-api-key': API_KEY, 'Content-Type': 'application/json'}

    try:
        response = requests.post(api_url, data=json.dumps(payload), headers=headers, timeout=60)
        if response.status_code != 200:
            print(f"   Fetch failed for {target_url}: HTTP {response.status_code}")
            return ""
        # Trafilatura is superior to newspaper3k for extracting core text from modern DOMs
        text = trafilatura.extract(response.text, include_comments=False, include_tables=False, fast=True)
        return text if text else ""
    except Exception as e:
        print(f"Scrape Error: {e}")
        return ""

def analyze_ngrams_sklearn(my_text, competitor_texts, n_gram_range=(1, 1)):
    """
    Performs N-Gram analysis using a TF-IDF style approach (CountVectorizer).
    1. Fits vocabulary on the Competitor Corpus (Market Standard).
    2. Transforms Target Text against that vocabulary to identify gaps.
    """
    if not competitor_texts: return pd.DataFrame()

    # Establish significance threshold: phrase must appear in at least 2 competitors (unless corpus is small)
    min_freq = 2 if len(competitor_texts) >= 3 else 1
   
    vec = CountVectorizer(ngram_range=n_gram_range, stop_words=CUSTOM_STOP_WORDS, min_df=min_freq)
   
    try:
        # Generate feature matrix for competitors
        X_comp = vec.fit_transform(competitor_texts)
    except ValueError:
        return pd.DataFrame() # Handle empty vocabulary cases

    feature_names = vec.get_feature_names_out()
   
    # Binarize occurrence (document frequency) rather than raw term frequency
    competitors_count = np.asarray((X_comp > 0).astype(int).sum(axis=0)).ravel()
   
    # Map target content against the established market vocabulary
    my_counts = vec.transform([my_text]).toarray()[0] if my_text else [0] * len(feature_names)

    # Aggregate and sort by market popularity
    df = pd.DataFrame({'Phrase': feature_names, 'Competitors_Count': competitors_count, 'My_Count': my_counts})
    return df.sort_values(by=['Competitors_Count', 'Phrase'], ascending=[False, True])

def run_analysis():
    # 1. Competitor Discovery
    competitor_urls = get_serp_links(TARGET_KEYWORD)
    if not competitor_urls:
        print("No competitors found via SERP API.")
        return

    print(f"\n--- CONTENT EXTRACTION & ANALYSIS ---")
   
    # 2. Target Site Extraction (Normalized Logging)
    my_text = scrape_content(MY_URL)
    my_text = re.sub(r'\s+', ' ', my_text).strip()
    my_wc = len(my_text.split())
   
    status_icon = "✅" if my_wc > 100 else "⚠️"
    print(f"{status_icon} [TARGET] {my_wc} words | {MY_URL}")

    # 3. Competitor Extraction
    competitor_texts = []
    print(f"\nScanning {len(competitor_urls)} competitors...")
   
    for url in competitor_urls:
        if url == MY_URL: continue
       
        txt = scrape_content(url)
        txt = re.sub(r'\s+', ' ', txt).strip()
        wc = len(txt.split())
       
        if wc > 200:
            print(f"✅ [COMP]   {wc} words | {url}")
            competitor_texts.append(txt)
        else:
            print(f"⚠️ [SKIP]   {wc} words | {url} (Insufficient content)")

    total_competitors = len(competitor_texts)
    if total_competitors == 0:
        print("No valid competitor content available for analysis.")
        return

    print(f"\nAnalyzing gaps against {total_competitors} validated competitors...\n")

    # 4. N-Gram Calculation
    dfs = {
        "MISSING KEYWORDS (1-Gram)": analyze_ngrams_sklearn(my_text, competitor_texts, (1, 1)),
        "MISSING PHRASES (2-Grams)": analyze_ngrams_sklearn(my_text, competitor_texts, (2, 2)),
        "MISSING LONG-TAIL (3-Grams)": analyze_ngrams_sklearn(my_text, competitor_texts, (3, 3))
    }

    # 5. Reporting
    print("="*60 + "\nGAP ANALYSIS REPORT\n" + "="*60)

    if my_wc <= 50:
        print("❌ Critical: Target site content could not be parsed. Analysis aborted.")
        return

    for title, df in dfs.items():
        if df.empty: continue
       
        # Filter for gaps (Frequency on target = 0) and take top 10
        subset = df[df['My_Count'] == 0].head(10)
       
        if subset.empty:
            print(f"\n### {title}: No significant gaps detected.")
            continue

        print(f"\n### {title}")
        print(f"{'Phrase':<30} | {'Competitors':<15} | {'My Count':<10}")
        print("-" * 65)
        for _, row in subset.iterrows():
            print(f"{row['Phrase']:<30} | {row['Competitors_Count']} of {total_competitors:<10} | ❌")

    print("\n" + "="*60 + "\nSHARED TERMS OVERVIEW\n" + "="*60)
    # Show terms where we align with competitors (using 1-gram data as proxy)
    shared = dfs["MISSING KEYWORDS (1-Gram)"]
    shared_subset = shared[shared['My_Count'] > 0].head(5)
   
    if not shared_subset.empty:
        for _, row in shared_subset.iterrows():
            print(f"{row['Phrase']:<30} | {row['Competitors_Count']} of {total_competitors:<10} | {row['My_Count']}")

if __name__ == "__main__":
    run_analysis()

The word-count floor filters out video and forum results, which otherwise drag the N-gram baseline toward nothing.

Output Example

The run prints this:

--- CONTENT EXTRACTION & ANALYSIS ---
✅ [TARGET] 5081 words | https://hasdata.com/blog/web-scraping-with-python

Scanning 9 competitors...
✅ [COMP]   6874 words | https://realpython.com/python-web-scraping-practical-introduction/
⚠️ [SKIP]   41 words | https://www.reddit.com/r/learnpython/comments/qzr8ir/how_to_start_web_scraping_with_python/ (Insufficient content)
✅ [COMP]   679 words | https://scrapy.org/
⚠️ [SKIP]   24 words | https://www.youtube.com/watch?v=DcI_AZqfZVc (Insufficient content)
✅ [COMP]   1657 words | https://www.geeksforgeeks.org/python/python-web-scraping-tutorial/
✅ [COMP]   395 words | https://guides.lib.utexas.edu/web-scrapping/scraping-with-python
✅ [COMP]   3609 words | https://hbs-rcs.github.io/hbsgrid-docs/tutorials/PythonWebScrape/
⚠️ [SKIP]   74 words | https://www.youtube.com/watch?v=8dTpNajxaH0 (Insufficient content)
⚠️ [SKIP]   52 words | https://github.com/scrapy/scrapy (Insufficient content)

Analyzing gaps against 5 validated competitors...

============================================================
GAP ANALYSIS REPORT
============================================================

### MISSING KEYWORDS (1-Gram)
Phrase                         | Competitors     | My Count  
-----------------------------------------------------------------
easier                         | 5 of 5          | ❌
making                         | 5 of 5          | ❌
used                           | 5 of 5          | ❌
automate                       | 4 of 5          | ❌
collect                        | 4 of 5          | ❌
…

### MISSING PHRASES (2-Grams)
Phrase                         | Competitors     | My Count  
-----------------------------------------------------------------
parse html                     | 4 of 5          | ❌
content html                   | 3 of 5          | ❌
html page                      | 3 of 5          | ❌
html using                     | 3 of 5          | ❌
lxml library                   | 3 of 5          | ❌
…

### MISSING LONG-TAIL (3-Grams)
Phrase                         | Competitors     | My Count  
-----------------------------------------------------------------
beautifulsoup parse html       | 2 of 5          | ❌
browser chrome firefox         | 2 of 5          | ❌
choice web scraping            | 2 of 5          | ❌
decode utf print               | 2 of 5          | ❌
extracting data websites       | 2 of 5          | ❌
…

============================================================
SHARED TERMS OVERVIEW
============================================================
built                          | 5 of 5          | 7
data                           | 5 of 5          | 21
http                           | 5 of 5          | 14
python                         | 5 of 5          | 49
query                          | 5 of 5          | 5

Four of the nine competitor URLs dropped out on the word-count floor, which is what those [SKIP] lines are. Two YouTube pages, a Reddit thread and a GitHub repo page don’t carry an article body to compare against. The five that remain expose the gap. The target page covers the topic and the shared-terms block confirms the core vocabulary, while the missing lists name mechanics competitors spell out and this one doesn’t, parse html, lxml library, beautifulsoup parse html.

Script 7. AI Overview Visibility Monitor

Rank trackers are often blind to AI Overviews. You may rank #1 organically yet lose traffic to an AI summary citing a competitor. This creates a “Phantom Traffic Loss” scenario where rankings remain stable but clicks evaporate.

The Code

This script audits SGE visibility by parsing the aiOverview object from the SERP API. It validates target domain presence within the citation array to derive two KPIs: AI Coverage (trigger frequency) and Citation Share of Voice.

import requests
import pandas as pd
import time
from urllib.parse import urlparse

# --- CONFIGURATION ---
API_KEY = "HASDATA_API_KEY"
TARGET_DOMAIN = "webmd.com" # The domain we are monitoring for citations

KEYWORDS = [
    "health benefits of coffee",
    "benefits of coffee for men",
    "benefits of coffee for women",
    "health benefits of black coffee",
    "health benefits of mushroom coffee",
    "health benefits of decaf coffee"
]

def normalize_domain(url):
    """
    Extracts the base domain from a URL to ensure accurate matching.
    e.g., 'https://www.sub.example.com/page' -> 'sub.example.com'
    """
    try:
        parsed = urlparse(url)
        # Returns netloc (e.g., www.example.com). We strip 'www.' for broader matching.
        return parsed.netloc.lower().replace('www.', '')
    except Exception:
        return ""

def fetch_serp_data(query):
    """
    Executes a SERP API request targeting Google's AI Overview.
    Note: AI Overviews are volatile; successful triggering depends on
    location (US) and device type (Desktop/Mobile).
    """
    endpoint = "https://api.hasdata.com/scrape/google/serp"
    params = {
        "q": query,
        "gl": "us",       # Geo-location: US is critical for SGE consistency
        "hl": "en",       # Language: English
        "deviceType": "desktop"
    }
    headers = {"x-api-key": API_KEY}
   
    try:
        response = requests.get(endpoint, params=params, headers=headers, timeout=30)
        if response.status_code == 200:
            return response.json()
        else:
            print(f"[API Error] Status: {response.status_code} for query '{query}'")
            return None
    except Exception as e:
        print(f"[Network Error] {e}")
        return None

def extract_ai_citations(serp_data):
    """
    Parses the JSON response to locate the 'aiOverview' block.
    Extracts structured references (citations) if present.
    """
    # HasData typically returns SGE data in the 'aiOverview' key
    ai_overview = serp_data.get('aiOverview')
   
    if not ai_overview:
        return None, [] # AI Overview not triggered for this query

    # Extract the 'references' array which contains the source links
    references = ai_overview.get('references', [])
   
    # Map into a cleaner format for analysis
    citations = []
    for ref in references:
        citations.append({
            'index': ref.get('index'),
            'title': ref.get('title'),
            'url': ref.get('link'),
            'source_name': ref.get('source')
        })
       
    return ai_overview, citations

def run_monitor():
    results = []
    print(f"--- Starting AI Overview Monitor for: {TARGET_DOMAIN} ---")
    print(f"Processing {len(KEYWORDS)} keywords...\n")

    for query in KEYWORDS:
        print(f"Analyzing: '{query}'...")
       
        data = fetch_serp_data(query)
       
        if not data:
            continue

        # 1. Check for AI Overview existence
        ai_block, citations = extract_ai_citations(data)
       
        is_triggered = ai_block is not None
        is_cited = False
        citation_rank = None
        found_url = None
       
        # 2. Analyze Citations if AI Overview exists
        if is_triggered:
            target_clean = normalize_domain(f"https://{TARGET_DOMAIN}")
           
            for cit in citations:
                cited_domain = normalize_domain(cit['url'])
               
                # Check for substring match (handles subdomains and exact matches)
                if target_clean in cited_domain:
                    is_cited = True
                    citation_rank = cit['index'] # 0-based index in the citation carousel
                    found_url = cit['url']
                    break # Stop after finding the first occurrence
       
        # 3. Aggregate Data
        results.append({
            "Keyword": query,
            "AI Triggered": is_triggered,
            "Is Cited": is_cited,
            "Citation Index": citation_rank if is_cited else "-",
            "Cited URL": found_url if is_cited else "-"
        })
       
        # Respect API rate limits
        time.sleep(1)

    # --- REPORTING ---
    df = pd.DataFrame(results)
   
    print("\n" + "="*60)
    print("AI OVERVIEW VISIBILITY REPORT")
    print("="*60)
   
    # formatting booleans for readability
    df['AI Triggered'] = df['AI Triggered'].map({True: '✅ Yes', False: '❌ No'})
    df['Is Cited'] = df['Is Cited'].map({True: '✅ YES', False: '❌ No'})
   
    # Handle display if dataframe is empty
    if not df.empty:
        # Use to_markdown if available, otherwise string
        try:
            print(df.to_markdown(index=False))
        except ImportError:
            print(df.to_string(index=False))
           
        # Summary Metrics
        total = len(df)
        triggered = len(df[df['AI Triggered'] == '✅ Yes'])
        cited = len(df[df['Is Cited'] == '✅ YES'])
       
        print("\n--- SUMMARY METRICS ---")
        print(f"AI Coverage: {triggered}/{total} keywords ({(triggered/total)*100:.1f}%)")
        if triggered > 0:
            print(f"Share of Voice: {cited}/{triggered} AI Overviews ({(cited/triggered)*100:.1f}%)")
        else:
            print("Share of Voice: N/A (No AI Overviews generated)")
           
    else:
        print("No data collected.")

if __name__ == "__main__":
    run_monitor()

The AI Overview block comes back inside the same SERP response, so one call per query covers both the ranking check and the citation check.

Output Example

The run prints this:

============================================================
AI OVERVIEW VISIBILITY REPORT
============================================================
| Keyword                            | AI Triggered   | Is Cited   | Citation Index   | Cited URL                                                                                                                                                                                  |
|:-----------------------------------|:---------------|:-----------|:-----------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| health benefits of coffee          | ✅ Yes         | ❌ No      | -                | -                                                                                                                                                                                          |
| benefits of coffee for men         | ✅ Yes         | ✅ YES     | 11               | https://www.webmd.com/diet/health-benefits-black-coffee#:~:text=Coffee%20is%20rich%20in%20several,antioxidants%20in%20most%20people's%20diets.                                             |
| benefits of coffee for women       | ✅ Yes         | ❌ No      | -                | -                                                                                                                                                                                          |
| health benefits of black coffee    | ✅ Yes         | ✅ YES     | 1                | https://www.webmd.com/diet/health-benefits-black-coffee#:~:text=Research%20shows%20that%20drinking%20coffee,builds%20up%20in%20your%20blood.                                               |
| health benefits of mushroom coffee | ✅ Yes         | ✅ YES     | 1                | https://www.webmd.com/diet/mushroom-coffee-health-benefits#:~:text=These%20have%20major%20antioxidant%20properties,to%20back%20this%20claim%20up.                                          |
| health benefits of decaf coffee    | ✅ Yes         | ✅ YES     | 1                | https://www.webmd.com/diet/what-to-know-decaf-coffee#:~:text=doctor%20allows%20it.-,What%20Are%20the%20Benefits%20of%20Drinking%20Decaf%20Coffee?,best%20option%20for%20health%20benefits. |

--- SUMMARY METRICS ---
AI Coverage: 6/6 keywords (100.0%)
Share of Voice: 4/6 AI Overviews (66.7%)

This provides the feedback loop missing from Search Console. High AI Trigger rates combined with low Citation Share indicates your content lacks the “Liftability” (clear definitions and direct answers) required for LLM extraction.

The Seven Scripts Side by Side

Seven jobs, and the SEO automation each one replaces, with what a run costs:

Script 1 is the only one whose cost scales with a constant you set, since the query count is alphabet-driven. The rest cost one or two calls per input.

Get the Seven Scripts from GitHub

All seven scripts live in one free repository, HasData/python-for-seo: google_suggest_harvester.py, trends_breakout_analyzer.py, paa_tree_builder.py, serp_intent_classifier.py, serp_similarity_matrix.py, content_gap_analyzer.py, and ai_overview_monitor.py, plus a requirements.txt and a seo_manager.py runner that calls any of them from one entry point. Clone it, add your API key to the environment, and the scripts run as published here.

Conclusion

Taken together, the scripts above are the backend of a headless SEO platform you own.

Enterprise suites bill for a UI, the overhead behind it, and features most teams never open. Python plus a SERP API covers the part that produces the deliverables, rank tracking, intent analysis, and AI Overview monitoring, and the code stays yours to change when the SERP does.

Signing up gets 1,000 free credits without a card, which covers the smaller scripts here several times over and one depth-2 harvest with room to spare.