Web Scraping with Axios and Cheerio in Node.js

axios fetches HTML and Cheerio parses it with CSS selectors. Together they cover static page scraping without launching a browser.

Static means the data is in the HTML the server sends back. Press Ctrl+U in Chrome on your target page and search for any text you see on screen. If it is there, axios and Cheerio will find it. If it only appears after JavaScript runs, Cheerio parses the empty shell and any selector returns nothing.

You need Node 22+ and basic async/await.

Setting up the project

Create a new folder, initialize a module-type package, and install both libraries.

mkdir cheerio-scraper && cd cheerio-scraper
npm init -y && npm pkg set type=module
npm install axios cheerio

npm pkg set type=module writes "type": "module" to package.json. Without it, import statements throw SyntaxError: Cannot use import statement outside a module and the project defaults to CommonJS.

Fetching HTML with axios

axios.get(url) returns a promise that resolves to a response object. The HTML string is at response.data.

import axios from 'axios';

const { data: html } = await axios.get('https://books.toscrape.com');
console.log(html.slice(0, 300));

On an open site that is enough. Most real targets reject the default axios/1.x.x User-Agent on contact. A browser-realistic header set fixes that.

import axios from 'axios';

const headers = {
  'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 15_7_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36',
  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
  'Accept-Language': 'en-US,en;q=0.9',
  'Accept-Encoding': 'gzip, deflate, br',
};

const { data: html } = await axios.get('https://books.toscrape.com', { headers });

Copy the User-Agent string from Chrome’s DevTools Network tab, which shows exactly what a real browser sends on each request. Chrome ships a new major version every four to six weeks, so a version number from six months ago is a detection signal.

Timeouts

axios has a built-in timeout option in milliseconds. When the request exceeds it, axios rejects with code ECONNABORTED.

const { data: html } = await axios.get(url, { headers, timeout: 10_000 });

Unlike the built-in fetch, which resolves on any HTTP response, axios throws on 4xx and 5xx status codes. axios sets error.response when the server replied with an error. When it is undefined, the request failed at the network level (timeout, DNS failure, or connection refused).

async function fetchHtml(url) {
  try {
    const { data } = await axios.get(url, { headers, timeout: 10_000 });
    return data;
  } catch (error) {
    if (error.response) {
      throw new Error(`HTTP ${error.response.status} for ${url}`);
    }
    throw error;
  }
}

Retries on transient failures

A 429 (rate-limited) or 503 (overloaded) usually clears on retry. A 404 is a real error. The URL is wrong and waiting changes nothing. Exponential backoff with a status check separates the two cases.

async function fetchWithRetry(url, retries = 3) {
  for (let attempt = 1; attempt <= retries; attempt++) {
    try {
      const { data } = await axios.get(url, { headers, timeout: 10_000 });
      return data;
    } catch (error) {
      const status = error.response?.status;
      if (attempt === retries || ![429, 503, 504].includes(status)) throw error;
      await new Promise(r => setTimeout(r, Math.min(1000 * 2 ** (attempt - 1), 8000)));
    }
  }
}

The delay starts at 1 second, doubles each retry, and caps at 8 seconds. Any status outside [429, 503, 504] (including 404 and network timeouts) throws immediately.

Parsing HTML with Cheerio

cheerio.load(html) returns a $ function that works like jQuery.

import * as cheerio from 'cheerio';

const $ = cheerio.load(html);
const heading = $('h1').text().trim();
console.log(heading); // "All products"

.map() iterates each matching element. Chain .get() at the end to get a plain JavaScript array.

const titles = $('article.product_pod h3 a')
  .map((_, el) => $(el).attr('title'))
  .get();

console.log(titles.slice(0, 3));
// ["A Light in the Attic", "Tipping the Velvet", "Soumission"]

.text() returns the joined text of an element and all its descendants. Call .trim() to strip the whitespace that HTML commonly leaves around tag content.

.attr('name') reads any attribute and returns undefined when the attribute is absent. Relative URLs need new URL(href, baseUrl).href to become absolute.

const priceText = $(el).find('.price_color').text().trim(); // "£51.77"
const href     = $(el).find('h3 a').attr('href');          // "catalogue/a-light..."
const fullUrl  = new URL(href, 'https://books.toscrape.com').href;

Missing elements return an empty selection. $(el).find('.maybe').text() returns '' rather than throwing. Check .length to test whether an element exists.

CSS selector patterns

Cheerio supports the same selectors a browser does. Right-click any element in the DevTools Elements panel and choose Copy selector to get a working selector without guessing.

The CSS selectors cheat sheet covers :nth-child, :not, attribute operators, and other patterns that come up with real site markup.

A complete scraper

Here is a full script that walks the books.toscrape.com catalog, normalizes the data, and writes JSON.

fetchHtml handles the request. extractBooks maps over each product card. It converts the price string to a number, the rating word to 1–5, and the relative link to an absolute URL. nextPageUrl reads the pagination link, and the while loop walks pages until it disappears.

// scraper.js
import axios from 'axios';
import * as cheerio from 'cheerio';
import { writeFileSync } from 'node:fs';

const BASE = 'https://books.toscrape.com/';
const RATINGS = { One: 1, Two: 2, Three: 3, Four: 4, Five: 5 };
const headers = {
  'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 15_7_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36',
  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
};

async function fetchHtml(url) {
  const { data } = await axios.get(url, { headers, timeout: 10_000 });
  return data;
}

function extractBooks($) {
  return $('article.product_pod').map((_, el) => {
    const ratingWord = ($(el).find('.star-rating').attr('class') || '')
      .replace('star-rating ', '');
    const priceText = $(el).find('.price_color').text().trim();
    const href = $(el).find('h3 a').attr('href');
    return {
      title:  $(el).find('h3 a').attr('title'),
      price:  Number(priceText.replace(/[^\d.]/g, '')),
      rating: RATINGS[ratingWord] ?? null,
      url:    new URL(href, BASE).href,
    };
  }).get();
}

function nextPageUrl($, currentUrl) {
  const href = $('li.next a').attr('href');
  return href ? new URL(href, currentUrl).href : null;
}

const allBooks = [];
let url = BASE;

while (url) {
  const html = await fetchHtml(url);
  const $ = cheerio.load(html);
  allBooks.push(...extractBooks($));
  url = nextPageUrl($, url);
}

writeFileSync('books.json', JSON.stringify(allBooks, null, 2));
console.log(`Saved ${allBooks.length} books`);

Run with node scraper.js. I got all 1,000 books across 50 pages in about 16 seconds. Wrapping fetchHtml calls in p-limit(5) drops that to around 2.5 seconds by running five pages concurrently.

When Cheerio returns nothing

If a selector returns an empty collection on a page where the data is clearly visible in the browser, the page is JS-rendered. The HTML axios received contains the container but not the content. The page fills it from JavaScript after load.

Check the Network tab in DevTools, filtered to Fetch/XHR, and reload the page. Most dynamic sites call a JSON API on boot and render from that response. Hitting that endpoint with axios directly is about 10x faster than launching Playwright, since it skips browser startup and page render entirely. The Node.js scraping guide covers how to find those endpoints with DevTools.

When there is no JSON shortcut and a browser is required, anti-bot systems like Cloudflare and DataDome raise the difficulty further. They inspect TLS fingerprints and JavaScript globals that axios with realistic headers cannot replicate. For those targets, HasData’s Web Scraping API handles rendering and proxy rotation without a local Playwright setup.

Using the Web Scraping API

The API accepts a URL and extraction rules and returns structured data regardless of whether the page is static or JS-rendered.

Sign up on HasData and copy your API key. This example scrapes article titles from medium.com.

import axios from 'axios';

const response = await axios.post(
  'https://api.hasdata.com/scrape/web',
  {
    url: 'https://medium.com/',
    extractRules: { Title: 'h2' },
  },
  {
    headers: {
      'x-api-key': process.env.HASDATA_API_KEY,
      'Content-Type': 'application/json',
    },
  }
);

console.log(response.data.extractedData.Title);

The structured data is at response.data.extractedData. To find the right selector for extractRules, open DevTools on the target page, right-click the element, choose Inspect, then right-click the highlighted node and choose Copy selector.

The HasData dashboard also has a request builder that generates ready-made code for any URL you paste in.

The builder outputs ready-to-run code you can paste directly into a Node.js script.

fetch vs axios in 2026

Built-in fetch is available in Node 22 with no installation. It cold-starts 4x faster than axios. We tested cold-start on Node 22.18.0 in the Node.js fetch article and got 109 ms for fetch, 469 ms for axios. For a new project without specific need for axios features, fetch is worth considering from the start.

The Cheerio integration is identical either way. Swap axios.get() for fetch() and read the body as text.

const res = await fetch(url, { headers });
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const html = await res.text();
const $ = cheerio.load(html);

fetch resolves on any HTTP response including 4xx and 5xx, so res.ok must be checked manually. axios throws on non-2xx responses by default, which is why the fetchWithRetry function above reads the status from error.response rather than the resolved value.

Use axios when the project already depends on it, when you need request interceptors or upload progress tracking, or when you want automatic JSON parsing. For scraping work that is just GET requests returning HTML, both clients work identically with Cheerio.