Web Scraping MCP Server

Q: What formats can the page come back in?

Set outputFormat to markdown, text, html or json. Markdown is the one agents digest best. Ask for several and they arrive together in one response.

Q: What is the difference between extractRules and aiExtractRules?

extractRules take CSS selectors and return exactly what they match, the same way every time. aiExtractRules take a field name and a description and let a model pull the value out of the HTML, which suits pages whose markup you do not know in advance.

Connect your AI agents to HasData's Web Scraping MCP server. Scrape any public page as markdown, rendered HTML, or CSS-extracted fields in structured JSON over streamable HTTP.

https://mcp.hasdata.com/api/mcp?apis=web_scraping

Transport: Streamable HTTP
Auth: OAuth or API key
Tools: 1 Web Scraping tool
Price: 1 credit / call

Get API Key Read the docs 1,000 free credits. No credit card required.

Data teams at

Playground

See the Web Scraping MCP server in action

Select a sample prompt to inspect the tool it chooses and the JSON payload your agent receives.

Fetch docs.hasdata.com/mcp-server as markdown

Web Scraping · 1 tool Enter to send

Fetch docs.hasdata.com/mcp-server as markdown

web_scraping · 200 OK sample call

The page comes back as markdown, about 7,500 characters, headings intact.

# MCP Server

## Endpoint

## Authentication

## Choosing Which APIs to Expose

## Billing

The site navigation and search box are in there too. Trim to the first h1 if you only want the article.

web_scraping { "url": "https://docs.hasdata.com/mcp-server", "outputFormat": ["markdown"] }

{

"url": "https://api.hasdata.com/scrape/web",

"status": 200,

"json": null,

"text": "# MCP Server\n\n## Endpoint\n\nhttps://mcp.hasdata.com/mcp\n\n## Authentication\n\n…"

}

Fetch docs.hasdata.com/mcp-server as markdown

web_scraping · 200 OK sample call

The page comes back as markdown, about 7,500 characters, headings intact.

# MCP Server

## Endpoint

## Authentication

## Choosing Which APIs to Expose

## Billing

The site navigation and search box are in there too. Trim to the first h1 if you only want the article.

web_scraping { "url": "https://docs.hasdata.com/mcp-server", "outputFormat": ["markdown"] }

{

"url": "https://api.hasdata.com/scrape/web",

"status": 200,

"json": null,

"text": "# MCP Server\n\n## Endpoint\n\nhttps://mcp.hasdata.com/mcp\n\n## Authentication\n\n…"

}

Pull the story titles and points from the Hacker News front page

web_scraping · 200 OK sample call

Thirty titles and their scores, as two arrays.

Gemini 3.8 Flash and 3.8 Flash Cyber · 1,031 points

Muse Spark 1.3 · 586 points

Pre-Release of Polars 2.0 · 156 points

The Browser's Main Thread Is Expensive · 68 points

Note: One newly submitted story does not have a point score yet.

web_scraping { "url": "https://news.ycombinator.com/", "extractRules": { "titles": ".titleline > a", "points": ".score" } }

{

"url": "https://api.hasdata.com/scrape/web",

"status": 200,

"json": {

"requestMetadata": { "id": "cb3a1268-e6f2-4b55-a118-a3532c0603a4", "status": "ok" },

"extractedData": {

"titles": ["Pre-Release of Polars 2.0", "The Browser's Main Thread Is Expensive", "Muse Spark 1.3", "…"],

"points": ["156 points", "68 points", "586 points", "…"]

}

Get the title, price and stock status from books.toscrape.com/catalogue/a-light-in-the-attic_1000

web_scraping · 200 OK sample call

A Light in the Attic · £51.77 · In stock (22 available)

web_scraping { "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html", "aiExtractRules": { "title": { "type": "string" }, "price": { "type": "string", "description": "price with currency" }, "availability": { "type": "string", "description": "stock status" } } }

{

"url": "https://api.hasdata.com/scrape/web",

"status": 200,

"json": {

"requestMetadata": { "id": "6482bee8-5597-4f22-b475-0fa92063c384", "status": "ok" },

"aiResponse": {

"title": "A Light in the Attic",

"price": "£51.77",

"availability": "In stock (22 available)"

}

Scrape the quotes on quotes.toscrape.com/js, which only appear once JavaScript runs

web_scraping · 200 OK sample call

Ten quotes rendered in a headless browser, each matched to its author.

Albert Einstein · “The world as we have created it is a process of our thinking…”

J.K. Rowling · “It is our choices, Harry, that show what we truly are…”

Albert Einstein · “There are only two ways to live your life…”

Quotes loaded successfully via JavaScript rendering.

web_scraping { "url": "https://quotes.toscrape.com/js/", "jsRendering": true, "waitFor": ".quote", "extractRules": { "quotes": ".quote .text", "authors": ".quote .author" } }

{

"url": "https://api.hasdata.com/scrape/web",

"status": 200,

"json": {

"requestMetadata": { "id": "f705b024-f51f-4334-83c3-3cbb58a9d955", "status": "ok" },

"extractedData": {

"quotes": ["“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”", "…"],

"authors": ["Albert Einstein", "J.K. Rowling", "Albert Einstein", "…"]

}

Tools

The Web Scraping tool your agent can call

1 tool

Typed parameters, required flags, and structured JSON fields for each tool. Fetch a page through a datacenter or residential proxy in the country you choose, render it in a headless browser, wait for a selector, and get back markdown, cleaned HTML, or the fields your CSS and AI rules describe.

+ hasdata_web_scraping_web_scraping_scrapeWebPage Any URL, in the shape you ask for 9 params 1 credit

Try

Parameters of hasdata_web_scraping_web_scraping_scrapeWebPage
Parameter	Type	Required	Example
url	string	required	"https://news.ycombinator.com/"
outputFormat	array	optional	["markdown"]
jsRendering	boolean	optional	true
waitFor	string	optional	".quote"
extractRules	object	optional	{ "titles": ".titleline > a" }
aiExtractRules	object	optional	{ "price": { "type": "string" } }
proxyType	enum	optional	"residential"
proxyCountry	enum	optional	"DE"
screenshot	boolean	optional	true

response

requestMetadata
content
text
headers
extractedData
aiResponse

Base calls cost 1 credit. Failed calls are not billed.

Connect

One endpoint, any MCP client

A config block for every major MCP client, in its native format. Authenticate with browser OAuth or an x-api-key header.

Claude Code

zsh — hasdata-web-scraping

$claude mcp add --transport http hasdata-web-scraping \

> https://mcp.hasdata.com/api/mcp?apis=web_scraping \

> --header "x-api-key: YOUR_API_KEY"

· One line, no restart. Run /mcp to confirm 1 tool.

Check the tools are thereIn the session

/mcp

hasdata-web-scraping · connected · 1 tool

Cursor

~/.cursor/mcp.json

{

"mcpServers": { "hasdata-web-scraping": {

"url": "https://mcp.hasdata.com/api/mcp?apis=web_scraping",

"headers": { "x-api-key": "YOUR_API_KEY" }

} }

}

· Add this entry to ~/.cursor/mcp.json. Servers already in the file stay as they are.

· Project-level config also works at .cursor/mcp.json

Check the tools are thereAfter saving the file

Settings → MCP

hasdata-web-scraping · 1 tool enabled

Claude Desktop

Recommended · OAuth connector

1.Settings → Connectors → Add custom connector

2.Paste https://mcp.hasdata.com/api/mcp?apis=web_scraping

3.Sign in with HasData

~/Library/Application Support/Claude/claude_desktop_config.json

{

"mcpServers": { "hasdata-web-scraping": {

"command": "npx",

"args": ["-y", "mcp-remote", "https://mcp.hasdata.com/api/mcp?apis=web_scraping",

"--header", "x-api-key:YOUR_API_KEY"]

} }

}

· Add this entry to ~/Library/Application Support/Claude/claude_desktop_config.json. Servers already in the file stay as they are.

· The config file only takes stdio servers, so a static key rides the mcp-remote bridge. On Windows the file lives at %APPDATA%\Claude\claude_desktop_config.json.

Check the tools are thereReopen the app

Settings → Connectors

hasdata-web-scraping · connected · 1 tool

VS Code / Copilot

.vscode/mcp.json

{

"servers": { "hasdata-web-scraping": {

"type": "http",

"url": "https://mcp.hasdata.com/api/mcp?apis=web_scraping",

"headers": { "x-api-key": "YOUR_API_KEY" }

} }

}

· Add this entry to .vscode/mcp.json. Servers already in the file stay as they are.

· Agent mode picks the server up on save.

Check the tools are thereAgent mode

Ctrl/⌘ + Shift + P → MCP: List Servers

hasdata-web-scraping · running · 1 tool

Windsurf

~/.codeium/windsurf/mcp_config.json

{

"mcpServers": { "hasdata-web-scraping": {

"serverUrl": "https://mcp.hasdata.com/api/mcp?apis=web_scraping",

"headers": { "x-api-key": "YOUR_API_KEY" }

} }

}

· Add this entry to ~/.codeium/windsurf/mcp_config.json. Servers already in the file stay as they are.

· Windsurf reads serverUrl, not url. Header auth only.

Check the tools are thereIn Cascade

Cascade → MCP servers

hasdata-web-scraping · 1 tool

Cline

cline_mcp_settings.json

{

"mcpServers": { "hasdata-web-scraping": {

"type": "streamableHttp",

"url": "https://mcp.hasdata.com/api/mcp?apis=web_scraping",

"headers": { "x-api-key": "YOUR_API_KEY" }

} }

}

· Add this entry to cline_mcp_settings.json. Servers already in the file stay as they are.

· Header auth only. The type field tells Cline this is a remote server, not a command.

Check the tools are thereIn the MCP panel

MCP Servers → Installed

hasdata-web-scraping · 1 tool

ChatGPT / Agents SDK

Recommended · OAuth connector

1.Settings → Security and login → Developer mode

2.Plugins → + → developer-mode app, paste https://mcp.hasdata.com/api/mcp?apis=web_scraping

3.Sign in with HasData

agent.py

from agents import Agent, HostedMCPTool

tool = HostedMCPTool(tool_config={

"type": "mcp",

"server_label": "hasdata-web-scraping",

"server_url": "https://mcp.hasdata.com/api/mcp?apis=web_scraping",

"authorization": "YOUR_API_KEY",

"require_approval": "never",

})

agent = Agent(name="Researcher", tools=[tool])

· The Agents SDK sends authorization as a Bearer token, which the endpoint accepts. The ChatGPT route needs no key: it signs in with OAuth.

Check the tools are thereRun the agent

Runner.run(agent, "…")

hasdata-web-scraping · 1 tool listed

Codex CLI

~/.codex/config.toml

[mcp_servers.hasdata-web-scraping]

url = "https://mcp.hasdata.com/api/mcp?apis=web_scraping"

env_http_headers = { "x-api-key" = "HASDATA_API_KEY" }

· Add this entry to ~/.codex/config.toml. Servers already in the file stay as they are.

· TOML, not JSON. Codex reads the key from the HASDATA_API_KEY environment variable, so export it in the shell that starts Codex.

Check the tools are thereIn the terminal

codex mcp list

hasdata-web-scraping · 1 tool

Gemini CLI

~/.gemini/settings.json

{

"mcpServers": { "hasdata-web-scraping": {

"httpUrl": "https://mcp.hasdata.com/api/mcp?apis=web_scraping",

"headers": { "x-api-key": "YOUR_API_KEY" }

} }

}

· Add this entry to ~/.gemini/settings.json. Servers already in the file stay as they are.

· httpUrl, not url: Gemini keeps url for SSE servers. Header auth only.

Check the tools are thereIn the session

/mcp

hasdata-web-scraping · 1 tool

Raw HTTP / curl

zsh — hasdata-web-scraping

$curl -N https://mcp.hasdata.com/api/mcp?apis=web_scraping \

> -H "x-api-key: YOUR_API_KEY" \

> -H "Content-Type: application/json" \

> -H "Accept: application/json, text/event-stream" \

> -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

· Fallback for any client not listed. Streamable HTTP wants both Accept types, and leaving one out returns 406.

Check the tools are thereCall it yourself

curl … tools/list

{ "tools": [ … 1 item ] }

Scope

Web scraping plus whatever else you need

Name the services in the query string and the server exposes only their tools. Every one draws from the same key and the same balance.

Web Scraping 1 tool See the full MCP server →

https://mcp.hasdata.com/api/mcp?apis=web_scraping,google_serp,google_maps

15tools in your server

Build vs Buy

Free on GitHub, paid in hours

Anyone can clone a free MCP to fetch a web page. Whether that stays free depends on the shape of your project.

Scenario

A script I run once

Self-hosted from GitHubyour machine, your upkeep

$0/ mo, runs locally

+ 3–6 h onceof someone’s time

Setup (clone, wire, run)3–6 h once

Markup changesnot yet

Retries and concurrencynot needed

Proxy and hosting$0

Nothing to own. You get the answer and delete the folder.

mcp.hasdata.comFree tier

$0/ mo

+ 0 hof someone’s time

Setup one line

Parser upkeep included

Retries and concurrency included

~250 calls 250 of 1,000 credits

The free tier renews every month. No card.

leans self-hosted

Take the free repo

Nothing has to keep working tomorrow, so upkeep never appears. Both cost nothing, so take whichever is already in front of you.

A weekly report

Self-hosted from GitHubyour machine, your upkeep

$0–40/ mo

+ 2–4 h / moof someone’s time

Setup (proxy, hosting, wiring)3–6 h once

Markup changes2–4 h / mo

Retries and concurrency2–4 h once

Proxy and hosting$0–40 / mo

Someone looks at it when the numbers come back wrong, which is how you find out.

mcp.hasdata.comFree tier

$0/ mo

+ 0 hof someone’s time

Setup one line

Parser upkeep included

Retries and concurrency included

~1,000 calls a month 1,000 of 1,000 credits

Exactly the free tier, so a heavier month moves you to Startup.

dead even

Either works

The repo is free if you have somewhere to run it, ours is free up to the credit ceiling. The two to four hours a month decide it.

A pipeline in production

Self-hosted from GitHubyour machine, your upkeep

$40–120/ mo

+ 6–10 h / moof someone’s time

Setup (proxy, hosting, wiring)3–6 h once

Markup changes2–4 h / mo

Retries, concurrency, rate limits4–8 h once

Proxy and hosting$40–120 / mo

Someone owns this permanently, and it is never the top of their list.

mcp.hasdata.comStartup, 200K credits, 5 concurrent

$49/ mo

+ 0 hof someone’s time

Setup one line

Parser upkeep included

Retries and concurrency included

50,000 calls 50K of 200K credits

A quarter of the plan used, and the rest is headroom.

leans hosted

Use the hosted server

The proxy and hosting bill alone is in the range of the whole plan, and that is before anyone spends the six to ten hours.

A feature in my product

Self-hosted from GitHubyour machine, your upkeep

$300+/ mo

+ on-callof someone’s time

Setup (proxy, hosting, wiring)3–6 h once

Markup changes, plus on-call2–4 h / mo

Retries, concurrency, rate limitsongoing

Proxy and hosting$300+ / mo

The thing your customers touch depends on one volunteer's spare evening.

mcp.hasdata.comBasic, 1M credits, 15 concurrent

$99/ mo

+ 0 hof someone’s time

Setup one line

Parser upkeep included

Retries and concurrency included

500,000 calls 500K of 1M credits

Same endpoint at ten calls and at ten million. One invoice.

leans hosted

Use the hosted server

A third of the price, and nobody has to carry a pager for a dependency your customers touch.

Pricing

One balance, every interface

A flat monthly price against a known credit ceiling. Move up a plan when you outgrow it.

Free

$0 /mo

Free forever

1,000 tool calls / month

1,000 credits / month

1 concurrent request

Every MCP tool on every plan

Failed calls are not billed

One balance across MCP and REST

Community support

Start free

Startup

$49 /mo

$0.25 / 1k tool calls

200K tool calls / month

Billed annually Save $118

200K credits / month

5 concurrent requests

Every MCP tool on every plan

Failed calls are not billed

One balance across MCP and REST

Email support

Get started

Basic

Recommended

$99 /mo

$0.10 / 1k tool calls

1M tool calls / month

Billed annually Save $238

1M credits / month

15 concurrent requests

Every MCP tool on every plan

Failed calls are not billed

One balance across MCP and REST

Priority email support

Get started

Growth

$208 /mo

$0.07 / 1k tool calls

Billed annually Save $498

3M credits / month

50 concurrent requests

Every MCP tool on every plan

Failed calls are not billed

One balance across MCP and REST

Dedicated account manager

Get started

Monthly tool call volume

Free 200K 1M 3M 5M 10M 20M

Best fit

Basic

tool calls / mo

Concurrency

$ / 1k tool calls

$0.10

Billed annually

$99 /mo

Get Started

Enterprise

Custom price based on required volume

Past 20M credits a month, or terms the self-serve plans do not cover. We shape the contract around your workload. Past 20M credits a month, or need terms the self-serve plans do not cover? We shape the contract, concurrency, and support around your workload.

Credits rollover

Unused credits carry into the next billing period.

Concurrency 2000+

Parallel request limits set to your peak load.

#1 request priority

Highest speed, always first in the queue.

Personal manager

A direct line to the founding team.

SSO

SAML single sign-on for the whole team.

SOC-2 compliance

Security review, DPA, and audit reports.

Talk to sales Quote within one business day

The same data, without the agent

▤ Web Scraping APIAny URL in, rendered HTML, markdown or extracted fields out. Scraper API

FAQ

Before you wire it in

Which sites can it fetch?

Any public URL. There is no allow-list, so the same tool reads a docs page, a product page or a JSON endpoint. Pages that need an account are out of scope, and the tool never sends cookies of yours.

What formats can the page come back in?

Set outputFormat to markdown, text, html or json. Markdown is the one agents digest best. Ask for several and they arrive together in one response.

When do I need jsRendering?

When the content you want is put on the page by a script after load. Single-page apps and infinite feeds are the usual cases. Pair it with waitFor so the fetch waits for the selector that proves the data has landed.

What is the difference between extractRules and aiExtractRules?

extractRules take CSS selectors and return exactly what they match, the same way every time. aiExtractRules take a field name and a description and let a model pull the value out of the HTML, which suits pages whose markup you do not know in advance.

How does billing work with rendering and proxies?

The tools table shows the base cost of a plain fetch. Rendering JavaScript or routing through a residential proxy raises it, and the exact ladder is on the Web Scraping API page.

How fresh is the data? Is anything cached?

Every tool call fetches the page at that moment. Nothing is served from a cache, which is why a call takes a couple of seconds instead of milliseconds.

How many calls can run in parallel?

One on the free tier, five on Startup, fifteen on Basic, and fifty on Growth. The larger Growth volumes raise it to 200, 300 and 500.

What happens when I run out of credits?

Nothing is charged on top of the plan. Calls stop until the billing cycle renews.

Does it work with Claude Desktop, Cursor, VS Code and Claude Code?

Yes, all four, plus anything else that speaks MCP over streamable HTTP. Claude Desktop connects through OAuth, and the rest take the URL and an API key header.

Is HasData affiliated with the sites I scrape?

No. HasData is an independent service and is not affiliated with, endorsed by, or sponsored by any website you point the tool at. Site names and trademarks belong to their respective owners.

How is this different from calling the Web Scraping API directly?

Same request, same infrastructure. The MCP server puts a tool schema and auth in front of it so an agent can pick the parameters itself, instead of you writing the request and parsing the response.