Node Unblocker for Web Scraping

Node Unblocker is a Node.js library for building a web proxy that fetches remote pages, rewrites their links and assets, and streams them back to the client. It started as a censorship-circumvention tool and grew into a general-purpose proxying library with a middleware API. Requests leave from the host that runs it, so the target site sees that host’s address and region, while the client’s own network sees only the connection to the proxy.

The package is maintained, installs on current Node.js with Express 5, and its latest release is 2.3.1. The tutorial below builds a proxy server with it, adds custom middleware, deploys the result to a free host, and ends with the cases where the library stops being enough.

What is Node Unblocker

Node Unblocker runs as Express middleware. A request to /proxy/http://example.com/ makes the server fetch the remote page, rewrite the URLs inside it so that links, scripts, and images keep going through the proxy, and stream the result back to the browser.

Custom middleware hooks into both directions. Request middleware can inspect or change the URL and headers before the request leaves, or answer it directly, and response middleware can transform the remote page before it reaches the client. The README describes the library as a proxy for evading internet censorship, and a scraper uses the same mechanics for a different end, to send its requests from a host of its choosing and to rewrite what comes back.

Step-by-Step Guide to Node Unblocker for Web Scraping

The build starts with a basic Express app that mounts the proxy on port 3000, then adds custom middleware next to the library’s built-in middleware, then deploys to a free host.

Prerequisites

Node.js and npm must be installed. The current LTS build from nodejs.org ships with npm, and node -v and npm -v confirm that both are on the PATH.

Install the two packages in an empty project folder.

npm install express unblocker

The examples use import syntax, so add "type": "module" to package.json (or name the file index.mjs). Node.js 22 reparses a file with import as an ES module and prints a MODULE_TYPELESS_PACKAGE_JSON warning, and Node.js 18 stops with SyntaxError: Cannot use import statement outside a module. Create index.js for the code below and run it with:

node index.js

Everything below goes into index.js.

Creating the Base Application

Start with the two imports:

import express from 'express';
import Unblocker from 'unblocker';

Create the Express app. Routes and middleware attach to it.

const app = express();

Create the Unblocker instance with the URL prefix the proxy answers on:

const unblocker = new Unblocker({
    prefix: '/proxy/'
});

Mount it before any other route, so proxied requests never reach your own handlers:

app.use(unblocker);

Routes for your own pages come after it. This one answers the root path / at http://localhost:3000/ with a short message:

app.get('/', (req, res) => {
    res.send('Welcome to the main page!');
});

Finally, listen on a port. process.env.PORT lets a hosting platform choose it, and 3000 is the local default:

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
    console.log(`Server started on port ${PORT}`);
});

Run it:

Requests that start with /proxy/ are proxied. http://localhost:3000/proxy/http://example.com/ fetches http://example.com/ through the server on http://localhost:3000/ and returns the rewritten page.

Using Middleware

Custom middleware processes requests and responses. Typical jobs are validating the target URL, modifying response content, handling cookies, and rewriting URLs.

The first example validates requests. If the target URL is not on google.com, the middleware answers the client with a 403 Forbidden and an error message, and the request never leaves the server.

Define the function before the Unblocker instance is created. data.clientResponse is the Express response object, so status() and send() work on it:

function validateRequest(data) {
    if (!data.url.match(/^https?:\/\/(www\.)?google\.com\//)) {
        data.clientResponse.status(403).send('Access denied.');
    }
}

Then, add the middleware to the Unblocker configuration, next to the prefix from the first version:

const config = {
    prefix: '/proxy/',
    requestMiddleware: [
        validateRequest
    ]
};

Pass the configuration to the constructor. This call replaces the earlier new Unblocker({ prefix: '/proxy/' }) line, so the file keeps one instance:

const unblocker = new Unblocker(config);

The same shape works for other middleware, such as one that redirects every request:

function redirect(data) {
    data.clientResponse.redirect('https://www.example.com');
}

Headers for the outgoing request are in data.headers. Changing data.clientRequest.headers edits the incoming request object and never reaches the target:

function modifyHeaders(data) {
    data.headers['x-custom-header'] = 'Custom Value';
}

Middleware runs in the order it is added, and the first function that sends a response stops the chain, so a validator placed after a redirect never sees the request.

Assembled, index.js looks like this:

import express from 'express';
import Unblocker from 'unblocker';

function validateRequest(data) {
    if (!data.url.match(/^https?:\/\/(www\.)?google\.com\//)) {
        data.clientResponse.status(403).send('Access denied.');
    }
}

function modifyHeaders(data) {
    data.headers['x-custom-header'] = 'Custom Value';
}

const unblocker = new Unblocker({
    prefix: '/proxy/',
    requestMiddleware: [validateRequest, modifyHeaders]
});

const app = express();
app.use(unblocker);

app.get('/', (req, res) => {
    res.send('Welcome to the main page!');
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
    console.log(`Server started on port ${PORT}`);
});

After a restart, http://localhost:3000/proxy/http://example.com/ comes back with the 403 from validateRequest, and google.com URLs go out with the extra header.

Deployment to Render and Next Steps

Heroku no longer has a free tier, so the free deployment target is Render, which runs a Node.js web service on a Free instance without a payment method. Railway offers a time-limited trial before its paid plans and Fly.io bills machines per second, so Render is the one that keeps a proxy up at no cost.

Push the project to a GitHub, GitLab, or Bitbucket repository, sign up at Render, and create a new Web Service from that repository. Render asks for a build command (npm install) and a start command (node index.js) and then for the instance type. Choose Free. The server already reads process.env.PORT, which is how Render tells the app which port to listen on, so the code stays as it is.

A Free instance spins down after 15 minutes without inbound traffic and spins back up on the next request, which can take up to a minute, and anything written to the local filesystem is lost at that point. Render itself describes Free instances as a testing tier. For a proxy that a few people use now and then that is fine, and anything permanent needs a paid instance or a small VPS.

Limitations of Node Unblocker

Node Unblocker covers standard login forms and most AJAX content reached from one server with one IP address. OAuth logins, sites like Discord or YouTube, Cloudflare checks, and proxy rotation are outside that scope.

OAuth and postMessage Calls Fail Behind the Proxy

Behind the proxy every page has the proxy’s origin instead of the site’s own, so postMessage calls that check the receiving window’s origin are dropped, and the README lists proxying postMessage data and fixing origins as an open to-do. OAuth login pop-ups from Google or Facebook pass the result back to the opener window with postMessage, so sites that sign in through OAuth fail behind the proxy.

Limited Support for Complex Sites

The proxy rewrites the URLs it finds in the page and injects a client script that forces XMLHttpRequest and WebSocket calls back through the proxy, so standard login forms and most AJAX content work. Sites built as large JavaScript applications still break. The README names Roblox, Discord, YouTube, and Instagram as sites that do not work and gives no timeframe for support. The one workaround in the repository is an example that detects YouTube video pages and replaces them with a custom page that streams only the video.

Cloudflare Detection

Cloudflare protects many sites and flags automated requests and proxy servers, answering with a challenge page, a 403, or a 429 Too Many Requests when one IP sends too much. Node Unblocker does nothing to avoid that, and every request leaves from the same server address.

No Proxy Pool or Rotation

Node Unblocker was written for censorship circumvention through a single server, and it has no support for proxy pools or proxy rotation. Rotating proxies yourself means keeping a pool of addresses, retiring the ones that start returning 403 or 429, and retrying failed requests on a fresh address.

For web scraping, our Web Scraping API sends each request through its own proxy pool, with datacenter or residential exits and JavaScript rendering on request, and returns the page HTML, so the scraper parses the response instead of running a proxy server.

Conclusion and Takeaways

Node Unblocker fits a personal proxy or a page-rewriting experiment. Two packages, under thirty lines of code, and a free Render instance that wakes up on the first request cover that, and the middleware API handles URL validation, redirects, and header changes. Scraping at volume is a different job. A single IP without rotation gets blocked, Cloudflare stops it, and OAuth flows break, so for data collection use a scraping API that runs the proxy layer for you.