How to Use a Proxy with Python Requests in 2026

Python’s Requests library takes a proxy through the proxies= argument on a single call, through session.proxies for every call on a session, and through the HTTP_PROXY family of environment variables for every call in the process. They look interchangeable and are not. I measured all their combinations on requests 2.34.2 for this article, and two of the results matter in production. An environment variable silently beats session.proxies, and a proxy configured only for http stops applying the moment a site redirects to https, so the next hop leaves with your real IP.

This guide covers the setup for each proxy type, authentication, rotation, the measured precedence rules, and what the error codes coming back through a proxy mean.

Basics of Using Proxies with Requests

A proxy in Requests is a dictionary mapping a URL scheme to a proxy address. The example target here is httpbin.org/ip, which answers with the address it saw, so a working proxy is visible in the response body. The addresses below use the documentation ranges, so swap in a real proxy before running:

import requests

proxies = {
    'http': 'http://203.0.113.10:8080',
    'https': 'http://203.0.113.10:8080',
}

response = requests.get('https://httpbin.org/ip', proxies=proxies, timeout=30)
print(response.json()['origin'])

The printed address is the proxy’s, and without the proxies= argument it is yours, which makes this two-line check the fastest way to confirm a proxy is actually in the path.

Both keys usually point at the same http:// address. The key is the scheme of the target URL, and the value is the proxy’s own address (the requests documentation on proxies covers the dictionary format), so https requests normally travel through an HTTP proxy too, tunneled with CONNECT. Set both keys unless you know why you are setting one, because a missing https key means HTTPS requests go out directly.

SOCKS proxies plug into the same dictionary once requests[socks] is installed:

pip install "requests[socks]"

The scheme in the value switches the protocol, and nothing else in the code changes:

proxies = {
    'http': 'socks5://203.0.113.20:1080',
    'https': 'socks5://203.0.113.20:1080',
}

socks5 resolves the target’s hostname on your machine, and socks5h pushes DNS resolution to the proxy as well, which matters when the DNS lookup itself would leak where you are going. When a scraping job justifies that extra setup is part of choosing proxies for web scraping.

Any HTTP method takes the same proxies= argument, so post, put, delete, head and patch calls route identically, and there is nothing proxy-specific to learn per method.

Setting Proxies with Environment Variables

The same configuration travels through environment variables, which keeps proxy settings out of the code entirely:

export HTTP_PROXY=http://203.0.113.10:8080
export HTTPS_PROXY=http://203.0.113.10:8080
export NO_PROXY=localhost,127.0.0.1

Requests reads them on every call, ALL_PROXY covers both schemes at once, and NO_PROXY lists hosts that bypass the proxy. The convenience cuts both ways once several configuration sources exist in one program.

Authentication with Proxies

Private proxies want credentials, and for HTTP, HTTPS and SOCKS proxies alike they ride inside the proxy URL:

proxies = {
    'http': 'http://scraper:s3cret@203.0.113.10:8080',
    'https': 'http://scraper:s3cret@203.0.113.10:8080',
}

The auth= argument of requests.get does a different job. It sends credentials to the target site as an Authorization header, while a proxy expects them in a Proxy-Authorization header built from the proxy URL, so passing proxy credentials through auth= leaves the proxy unsatisfied and hands the credentials to every site you visit.

SOCKS credentials follow the same URL form. The socks5 scheme sends username and password inside the SOCKS handshake rather than as an HTTP header, and the URL syntax hides that difference completely:

proxies = {
    'http': 'socks5://scraper:s3cret@203.0.113.20:1080',
    'https': 'socks5://scraper:s3cret@203.0.113.20:1080',
}

When credentials in this form still get rejected, the response names the problem.

Handling Error 407

HTTP 407, Proxy Authentication Required, is the proxy rejecting the request before it goes anywhere. The usual causes, in the order I check them.

1. Credentials missing from the proxy URL. The request rides through the proxy with no Proxy-Authorization at all, often because the credentials sit in auth= instead.
2. A typo in the credentials. The header is present and wrong.
3. The wrong proxy type. This one clears itself from the list rather than joining it. A socks5:// URL pointed at an HTTP proxy port, or the reverse, fails as ProxyError or ConnectTimeout rather than a 407, so a real 407 actually rules a type mismatch out.
4. Source-IP restrictions on the proxy side. Providers that whitelist client IPs answer 407 from any address they do not know, with perfectly correct credentials.

Checking with curl separates a code problem from a credentials problem:

curl -x http://scraper:s3cret@203.0.113.10:8080 https://httpbin.org/ip

If curl succeeds where the script fails, the bug is in the Python side, usually credentials that ended up in auth= instead of the proxy URL. If curl fails too, the credentials or the proxy address are wrong.

Rotating Proxies

Rotation happens in one of two places, the provider’s gateway or your own code. Either way the point is the same, a target that rate-limits or blocks by address stops seeing one address and starts seeing many.

A rotating proxy service rotates on its side. The code configures one endpoint and every request leaves from a different exit:

import requests

proxies = {
    'http': 'http://scraper:s3cret@gate.example-provider.com:8080',
    'https': 'http://scraper:s3cret@gate.example-provider.com:8080',
}

for _ in range(5):
    response = requests.get('https://httpbin.org/ip', proxies=proxies, timeout=30)
    print(response.json()['origin'])   # a different exit address each time

A proxy pool keeps the rotation in your code. The scraper carries a list and cycles through it, skipping entries that fail:

import itertools

import requests

pool = itertools.cycle([
    'http://203.0.113.10:8080',
    'http://203.0.113.11:8080',
    'http://203.0.113.12:8080',
])

def fetch(url, retries=3):
    for _ in range(retries):
        address = next(pool)
        try:
            return requests.get(url, proxies={'http': address, 'https': address},
                                timeout=15)
        except requests.RequestException:
            continue                    # dead proxy, take the next one
    raise RuntimeError(f"no working proxy for {url}")

itertools.cycle walks the list forever and evenly, which replaces the index arithmetic that off-by-one bugs love. The pool’s real cost is upkeep. Addresses die, get banned per target and need health checks, and rotating proxies for web scraping covers running that pool seriously.

Ignoring SSL Certificates in Rotating Pools

Rotating through many exits eventually hits a proxy that intercepts TLS with its own certificate, and requests refuses with an SSLError. Disabling verification with verify=False keeps the run going. It also removes the only proof that you are talking to the site itself, leaving anything between you and the site free to read and rewrite the traffic, so it belongs on throwaway collection jobs and never near credentials or payments.

response = requests.get('https://httpbin.org/ip', proxies=proxies,
                        verify=False, timeout=30)

Requests also prints an InsecureRequestWarning on every such call as a reminder of what is switched off.

Which Proxy Setting Wins

The proxies= argument on the call beats everything, an environment variable beats session.proxies, and trust_env = False silences the environment. Those rules come from measurement rather than the docs. Each configuration source pointed at its own local marker proxy, every combination fired one request, and the marker that answered names the winner. On requests 2.34.2:

Look at the fourth row. session.proxies loses to an environment variable, because Requests merges environment settings over the session’s attribute at request time. A scraper that sets session.proxies and runs on a machine with a forgotten HTTP_PROXY uses the wrong proxy, and nothing in the code says so. trust_env = False on the session is the fix, and it also switches off NO_PROXY and .netrc handling, after which only your own configuration applies. The proxies= argument on the call outranks everything, so per-request configuration is the one that behaves the way people expect.

The last table row is the useful curiosity. A per-host key such as 'http://example.com' beats the scheme-wide 'http' key for that host, which routes one difficult site through a dedicated exit while the rest of the pool serves everything else.

The Redirect That Drops Your Proxy

The scheme key is evaluated per hop, not per request. I ran this against a live site that answers http:// with a 301 to https://, through a local forwarding proxy that logs every connection:

With only the http key set, the script finishes with a clean 200 while the proxy log shows a single entry, because the redirect hop to https never touched it. Nothing raises, nothing warns, and the target has logged your real address. Since most sites redirect http to https immediately, a one-key configuration effectively sends almost all real traffic around the proxy. Set both keys, always.

Managing Sessions with Proxies

A session reuses connections, keeps cookies, and carries settings across requests, so it is the natural home for a proxy that every request should use:

import requests

session = requests.Session()
session.trust_env = False           # environment variables no longer override
session.proxies = {
    'http': 'http://203.0.113.10:8080',
    'https': 'http://203.0.113.10:8080',
}

response = session.get('https://httpbin.org/ip', timeout=30)
print(response.json()['origin'])
session.close()

The trust_env = False line is there because of the precedence table above. Without it, the session’s proxies hold only until someone exports HTTP_PROXY on the host. Sessions also matter for performance, since the CONNECT tunnel and the TLS handshake to the target happen once per reused connection instead of once per request, and connection reuse is most of the speed difference between a session and bare requests.get calls in a loop.

Handling 403, 429 and Timeouts

Through a proxy, error codes gain a second possible sender, and telling them apart decides the fix.

HTTP 403 coming back means the target refused the request, and with a proxy in play the usual reason is that the exit address itself is on a blocklist. Retrying through the same exit changes nothing, so rotate. HTTP 429 means too many requests from that address, and it clears by slowing down or spreading load across the pool. HTTP 503 is the ambiguous one, sent both by overloaded targets and by anti-bot layers that dress up a block as unavailability.

Timeouts through a proxy mean either a dead proxy or a slow tunnel, and the exception does not say which. The timeout argument takes a tuple to tell those cases apart at configuration time:

response = requests.get('https://httpbin.org/ip', proxies=proxies,
                        timeout=(5, 30))   # 5 s to connect, 30 s to read

A dead proxy fails the 5-second connect and surfaces fast as ConnectTimeout or ProxyError, while a live proxy on a slow target gets the full 30 seconds to read. Treating ProxyError as “rotate now” and ReadTimeout as “retry once, then rotate” covers both without special-casing. A managed alternative to owning this whole layer is a Web Scraping API, which keeps the proxy pool, rotation and retries behind one endpoint and answers with the page.

Conclusion

Set proxies per request when different requests need different exits, per session for one exit everywhere, and through environment variables only when the machine’s operator owns that configuration. When sources conflict, the precedence table above is the reference, and trust_env = False is the one switch that makes a session’s configuration final. Everything else in this article is a special case of checking what actually went over the wire, and the two-line origin check from the basics section stays the fastest way to do that.