HasData
Back to all posts

How to Use cURL with a Proxy

curl -x proxy_address:port URL sends a request through a proxy, and that one flag plus a handful of variations covers most of what proxies in cURL involve. This guide pairs each task with its flag, records how the proxy environment variables behave, and matches each common proxy failure to the exact message and exit code curl 8.10 prints, every case reproduced against a live proxy.

Installing cURL

Windows 10 and 11 include curl.exe since version 1803, so on a current system there is nothing to install, and curl --version in any terminal confirms it. For an older Windows, the official cURL site offers installers, and Chocolatey users can run:

choco install curl

On Debian or Ubuntu the package manager does it:

sudo apt-get install curl

And on Fedora, CentOS and relatives:

sudo dnf install curl

macOS ships cURL by default. Whatever the platform, the same check proves the install worked:

curl --version

The first line names the version and the TLS backend (Schannel on Windows, OpenSSL on most Linux builds), and the backend matters later, because TLS error messages differ between them.

Basic cURL Syntax

A bare cURL call performs a GET request and prints the response body:

curl https://www.example.com

Flags shape everything else. A GET request with a header, against an endpoint that echoes the headers back:

curl https://httpbin.org/headers -H "X-Test-Header: hello"

The response repeats the request headers, X-Test-Header included, which makes it a quick check that a header actually left the machine. A POST request carries its payload in -d, and JSON bodies want the matching Content-Type header. This call fetches an Amazon search page through the Web Scraping API, with the proxy pool on the API’s side:

curl -X POST "https://api.hasdata.com/scrape/web" -H "x-api-key: PUT-YOUR-API-KEY" -H "Content-Type: application/json" -d "{\"url\": \"https://www.amazon.com/s?k=tablets\", \"proxyType\": \"residential\", \"proxyCountry\": \"US\"}"

The \" escaping is for the Windows command line, where the JSON body goes inside double quotes. On bash, wrap the whole -d payload in single quotes and drop the backslashes.

cURL Flags

The general-purpose flags this guide leans on, in one place:

  1. -H, --header. Adds a request header.
  2. -d, --data. The request body, switching the method to POST.
  3. -X, --request. Overrides the HTTP method.
  4. -o, --output. Writes the response to a file.
  5. -L, --location. Follows redirects.
  6. -A, --user-agent. Sets the User-Agent string.

cURL 8.10 lists 265 options under curl --help all, and these six plus the proxy flags in the table below cover proxy work.

Specifying Proxies in cURL

The -x flag takes the proxy address, and everything after it works as usual:

curl -x proxy_address:proxy_port URL

With a concrete address, requesting a page that echoes the caller’s IP shows the proxy working:

curl -x 203.0.113.10:8080 https://httpbin.org/ip

The origin field in the response holds the address the target saw, so when it matches the proxy rather than your own connection, the proxy is in the path. Without a scheme, cURL assumes an HTTP proxy. HTTPS targets still work through an HTTP proxy, tunneled with CONNECT, which is the standard setup rather than an exception.

The tasks and their flags, side by side:

TaskFlagExample
Route through an HTTP proxy-xcurl -x 203.0.113.10:8080 URL
Authenticate to the proxy--proxy-user or credentials in the URLcurl -x http://user:pass@203.0.113.10:8080 URL
Route through SOCKS5-x socks5://curl -x socks5://203.0.113.10:1080 URL
SOCKS5 with remote DNS-x socks5h://curl -x socks5h://203.0.113.10:1080 URL
Skip the proxy for some hosts--noproxycurl --noproxy example.com URL
Ignore configured proxies entirely-x ""curl -x "" URL
Debug the proxy handshake-vcurl -v -x 203.0.113.10:8080 URL
Tolerate a TLS-intercepting proxy-kcurl -k -x 203.0.113.10:8080 URL

Auth, SOCKS, --noproxy and the -x "" override each come up again below, and -k gets its own section after the failure table.

SOCKS Proxies

A SOCKS proxy is the same -x flag with a socks5:// or socks4:// scheme, and the dedicated --socks5 flag does the same job. The one choice that matters is socks5:// against socks5h://, since the h variant resolves DNS on the proxy instead of your machine, keeping the lookup itself off your network. Which jobs justify SOCKS over plain HTTP proxies is part of choosing proxies for web scraping.

Using a Rotating Proxy with cURL

A rotating proxy service gives one gateway address and switches the exit behind it, so to cURL it looks like any single proxy. Point -x at the gateway with your credentials, and every request leaves via a different address. The rotation mechanics and when a pool beats a gateway are a topic of their own, and nothing about them changes the cURL syntax.

Proxy Authentication with cURL

Credentials go either inside the proxy URL or in a separate flag, and both forms produce the same Proxy-Authorization header:

curl -x http://scraper:s3cret@203.0.113.10:8080 https://httpbin.org/ip

The flag form keeps the credentials out of the proxy URL, which matters when the command lands in shell history or logs:

curl --proxy-user scraper:s3cret -x 203.0.113.10:8080 https://httpbin.org/ip

For proxies that use digest or NTLM schemes, --proxy-digest and --proxy-ntlm switch the mechanism while --proxy-user still carries the credentials, and the debugging table below shows the failed-auth case with its exit code.

Skipping the Proxy for Specific Hosts

The --noproxy flag names hosts that bypass the proxy even when one is configured:

curl -x proxy_address --noproxy httpbin.org https://httpbin.org/ip

That request goes directly to httpbin.org while everything else keeps using the proxy, which is the usual arrangement for internal hosts that a corporate proxy would refuse. The no_proxy environment variable in the next section holds the same kind of host list without touching the command line.

Environment Variables for cURL Proxy

cURL reads proxy settings from environment variables on its own, so once they are set, a plain curl URL is already proxied, with no -x anywhere:

export http_proxy="http://scraper:s3cret@203.0.113.10:8080"
export https_proxy="http://scraper:s3cret@203.0.113.10:8080"
export no_proxy="localhost,127.0.0.1"

On the Windows command line the same variables are set without quotes:

set http_proxy=http://scraper:s3cret@203.0.113.10:8080
set https_proxy=http://scraper:s3cret@203.0.113.10:8080

http_proxy applies to http:// targets and https_proxy to https:// targets, so both usually point at the same proxy. ALL_PROXY covers both schemes in one variable. Spelling matters on Unix systems. cURL reads http_proxy only in lowercase there (the uppercase form is deliberately ignored for http), while Windows treats environment variable names case-insensitively, so a setup that works on a Windows machine can quietly stop proxying after a move to Linux. The lowercase forms work everywhere.

When the variables are set and one request must go out differently, -x wins over the environment, and an empty -x "" forces a direct connection:

curl -x "" https://httpbin.org/ip

That override is the fastest way to check whether a misbehaving request is the proxy’s fault.

Using an Alias in cURL

An alias is a shell feature rather than a cURL one, and it saves retyping the proxy flags. On bash or zsh, add a line to .bashrc or .zshrc:

alias mycurl='curl --proxy-user scraper:s3cret -x 203.0.113.10:8080'

After reloading the shell, mycurl https://httpbin.org/ip runs the full command. PowerShell reaches the same effect with a function instead:

function mycurl { curl.exe --proxy-user scraper:s3cret -x 203.0.113.10:8080 @args }

The curl.exe spelling matters in Windows PowerShell, where bare curl is an alias of Invoke-WebRequest with different flags.

Using a .curlrc File

The .curlrc file holds defaults that apply to every cURL run, which suits a proxy better than aliases when everything should use it. On Linux and macOS the file lives at ~/.curlrc. On Windows, cURL looks for _curlrc in %APPDATA%, and the CURL_HOME environment variable overrides the location on either system. One option per line:

proxy = "http://203.0.113.10:8080"
proxy-user = "scraper:s3cret"

With the file in place, a plain request picks up the proxy:

curl https://httpbin.org/ip

Command-line flags beat .curlrc, and both beat the environment variables.

When the Proxy Fails

Every failure mode in this table was reproduced against a local proxy on the Windows build of curl 8.10, so the messages are what it actually prints. The message wording varies by platform, but the exit codes are libcurl error numbers and stay the same everywhere. The exit code is what your script sees in $?, and two rows return 0.

What happenedWhat cURL saysExit codeThe fix
Nothing listens on the proxy portFailed to connect to 127.0.0.1 port 9 ... Could not connect to server (Linux builds print Connection refused)7Check address and port, then whether the proxy is up
Proxy hostname does not resolveCould not resolve proxy: no-such-proxy.invalid5Fix the hostname or DNS
Auth required or wrong, HTTPS targetCONNECT tunnel failed, response 40756Add or correct --proxy-user or URL credentials
Auth required, HTTP targetThe proxy’s 407 page arrives as the response body0Same fix, but no error signals it. Check the body
Proxy cannot or will not tunnelCONNECT tunnel failed, response 50256The proxy cannot tunnel HTTPS. Use another
Proxy breaks the TLS tunnelschannel: ... SEC_E_INVALID_TOKEN (message varies by TLS backend)35Rotate. -k does not help, the handshake never completes
Proxy intercepts TLS with its own certificateschannel: SEC_E_UNTRUSTED_ROOT ... certificate chain ... not trusted60-k if you accept interception, a cleaner proxy if not
SOCKS flag pointed at an HTTP proxyconnection to proxy closed97Match the scheme to the proxy’s actual type
Proxy address unroutableFailed to connect ... Timeout was reached (Linux builds print Connection timed out)28Set --connect-timeout low and rotate to the next proxy
Proxy accepts and closesRecv failure: Connection was aborted56The proxy is broken or filtering. Rotate
Proxy substitutes the responseNothing, the wrong body arrives0Request a page with known text and compare the content

The exit-28 row is the one worth capping in scripts, because the default connect wait is long enough to stall a rotation loop. A short cap fails over to the next proxy in seconds:

curl --connect-timeout 10 -x 203.0.113.10:8080 https://httpbin.org/ip

When the table alone does not settle it, -v prints the proxy handshake itself. A healthy CONNECT tunnel looks like this, and any deviation from it names the failing step:

> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> User-Agent: curl/8.10.1
< HTTP/1.1 200 Connection established
* CONNECT phase completed
* CONNECT tunnel established, response 200

A 407 in place of the 200 is the auth rows, a 502 is the tunnel row, and a connection that dies right after CONNECT is the broken-proxy row.

Both zero-exit rows break scripts the same way. A script that checks $? and moves on treats both as success, records a 407 page or an injected body as data, and fails much later in parsing, so a content check against a page with known text is the only reliable proxy test. When errors do come back from the target rather than the proxy, they carry the usual meanings, 403 for an address the site refuses, 429 for rate limiting, and 503 for overload or an anti-bot layer dressed up as one.

How to Bypass SSL Certificate Errors When Using cURL Proxy

Exit code 60 means the certificate cURL received is not signed by a trusted authority, and behind a proxy that usually means the proxy intercepts TLS and presents its own certificate. The -k or --insecure flag accepts that interception:

curl --proxy http://203.0.113.10:8080 --insecure https://httpbin.org/ip

Exit code 35 is a different failure, a tunnel that never completes a handshake, and -k changes nothing there because verification is never reached. Both cases are measured in the failure table above. The -k flag removes the proof that you are talking to the real site, so anything between you and the target can read and rewrite the traffic. It belongs in debugging sessions and throwaway collection jobs, never near credentials.

Conclusion

Configure the proxy with -x for one-off requests, environment variables or .curlrc for everything at once, and remember which layer wins when they disagree, flags first, then the config file, then the environment. When a request misbehaves, -x "" isolates the proxy’s share of the blame in one run, -v shows the handshake, and the exit code, matched against the table above, names the failure before any guessing starts.

Valentina Skakun
Valentina Skakun
Valentina is a software engineer who builds data extraction tools before writing about them. With a strong background in Python, she also leverages her experience in JavaScript, PHP, R, and Ruby to reverse-engineer complex web architectures.If data renders in a browser, she will find a way to script its extraction.
Articles

Might Be Interesting