Your request looks perfect. You copied the browser's User-Agent, set the right Accept headers, you are on a clean IP, and you are asking for a page any visitor can load. The server answers anyway with a bare 403 Forbidden, or a challenge page, or silence.
The mistake almost everyone makes is treating the request as just the URL plus the headers. Before your first byte of HTTP flies, your client has already introduced itself twice: once in the TLS handshake and once in the HTTP/2 connection prelude. Both introductions are extremely revealing, and neither contains anything you set in your code. A Python script that claims to be Chrome can be unmasked in milliseconds by a handshake that no Chrome has ever produced. This guide walks through the three fingerprints a server sees, shows what popular tools actually transmit (with live captures you can reproduce in one command each), and then gives you a debugging ladder that fixes 403s in the right order: client first, IP second, pacing third.
The three fingerprints
1. The TLS fingerprint (JA3 and JA4)
Every HTTPS connection starts with a TLS Client Hello. Inside it: the protocol version, the cipher suites the client supports, the TLS extensions, the signature algorithms, and the ALPN protocols (usually h2 and http/1.1). Boring cryptography to most developers, but each field is a choice, and different software chooses differently. Chrome offers 15 ciphers in a specific order. OpenSSL in its default configuration offers a different list in a different order. Go, Java and Rust each have their own defaults.
Two widely deployed hashing schemes compress this into a short string:
- JA3 concatenates the version, ciphers, extensions and curves and hashes the result. Simple, but it changes when a single field changes, which makes it brittle.
- JA4 (from FoxIO) builds a more readable fingerprint: protocol, TLS version, SNI presence, counts, ALPN, then two truncated SHA-256 hashes, one over the sorted cipher list and one over the extensions plus signature algorithms. GREASE values, the random-looking reserved values Chrome inserts deliberately, are ignored.
A real JA4 looks like this:
t13d1516h2_8daaf6152771_e5627efa2ab1
Read it as: t = TLS over TCP, 13 = TLS 1.3, d = SNI present, 15 ciphers, 16 extensions, first ALPN h2. The two hashes identify the exact cipher set and extension set. The point for scrapers: a JA4 of a real Chrome and the JA4 of a Python script are different strings, and the server can compare them long before it evaluates a single header you set.
2. The HTTP/2 fingerprint
If ALPN negotiates HTTP/2, the client sends a SETTINGS frame and a window update as the very first act of the connection, and then each request as a set of HEADERS with pseudo-headers (:method, :authority, :scheme, :path). Anti-bot vendors record all of it: the SETTINGS values and their order, the initial window sizes, and the order of the pseudo-headers. The result is usually written as an "Akamai-style" fingerprint string like:
1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p
That tail is the pseudo-header order. Chrome sends m,a,s,p. curl sends m,s,a,p. It is one swapped letter, it is invisible to your code, and it is trivially detectable.
3. The header fingerprint
Finally, the headers themselves. Not just which headers, but their order and casing, the exact Accept-Language spelling, the sec-ch-ua client hint set, and whether the values are internally consistent. Header order alone is surprisingly identifying: browsers have stable, version-specific header orders, and HTTP libraries alphabetise or randomise them depending on nothing at all.
What your tools actually send
You do not have to trust anyone's claims about this. Point your client at a TLS echo service such as tls.peet.ws/api/all and it returns your own fingerprints as JSON. Here is a live comparison from our test machine, three clients, one endpoint:
curl 8.5.0
| Field | Value |
|---|---|
| JA3 hash | 0149f47eabf9a20d0893e2a44e5a6323 |
| JA4 | t13d3112h2_e8f1e7e78f70_b26ce05bbdd6 |
| HTTP | h2 |
| H2 pseudo-header order | m,s,a,p |
Python requests 2.34.2
| Field | Value |
|---|---|
| JA3 hash | 0149f47eabf9a20d0893e2a44e5a6323 |
| JA4 | t13d3112h1_e8f1e7e78f70_b26ce05bbdd6 |
| HTTP | HTTP/1.1 |
| H2 pseudo-header order | none |
curl_cffi 0.16.3 (impersonate="chrome")
| Field | Value |
|---|---|
| JA3 hash | 3123cfe0657b90cafc8b99f6b8985527 |
| JA4 | t13d1516h2_8daaf6152771_806a8c22fdea |
| HTTP | h2 |
| H2 pseudo-header order | m,a,s,p |
Reproduce each capture yourself:
# curl
curl -s https://tls.peet.ws/api/all | python3 -c "import json,sys; d=json.load(sys.stdin); print(d['tls']['ja4'], (d.get('http2') or {}).get('akamai_fingerprint'))"
# Python requests
import requests
d = requests.get("https://tls.peet.ws/api/all", timeout=20).json()
print(d["tls"]["ja4"], d.get("http_version"))
# curl_cffi impersonating Chrome
import curl_cffi
d = curl_cffi.get("https://tls.peet.ws/api/all", impersonate="chrome", timeout=20).json()
print(d["tls"]["ja4"], d["http2"]["akamai_fingerprint"])
Four things jump out of these captures:
- curl and requests produce the identical JA3. Both ride the system OpenSSL, so the cipher list is OpenSSL's, not the tool's. Anti-bot rules like "block that hash" catch every OpenSSL client on the distribution at once.
- requests negotiates HTTP/1.1 only (see the
h1in its JA4). Almost no real browser browses an h2-capable site over HTTP/1.1 in 2026. That lone character is a quiet confession. - curl does speak h2 but with curl's SETTINGS and
m,s,a,porder, which no Chrome produces. - curl_cffi's Chrome profile produces
t13d1516h2_8daaf6152771_...with 15 ciphers and 16 extensions, and the trailing hash over the extension set. Its cipher hash even matches the worked Chrome example in the JA4 specification. This is what "fixing the client" looks like: the handshake stops contradicting the headers.
Why perfect headers don't save you
The most common scraper fix is copying a real browser's headers into the HTTP library. It is also the fastest way to look more suspicious, not less, because you create contradictions a real browser can never produce:
- User-Agent says
Chrome/153on Windows, but the JA4 says 31 OpenSSL ciphers and the ALPN says HTTP/1.1. sec-ch-uaheaders claim Chrome, but the H2 SETTINGS frame says curl.- Accept-Language says
en-GB,en, but the header order is alphabetical, which browsers do not do.
Bot detection in 2026 is mostly consistency checking. A mediocre-but-consistent story (plain curl, honest UA) is less alarming than a perfect header set stapled onto an impossible handshake. Detectors do not need to know what you are; they only need to prove your parts do not fit together.
What actually fixes it
Fix the client first
If you need plain HTTP requests (most scraping jobs), swap the HTTP layer for one that impersonates a browser's handshake end to end. In Python that is curl_cffi, the maintained binding of curl-impersonate:
pip install curl_cffi
import curl_cffi
# drop-in for requests, plus the impersonate parameter
r = curl_cffi.get("https://example.com/data", impersonate="chrome")
# pin a version when you want reproducible fingerprints
r = curl_cffi.get("https://example.com/data", impersonate="chrome124")
It sends Chrome's TLS profile, Chrome's H2 SETTINGS and pseudo-header order, and a matching header set. On the CLI, curl-impersonate gives you the same thing as a curl binary; curl_cffi also ships a curl-cffi command since v0.15.
If the page needs JavaScript, drive a browser that does not leak automation signals. Patchright (Apache 2.0) is a drop-in patch set for Playwright that closes the leaks stock Playwright has: it avoids the Runtime.enable CDP call that sites listen for, patches Console.enable, drops automation command-line flags like --enable-automation, and can work with closed shadow roots. The maintainers' recommended setup is a persistent context with the real Chrome channel, headed, and without injected headers or user agents: patch the leaks, then let the browser be itself.
Two honesty notes. First, "undetected" is a snapshot, not a property: vendors and tool authors move monthly, so pin tool versions and re-verify. Second, some protected pages will still challenge you; the goal is to stop failing on arrival.
Then fix the IP
Once your client is consistent, IP reputation becomes the next signal, and the hierarchy matters:
| IP type | How sites treat it |
|---|---|
| Datacenter | Known cloud/hosting ranges; highest scrutiny, often blocked on sight |
| Residential | Real ISP customers; good trust, but ranges get resold and flagged |
| Mobile (4G/5G) | Carrier-grade NAT: thousands of real phones share each public IP, so blocking one IP hurts real users; generally the most lenient treatment |
Mobile exits earn their trust structurally, not through cleanliness: a carrier NAT IP serves so many legitimate devices that blanket blocking is expensive for the site. The flip side is honest too: if your client fingerprint screams Python, the best mobile IP on earth will not save the request. Fix the handshake first, then choose IP quality. If you want a walkthrough of routing scrapers through UK mobile exits (both HTTP and SOCKS5, with code for requests, Scrapy and Selenium), see our web scraping guide.
Then fix the behaviour
A perfect client on a perfect IP that hammers 40 pages a second still gets caught, because rate is a fingerprint too. Three habits cover most of it:
- Pace like a person. Requests per minute and, more importantly, a plausible pattern (bursts and pauses, not a metronome).
- Sticky where accounts matter. Logging in from a new IP on every request is its own anomaly. Pin a session to one exit IP for its natural lifetime, rotate between tasks. (With Simply Proxies, appending
-session-idto the proxy username pins one device for a sliding window, 5 minutes to 12 hours.) - Warm up and back off. Respect 429s with real backoff; a client that retries at full speed teaches the site exactly how to identify it.
The debugging ladder
Run this order and you will find the failing layer in minutes instead of days:
- Reproduce with plain curl. If curl gets the page but your script doesn't, the problem is your client stack, not the site.
- Echo your fingerprints. Hit
tls.peet.ws/api/allwith your exact scraper setup. Note the JA4, HTTP version, and H2 fingerprint. - Compare with a real browser. Open the same echo URL in Chrome and diff the three fields. Big differences in JA4 or HTTP version are smoking guns.
- Swap the client, keep everything else. Move requests to
curl_cffiwithimpersonate="chrome"(or the scraper to Patchright). Retest the actual target. - Only then change the IP. If a consistent client still gets 403s, try a different IP class: residential or mobile instead of datacenter.
- Finally, throttle. If step 4 and 5 pass but sustained runs degrade, it is rate. Add pacing and sticky sessions.
Most 403 investigations die at step 4: the request starts succeeding and nobody asks why. Keep the ladder anyway; the site that ignores fingerprints today may not tomorrow.
Summary
A 403 with "perfect" headers usually means the server read your handshake, not your headers. TLS (JA3/JA4), HTTP/2 (SETTINGS and pseudo-header order) and header order together form a consistency picture that is far more identifying than any single value, and stock HTTP libraries contradict a browser identity before your first request byte. The fix order is: impersonate a coherent client with curl_cffi or Patchright, then match the IP class to the job (mobile exits get structurally lenient treatment), then pace like a human. Echo your own fingerprints at tls.peet.ws/api/all whenever a site suddenly starts refusing you; the diff usually tells the whole story.
Last verified 14 September 2026, using curl 8.5.0, Python requests 2.34.2, curl_cffi 0.16.3 and Patchright 1.62.3 on Linux. All fingerprints shown above are real output from a live echo service.
Try It With Real UK Mobile IPs
Run the examples in this guide through genuine UK 4G/5G exits. 500 MB free, no card required.
Start Free Trial