How to Use Mobile Proxies for Web Scraping (2026 Guide)

Web scraping is straightforward in theory: send HTTP requests, parse the response, extract data. In practice, most interesting targets block scrapers. The most common block is IP-based — the target identifies your IP as a scraper and returns a CAPTCHA, an error page, or nothing at all.

Mobile proxies are the most effective IP-level countermeasure available. This guide walks through how to use UK mobile proxies for web scraping with real code examples in Python.

Why Mobile Proxies for Scraping?

Datacentre IPs are trivially easy to identify and block. Residential IPs are better but still identifiable by their ASN. Mobile IPs — from real UK mobile networks — carry the highest trust score because they belong to carrier-grade NAT ranges shared by thousands of legitimate users.

For UK-specific scraping targets (e-commerce, property listings, job boards, UK SERPs), a UK mobile proxy gives you the appearance of a genuine UK smartphone user. This is the hardest profile for anti-bot systems to distinguish from real traffic.

Prerequisites

To follow this guide you need:

A Simply Proxies account (sign up free — includes 500 MB)
Your proxy credentials (username and password from the dashboard)
Python 3.8+ installed

Your proxy credentials look like this:

Setting	Value
HTTPS endpoint	`proxy.simplyproxies.com:6889`
SOCKS5 endpoint	`proxy.simplyproxies.com:6890`
Username	`aw_your_username`
Password	Your generated password

Method 1: Python Requests (Simple Scraping)

The requests library is the simplest way to make HTTP requests through a proxy. Install it first:

pip install requests

Basic proxy usage with authentication:

import requests

proxy_host = "proxy.simplyproxies.com"
proxy_port = 6889
proxy_user = "aw_your_username"
proxy_pass = "your_password"

proxies = {
    "http": f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}",
    "https": f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}",
}

response = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=30)
print(response.json())

The response will show a UK mobile IP address — not your real IP.

Adding headers for realism

A bare HTTP request with no headers is a red flag. Always send a realistic User-Agent and Accept headers:

headers = {
    "User-Agent": "Mozilla/5.0 (Linux; Android 14; Pixel 8) "
                  "AppleWebKit/537.36 (KHTML, like Gecko) "
                  "Chrome/125.0.6422.113 Mobile Safari/537.36",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-GB,en;q=0.9",
    "Accept-Encoding": "gzip, deflate, br",
}

response = requests.get(
    "https://example.co.uk/products",
    proxies=proxies,
    headers=headers,
    timeout=30,
)
print(response.status_code)

Use a mobile User-Agent string (not a desktop one) to match the mobile IP. This keeps your fingerprint consistent.

Handling retries

Any proxy connection can occasionally time out. Wrap requests in a retry loop:

import time
from requests.exceptions import RequestException

def fetch_with_retry(url, proxies, headers, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = requests.get(url, proxies=proxies, headers=headers, timeout=30)
            response.raise_for_status()
            return response
        except RequestException as e:
            wait = 2 ** attempt
            print(f"Attempt {attempt + 1} failed: {e}. Retrying in {wait}s...")
            time.sleep(wait)
    return None

Method 2: Scrapy (Large-Scale Scraping)

Scrapy is a production-grade scraping framework. To route all Scrapy requests through your mobile proxy, configure the DOWNLOADER_MIDDLEWARES and HTTPPROXY_AUTH_ENCODING:

# settings.py

DOWNLOADER_MIDDLEWARES = {
    "scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": 110,
}

HTTPPROXY_AUTH_ENCODING = "utf-8"

PROXY_URL = "http://aw_your_username:your_password@proxy.simplyproxies.com:6889"

Then in your spider, set the proxy per request:

# spider.py
import scrapy
from urllib.parse import urlencode

class UkRetailSpider(scrapy.Spider):
    name = "uk_retail"
    custom_settings = {
        "HTTPPROXY_AUTH_ENCODING": "utf-8",
    }

    def start_requests(self):
        urls = [
            "https://example.co.uk/category/electronics",
            "https://example.co.uk/category/clothing",
        ]
        for url in urls:
            yield scrapy.Request(
                url,
                callback=self.parse,
                meta={"proxy": "http://aw_your_username:your_password@proxy.simplyproxies.com:6889"},
                headers={
                    "User-Agent": "Mozilla/5.0 (Linux; Android 14; Pixel 8) "
                                  "AppleWebKit/537.36 Chrome/125.0.6422.113 "
                                  "Mobile Safari/537.36",
                },
                dont_filter=True,
            )

    def parse(self, response):
        for product in response.css("div.product-card"):
            yield {
                "name": product.css("h3::text").get(),
                "price": product.css("span.price::text").get(),
                "url": product.css("a::attr(href)").get(),
            }

        next_page = response.css("a.next-page::attr(href)").get()
        if next_page:
            yield response.follow(
                next_page,
                callback=self.parse,
                meta={"proxy": "http://aw_your_username:your_password@proxy.simplyproxies.com:6889"},
            )

Rate limiting in Scrapy

Don't hammer the target. Set polite download delays:

# settings.py
DOWNLOAD_DELAY = 2
RANDOMIZE_DOWNLOAD_DELAY = True
CONCURRENT_REQUESTS_PER_DOMAIN = 1
AUTOTHROTTLE_ENABLED = True
AUTOTHROTTLE_TARGET_CONCURRENCY = 1.0

Method 3: Selenium (JavaScript-Heavy Sites)

Some sites require a real browser to render content. Selenium with ChromeDriver supports proxy authentication:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service

options = Options()
options.add_argument("--proxy-server=http://proxy.simplyproxies.com:6889")
options.add_argument("--headless=new")
options.add_argument(
    "--user-agent=Mozilla/5.0 (Linux; Android 14; Pixel 8) "
    "AppleWebKit/537.36 (KHTML, like Gecko) "
    "Chrome/125.0.6422.113 Mobile Safari/537.36"
)

driver = webdriver.Chrome(options=options)

driver.get("https://example.co.uk")
print(driver.title)

driver.quit()

Selenium's built-in proxy support does not handle username/password authentication. For authenticated proxies, use a Selenium extension or switch to Playwright which handles proxy auth natively:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(
        headless=True,
        proxy={
            "server": "http://proxy.simplyproxies.com:6889",
            "username": "aw_your_username",
            "password": "your_password",
        },
    )
    page = browser.new_page()
    page.goto("https://example.co.uk")
    print(page.title())
    browser.close()

Method 4: SOCKS5 Proxy

SOCKS5 is a lower-level protocol that works with any traffic type, not just HTTP. Use it when you need to proxy non-HTTP protocols or want more flexibility:

import requests

proxies = {
    "http": "socks5h://aw_your_username:your_password@proxy.simplyproxies.com:6890",
    "https": "socks5h://aw_your_username:your_password@proxy.simplyproxies.com:6890",
}

response = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=30)
print(response.json())

The socks5h:// prefix (note the h) tells requests to resolve DNS through the proxy, not locally. This requires the PySocks package:

pip install requests[socks]

Scraping Best Practices

1. Match your fingerprint to your IP

If you're using a UK mobile proxy, send a mobile User-Agent. If you're scraping desktop-only content, a desktop User-Agent may be fine — but a mobile User-Agent + mobile IP is the most consistent and hardest to flag.

2. Respect rate limits

Even with a high-trust mobile IP, sending 100 requests per second will trigger behavioural analysis. Keep requests to 1-2 per second for most targets. Add random delays between requests.

3. Rotate your IP when blocked

Mobile IPs rotate naturally when the device reconnects to the network. If you hit a block, wait a few minutes for a natural rotation.

4. Handle CAPTCHAs gracefully

Even mobile IPs can trigger CAPTCHAs on very aggressive targets. Build your scraper to detect CAPTCHA responses (HTTP 403, known CAPTCHA page content) and back off rather than hammering the endpoint.

5. Use sessions for sequential requests

When scraping a site that requires login or tracks session state, use requests.Session() to persist cookies across requests through the same proxy:

session = requests.Session()
session.proxies = proxies
session.headers.update(headers)

session.get("https://example.co.uk/login")
session.post("https://example.co.uk/login", data={"user": "x", "pass": "y"})
session.get("https://example.co.uk/dashboard")

6. Monitor your usage

Simply Proxies shows per-credential traffic usage in the dashboard. Keep an eye on it — if you're burning through data faster than expected, you may have a loop or a runaway process.

Common UK Scraping Targets

Target	Why Mobile Proxies Help	Recommended Method
E-commerce (Amazon UK, Argos, Currys)	Aggressive anti-bot, geo-pricing	Requests or Scrapy with delays
Property listings (Rightmove, Zoopla)	Rate limiting, geo-restricted listings	Scrapy with autothrottle
Job boards (Indeed UK, Reed)	Geo-specific job listings	Requests with session
Travel (Skyscanner, Booking.com)	Dynamic pricing by region	Playwright for JS-heavy pages
News aggregation	Paywall bypass, geo-restricted content	Requests with rotation
SERP monitoring	Location-specific results	Requests or Selenium

When Mobile Proxies Are Overkill

Not every scraping task needs mobile proxies. If your target:

Has no anti-bot protection
Doesn't geo-restrict content
Doesn't show different content to different device types

...then datacentre or residential proxies will work fine and cost less. Save mobile proxies for targets where IP quality actually matters.

Start Scraping with UK Mobile Proxies

Get 500 MB free. Real UK mobile IPs from genuine UK mobile networks. HTTP + SOCKS5.

Start Free Trial