Whitepages is the best-known people search site in the US: type a name and the free result page lists everyone who matches, with their current address, a phone number, an age range and relatives. For anyone building a lead list or checking contact data, that is structured data waiting to be exported. Here is a Python scraper that turns a name search into a CSV, then opens person pages for more, and the three things that stop the usual approach.
The block: a US-only site behind Cloudflare
import requests
r = requests.get("https://www.whitepages.com/name/John-Smith/Seattle-WA")
# outside the US: the connection is dropped, no error page
# from a datacenter IP: a Cloudflare challenge instead of the results
Whitepages only answers visitors in the United States. Traffic from anywhere else, from datacenter IPs and from most VPNs is dropped at the edge with no redirect and no message. Inside the US, every page sits behind Cloudflare, whose JavaScript challenge checks for a real browser: requests gets "Just a moment..." instead of HTML.
The third limit is the site's own. The free pages show the current address, the primary phone and an age range ("in their 50s"); exact ages, emails, phone history and background reports are behind the paid tier. This scraper only reads what any visitor can see.
The fix: one request from a US residential IP
Send the URL through Scrape.do with super=true (residential IP plus anti-bot handling) and geoCode=us:
import csv
import json
import re
import requests
from bs4 import BeautifulSoup
TOKEN = "<your_token>"
def fetch(url):
response = requests.get("https://api.scrape.do/", params={
"token": TOKEN, "url": url, "super": "true", "geoCode": "us",
}, timeout=90)
response.raise_for_status()
return BeautifulSoup(response.text, "html.parser")
Every page came back with HTTP 200 in about 3 seconds, for 10 credits.
Where the data is: 25 people in one JSON-LD block
The obvious way is to parse the result cards. They are a trap: the phone on each card is masked as "(206) *** ****".
The page also carries a <script type="application/ld+json"> block, the structured data Whitepages publishes for search engines. It is an ItemList of up to 25 schema.org Person items, and each one has the full name, the current street address, the phone numbers in full, an Age Range property, the neighborhood, previous locations, relatives (relatedTo) and the link to the person page:
def jsonld(soup):
for script in soup.find_all("script", type="application/ld+json"):
try:
data = json.loads(script.string)
except (json.JSONDecodeError, TypeError):
continue
for item in data if isinstance(data, list) else [data]:
if isinstance(item, dict):
yield item
def as_list(value):
return value if isinstance(value, list) else [value] if value else []
def phone(value):
digits = re.sub(r"\D", "", value or "")[-10:]
return f"({digits[:3]}) {digits[3:6]}-{digits[6:]}" if len(digits) == 10 else ""
def to_row(person):
address = next((a for a in as_list(person.get("address")) if isinstance(a, dict)), {})
props = {p.get("name"): p.get("value") for p in as_list(person.get("additionalProperty"))}
return {
"name": person.get("name"),
"age_range": props.get("Age Range"),
"address": address.get("streetAddress"),
"city": address.get("addressLocality"),
"state": address.get("addressRegion"),
"zip": address.get("postalCode"),
"phones": [p for p in map(phone, as_list(person.get("telephone"))) if p],
"relatives": [r.get("name") for r in as_list(person.get("relatedTo")) if isinstance(r, dict)],
"url": "https://www.whitepages.com" + person.get("url", "") if person.get("url", "").startswith("/") else person.get("url"),
}
def people_on(soup):
for block in jsonld(soup):
if block.get("@type") == "ItemList":
return [to_row(el["item"]) for el in block.get("itemListElement", [])
if isinstance(el.get("item"), dict) and el["item"].get("@type") == "Person"]
return []
On a fictional test page (no real person) one row looks like this:
{
"name": "Jane Doe",
"age_range": "40s",
"address": "100 Example St Apt 1",
"city": "Seattle",
"state": "WA",
"zip": "98101",
"phones": ["(206) 555-0101"],
"relatives": ["John Doe", "Mary Doe"],
"url": "https://www.whitepages.com/name/Jane-Doe/Seattle-WA/PExample001"
}
From a name to a CSV
A search lives at /name/First-Last, narrowed by state (/name/John-Smith/WA) or by city and state (/name/John-Smith/Seattle-WA). Each page holds up to 25 people and the next one is ?page=2, so the loop stops on a short page:
def search_url(name, city="", state="", page=1):
url = "https://www.whitepages.com/name/" + "-".join(name.split())
if city and state:
url += f"/{'-'.join(city.split())}-{state.upper()}"
elif state:
url += f"/{state.upper()}"
if page > 1:
url += f"?page={page}"
return url
rows = []
for page in range(1, 3):
people = people_on(fetch(search_url("John Smith", "Seattle", "WA", page)))
rows += people
if len(people) < 25:
break
with open("people.csv", "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
writer.writeheader()
for row in rows:
writer.writerow({k: " | ".join(v) if isinstance(v, list) else v for k, v in row.items()})
print(f"Saved {len(rows)} people, {sum(1 for r in rows if r['phones'])} with a phone")
For "John Smith" in Seattle, WA, two pages gave 50 people in 2 requests (20 credits): 50 with an address, 45 with a phone number and 47 with an age range.
Person pages: associates and the rest
Each row links to a person page. Its JSON-LD block is a single Person with the phones under contactPoint, relatives and associates (knows). The address is often only in the HTML (.address-line1, .address-line2), and the age range is in the description text ("is in their 50s"):
def parse_person_page(soup, url):
person = next((b for b in jsonld(soup) if b.get("@type") == "Person"), None)
if person is None:
raise ValueError("No Person JSON-LD: blocked page or not a person page")
line1 = soup.select_one(".address-line1")
line2 = soup.select_one(".address-line2")
age = re.search(r"in (?:their|his|her) (\d{2}s|\d{2}\+)", person.get("description", ""))
return {
"name": person.get("name"),
"age_range": age.group(1) if age else None,
"address": line1.get_text(" ", strip=True) if line1 else None,
"city_state_zip": line2.get_text(" ", strip=True) if line2 else None,
"phones": [phone(c.get("telephone")) for c in as_list(person.get("contactPoint"))],
"relatives": [r.get("name") for r in as_list(person.get("relatedTo")) if isinstance(r, dict)],
"associates": [k.get("name") for k in as_list(person.get("knows")) if isinstance(k, dict)],
"url": url,
}
That is one more request per person, so open person pages only for the people you need.
Skip the boilerplate
The same scraper is open source, with retries for Cloudflare pages, pagination, person pages and a CLI:
pip install whitepages-scraper
export SCRAPEDO_TOKEN=your_token
whitepages-scraper --name "John Smith" --city Seattle --state WA --pages 2 -f csv -o people.csv
whitepages-scraper "https://www.whitepages.com/name/<Name>/<City-ST>/<id>"
Source: github.com/bugrahansss/whitepages-scraper. There is also a no-code version on Apify (Whitepages Scraper) and a browser demo on Hugging Face.
Use it responsibly
This is personal data about real people. Whitepages is not a consumer reporting agency, so don't use it for decisions about employment, housing, credit or insurance (the FCRA regulates those in the US). Honor opt-out requests, follow the privacy laws that apply to you (CCPA, GDPR) and Whitepages' terms, and keep request rates modest.
Run it at scale with Scrape.do
Every page is one Scrape.do call through a US residential IP: no proxy pool, no Cloudflare solving, no headless browser. A search page holds 25 people, so 1,000 people cost about 400 credits. Get 1000 free credits and start scraping with Scrape.do.
Full walkthrough on the Scrape.do blog: Whitepages.com Scraping: Bypass Cloudflare and Extract People Data
Top comments (0)