B2B data extraction: get structured company and contact data at scale
Why scraping profiles and websites is slow and fragile, and how to pull the same company and contact data as structured JSON from an API.
Dag Holmen · Updated · 5 min read
B2B data extraction means getting company and contact records out of wherever they live and into your own systems as clean, structured rows. Most teams start by scraping: a script that visits company websites, directories or profile pages and parses out names, titles and company details. It works for a while. Then a layout changes, requests start getting blocked, and the person who wrote the scraper spends Fridays fixing it.
The alternative is to query data that's already structured. This guide is fair about where scraping makes sense, explains why it gets expensive at scale, and shows how to pull the same people and company data as JSON from the B2Benrich API.
What B2B data extraction usually involves
A typical scraping pipeline has four parts:
- Collect. Fetch pages: team pages, directories, search results, profiles.
- Parse. Pull fields out of HTML with selectors or a model.
- Normalize. Turn "VP, Mktg" and "Vice President of Marketing" into one value, and "Acme Inc." and "ACME" into one company.
- Maintain. Rerun it, fix what broke, and work out which records changed.
Steps 3 and 4 are where most of the time goes, and they never really finish.
The practical problems with scraping
It breaks. Scrapers depend on the structure of someone else's page. When a site redesigns, renames a CSS class or moves content behind a script, your selectors return nothing, often silently. You find out when a list comes back half empty.
It gets blocked. Sites that don't want automated traffic detect and limit it, with rate limits, CAPTCHAs and IP blocks. Working around that means proxies, headless browsers and retry logic, which adds cost and still fails some of the time.
Terms of service apply. Many websites' terms restrict automated collection, and professional networks are usually strict about it. Whether a given scraping project is allowed depends on the site, the data and where you operate. Read the terms and ask your own counsel. This guide isn't legal advice.
The output is unstructured. A scraper gives you strings. Job titles are free text, company names are inconsistent, locations come in every format. There are no stable IDs, so deduping across runs and joining people to companies is guesswork.
You only see one source. A scraped team page shows who the company chose to list. It won't tell you headcount, revenue band or industry unless you scrape and match those somewhere else too.
Scraping is still the right tool for some jobs: content only one site has (a pricing page, a public changelog, job posts on a careers page), a one-off research pull, or data you have clear permission to collect. For people and company records, which many teams need in the same shape, it's usually cheaper to query them.
Scraping vs querying an API
| Scraping | Structured data API | |
|---|---|---|
| Output | HTML you parse into strings | JSON records with named fields |
| Maintenance | Breaks when pages change | The request stays the same |
| Blocking | Proxies, CAPTCHAs, retries | An API key in a header |
| Filtering | Whatever the page exposes | Title, level, function, industry, size, location |
| Dedupe | Fuzzy matching on names | Stable id and company_id |
| Normalized values | You build the mapping | Allowed values listed by the meta endpoint |
Extract the people at a company
The most common scraping job is "get everyone on the team page". With the API, that's /v1/search/company-employees, by domain, LinkedIn URL or company ID, with optional filters like job_function, job_level or country_name.
import os, csv, requests
BASE = "https://api.b2benrich.com"
HEADERS = {"X-Api-Key": os.environ["B2BENRICH_KEY"]}
FIELDS = ["id", "full_name", "job_title", "linkedin_url", "company_name",
"company_domain", "city", "country_name"]
def extract_team(domain, page_size=50, **filters):
page = 1
while True:
r = requests.post(f"{BASE}/v1/search/company-employees", headers=HEADERS,
json={"domain": domain, "page": page, "page_size": page_size, **filters})
r.raise_for_status()
records = r.json()["records"]
yield from records
if len(records) < page_size:
return
page += 1
with open("acme_team.csv", "w", newline="") as f:
w = csv.DictWriter(f, fieldnames=FIELDS, extrasaction="ignore")
w.writeheader()
for person in extract_team("acme.com", job_function="Engineering"):
w.writerow(person)
Every row has the same columns, an id you can dedupe on, and a company_id that joins back to the company record. The people search API guide covers the other people endpoints, including title search and plain-English AI search.
Extract companies that match criteria
Instead of crawling a directory, describe the companies you want and page through /v1/search/companies:
curl --request POST \
--url https://api.b2benrich.com/v1/search/companies \
--header 'Content-Type: application/json' \
--header 'X-Api-Key: YOUR_API_KEY' \
--data '{
"industry_linkedin": "Financial Services",
"headquarters_country_name": "United Kingdom",
"page": 1,
"page_size": 100
}'
Each record has name, domain, industry, hq_city, hq_country, employees_range, revenue_range and linkedin_url, and the response includes total_count so you know the size up front. Get exact industry values from /v1/search/meta/industry_linkedin. The company search API guide covers every filter.
Extract from identifiers you already have
If you have a column of LinkedIn profile URLs or company domains, you don't need to visit those pages. Send them to the enrichment endpoints and get structured records back:
/v1/enrich/person/linkedin-to-personwith a LinkedIn profile URL returns the person's profile./v1/enrich/companies/linkedin-to-companywith a LinkedIn company URL returns the company./v1/enrich/companies/domain-to-companywith a domain returns the company record.
That covers most of what people mean by "LinkedIn data extraction" without your code ever loading a LinkedIn page.
Extraction vs enrichment
Extraction, in this sense, starts from criteria and discovers records you didn't have. Enrichment starts from a record you have and fills in the missing fields. Most pipelines do both: search to build the list, enrichment to complete records that arrive from forms, events and imports.
Running it as a pipeline
Write results to JSONL or a warehouse table rather than straight to a CRM, so you keep the raw records. Store the request body next to each run. Key people on id and companies on domain or company_id, and diff each run against the last to find what changed. B2Benrich is unlimited API usage for $500 a month, with no credits, so rerunning the same extraction weekly costs nothing extra. For a broader tour of what's in the data, see the B2B data API guide.
Frequently asked questions
- What is B2B data extraction?
- B2B data extraction is getting company and contact records, such as names, titles, company details and locations, out of their sources and into your own systems as structured data. It’s done by scraping websites or by querying a data API that already holds the records in structured form.
- Is it legal to scrape B2B data?
- It depends on the website, its terms of service, the kind of data and the laws where you operate. Many sites restrict automated collection in their terms. Read them and get your own legal advice before you scrape.
- What’s an alternative to web scraping for B2B data?
- Query a B2B data API. You send filters or identifiers and get JSON records back with consistent fields and stable IDs, without parsing HTML, managing proxies or fixing selectors when pages change.
- How do I extract company data from a list of domains?
- Send each domain to /v1/enrich/companies/domain-to-company. It returns the company record, which you can write to a CSV or database keyed on the domain.
- What’s the difference between data extraction and data enrichment?
- Extraction starts from criteria and finds new records, such as every engineer at a company. Enrichment starts from a record you already have, such as a domain or LinkedIn URL, and fills in the missing fields.
- What format does extracted data come in?
- The B2Benrich API returns JSON. You can turn it into CSV, load it into a warehouse or pass it to any tool that accepts JSON.
Related guides
- B2B data API: what’s in it and how to query itWhat a B2B data API gives you that a database UI doesn’t, the people and company data in B2Benrich, and how to query it with one key.
- People search API: find contacts by title, company and locationSearch for people by job title, company, seniority and location, list everyone at a company, or describe who you want in plain English.
- Company search API: find accounts by industry, size and locationSearch companies by name, industry, headcount, revenue, location and founding year, and turn the results into an account list.