Bulk Company Lookup API: Batch Enrichment Without Waste

A bulk company lookup API accepts many domains in one request and returns a result per domain. It cuts HTTP overhead and fits CRM clean-ups, list enrichment and nightly refreshes. Design the job around deduplication, batch limits, rate limits and checkpoints, not around raw speed.

When bulk beats single lookups

Single lookups suit real-time events: a signup, a form fill. Bulk suits everything with a list: a CRM export, a conference attendee file, a purchased list you are cleaning. You get fewer round trips, simpler retry logic and easier cost control.

The BusinessMCP Company Data API documents a POST bulk endpoint taking a domains array of 1 to 100 items at $0.02 per item found, with the same data as single lookup. The provider’s enrichment guide adds that a bulk call reserves the full price up front and refunds misses, so balance must cover the whole batch before you send it.

Job design

Stage	What to do	Why
Clean	Normalise, drop free-mail and invalid domains	Fewer wasted slots
Deduplicate	One row per unique domain	Never pay twice
Skip cached	Drop domains with a fresh result	Saves money
Chunk	Split into batches of at most 100	Endpoint limit
Throttle	Stay under the rate limit	Avoid 429s
Checkpoint	Save results after every batch	Resume on failure
Reconcile	Join results back to original rows	Fill your table

Key takeaway: the cheapest lookup is the one you do not make. Cleaning, deduplicating and caching typically save more than any negotiated price.

Respecting limits

The provider’s guide lists 60 requests per minute for bulk and per-unit tools, against 600 per minute per key for ordinary calls. At 100 domains per request and 60 requests per minute the ceiling is 6,000 domains a minute, which is more than most jobs need; design for steady pacing rather than bursts. Add exponential backoff for rate-limit responses and retry only when the error says it is retryable.

Budgeting a big run

Before running 200,000 domains, run a 500-domain sample and measure the hit rate. Then estimate: unique domains times hit rate times $0.02. Because misses are refunded according to the provider, the sample’s hit rate is the main uncertainty. Make sure the balance exceeds the reserved amount for each batch, since the full batch price is reserved first. Our pricing article has the formula and the hidden costs list.

Idempotency and retries

If a request times out, you do not know whether it was processed. Send an idempotency key with each batch so a retry is not billed twice; the provider’s guide recommends this. Use a deterministic key such as a hash of the batch’s sorted domains plus the job id.

Handling the output

Each result should include a found flag. Write three outcomes to your table: found with fields, found partial, not found. Add a fetched-at date. Do not collapse not-found into empty strings; keep it as an explicit status so later jobs can decide when to retry (weeks, not days).

Verifying quality on a big list

Bulk makes it easy to ingest garbage at scale. After the run:

1. Compute found rate overall and by segment (country, size).
2. Randomly sample 50 found records and check them against the companies’ websites.
3. Look for suspicious clusters, such as many domains resolving to the same name.
4. Check brand-platform domains; the provider says google.com-type domains return name only.
5. Spot-check a few not-found domains by hand to see whether they are genuinely obscure.

Combining with other endpoints

After a bulk pass you may want to drop dead sites using the website status endpoint or pull a candidate pool with company search. Keep these as separate jobs with separate budgets, since they are priced differently.

Where bulk fits in a CRM

For a recurring CRM refresh, select accounts with old last-enriched dates, run the job weekly and write results under field-ownership rules. The CRM integration guide covers mapping and overwrite policy.

A minimal pseudocode outline

domains = unique(clean(source))
domains = [d for d in domains if not cache.fresh(d)]
for batch in chunks(domains, 100):
    ensure balance >= batch_price
    resp = post_bulk(batch, idempotency_key=hash(batch))
    store(resp)         # includes found flag per domain
    sleep(pacing)

Keep the loop boring and resumable. A job you can safely restart at 3 a.m. is better than a clever one you cannot.

Common failure modes in large jobs

Partial failure with no checkpoint. A job dies at batch 1,400 of 2,000 and restarts from zero, re-paying for everything. Fix: write each batch’s results to disk or a table before moving on, and have the job skip any domain that already has a result.

Silent truncation. A batch of 150 domains is sent to an endpoint that accepts 100. Depending on the API this errors or drops items. Fix: assert batch size in code and test with exactly 100 and 101 items.

Case and whitespace mismatches. The API returns results keyed by the domain you sent, but your join key is the original messy value. Fix: keep an explicit mapping from original value to cleaned domain and join through it.

Treating a refunded miss as an error. Under a found-only billing model a miss is a normal outcome. Alerting on it will page you for nothing; alert on error responses instead.

Cost surprise at the end. Fix: log running spend per batch from the response metadata or cost headers; the provider’s guide says cost headers and a meta block are available for tracking spend.

Verifying the first batch before the rest

Always run one batch of 100 and inspect it by hand. Check that found and not-found counts look plausible, that fields land in the right columns, and that the cost matches your estimate at the listed $0.02 per found item. Only then release the full job. The API reference documents the response schema, so confirm field names there instead of reverse-engineering them from one sample.

Scheduling and refresh policy

For a standing database, bulk jobs become a refresh policy. A simple design is three tiers: accounts touched this month are refreshed if older than 30 days; active pipeline accounts are refreshed if older than 14 days; everything else is refreshed quarterly. That keeps spend proportional to value. Because misses are cheap to repeat under found-only billing, you can retry not-found domains monthly in case the provider’s coverage has grown, though the provider’s guide suggests a week of caching for misses as a baseline.

What to do with the results next

Bulk output is raw material. Feed it to the CRM integration under field-ownership rules, and run a website status pass if you plan outreach. If you need to find more companies like the ones you just enriched, switch to company search with filters learned from your best accounts.