What Happens After You Click Enrich
From the outside, waterfall enrichment looks simple. Submit a contact, get back verified data. Under the hood, it is a surprisingly complex distributed system that manages API orchestration, conditional routing, error handling, result ranking, and data normalization across 17 or more external data providers. If you are an engineer building enrichment into your product, or a technical buyer evaluating waterfall platforms, understanding this architecture helps you ask better questions and make better decisions.
The Request Lifecycle
A waterfall enrichment request follows a specific lifecycle from submission to result delivery. Here is the full path:
1. Request ingestion. The client submits a contact record via API (name, company, domain, and optionally LinkedIn URL or existing email). The system validates the input, normalizes formatting (lowercase emails, standardized company names), and creates a processing job.
2. Cascade initialization. The system loads the cascade configuration: which providers to query, in what order, with what parameters, and what stopping conditions apply. This configuration may be static (same for all requests) or dynamic (varying by contact geography, industry, or other attributes).
3. Sequential provider querying. The system queries Provider 1 via API. If Provider 1 returns a result that meets the minimum quality threshold, the cascade stops. If not, the system moves to Provider 2, then Provider 3, and so on through the full cascade.
4. Result validation. Each provider's response is validated before being accepted. For email results, this includes format checking, domain verification, and potentially SMTP mailbox verification. For phone numbers, format validation and line-type detection.
5. Result delivery. The validated result is returned to the client, either synchronously (for simple lookups) or asynchronously via webhook (for waterfall queries that take 30-90 seconds to cascade through multiple providers).
Synchronous vs. Asynchronous Processing
The processing model is one of the most important architectural decisions in waterfall enrichment. Single-source tools can return results synchronously because they query one database and respond immediately. Waterfall systems face a timing challenge: cascading through 17+ providers can take 30-90 seconds, which exceeds typical API timeout windows.
The standard solution is asynchronous processing with webhook delivery. The client submits a request and receives an immediate acknowledgment with a job ID. The system processes the cascade in the background. When results are ready, the system sends them to the client's webhook endpoint.
The webhook endpoint on the client side needs to be designed carefully. It should accept the incoming payload quickly (return a 200 response within a few seconds) and process the data asynchronously on its own side. Long-running processing in the webhook handler risks timeouts and missed deliveries.
Some platforms offer a hybrid model: synchronous for the first provider query (returning within 2-3 seconds if Provider 1 finds a match) and asynchronous for the full cascade (returning via webhook if the request needs to cascade deeper). This gives clients fast results for easy lookups while handling complex cascades in the background.
Cascade Logic and Decision Trees
The cascade logic is the core intellectual property of any waterfall platform. At its simplest, it is a sequential if-then chain: if Provider 1 returns nothing, try Provider 2. But production systems add several layers of sophistication.
Confidence scoring. Not all results are equal. A provider might return an email address with low confidence (pattern-matched but not SMTP-verified) or high confidence (confirmed via multiple signals). The cascade might continue even after getting a result if the confidence is below a threshold, seeking a higher-confidence result from the next provider.
Data type routing. The cascade for email enrichment is typically different from the cascade for phone enrichment. Some providers are email-only; others specialize in phone numbers. The system routes different data type requests through different cascades, potentially in parallel.
Geographic routing. A contact at a German company might be routed through a cascade optimized for European data (leading with GDPR-compliant providers with strong EMEA coverage) while a US contact goes through a different cascade. This geographic awareness improves both coverage and compliance.
Provider health monitoring. If a provider's API is returning errors, timing out, or returning unusually low-quality results, the system should detect this and either skip the provider temporarily or move it lower in the cascade. Health checks can be passive (monitoring real request outcomes) or active (periodic test queries).
Error Handling and Retry Logic
With 17+ external API dependencies, errors are not exceptional; they are expected. Every provider will occasionally return errors, time out, or deliver malformed responses. Robust error handling is what separates a reliable waterfall system from a fragile one.
The standard pattern is exponential backoff with jitter. When a provider returns a transient error (HTTP 429 rate limit, HTTP 503 service unavailable), the system waits 1 second, then retries. If the retry fails, it waits 2 seconds, then 4 seconds, then 8 seconds. Adding random jitter (slight variation in wait times) prevents multiple concurrent requests from all retrying simultaneously and creating a thundering herd problem.
Not all errors deserve retries. A 404 (not found) or 400 (bad request) is a permanent failure that will not resolve with retrying. A 500 (server error) might be transient. Rate limit errors (429) definitely are transient and should be retried after the indicated cooldown period.
The system should also implement circuit breakers. If a provider fails consecutively 10 or more times, the circuit breaker trips and the system stops querying that provider entirely for a cooldown period (typically 5-15 minutes). This prevents wasting time and credits on a provider that is clearly experiencing an outage.
Data Normalization
Different providers return data in different formats. One might return a phone number as +1-555-123-4567, another as 15551234567, and a third as (555) 123-4567. Job titles might come back as VP of Marketing, Vice President, Marketing, or VP Marketing. Company names might be Acme Corp, Acme Corporation, or ACME CORP.
A normalization layer sits between the raw provider responses and the final output. It standardizes phone number formatting (typically E.164 international format), normalizes job titles to a consistent taxonomy, standardizes company names, and resolves other formatting inconsistencies.
This normalization is important not just for clean output but for deduplication and conflict resolution. When two providers return results for the same contact, you need standardized data to compare them and decide which result to keep.
Result Ranking and Conflict Resolution
When multiple providers return different data for the same contact, the system needs rules for choosing the best result. Provider A might return john@company.com while Provider B returns j.smith@company.com. Which is correct?
Common resolution strategies:
- Recency: Prefer the result from the provider with the most recently updated data for this contact.
- Agreement: If two providers return the same result, weight it higher than a result from only one provider.
- Provider ranking: Assign static quality tiers to providers based on historical accuracy, and prefer results from higher-tier providers.
- Verification: Run both candidate results through SMTP verification and keep the one that passes. This is the most expensive approach but the most reliable.
Logging and Observability
Given the complexity of orchestrating 17+ external services, comprehensive logging is not optional. Every request should log:
- The input parameters (name, company, domain)
- Which providers were queried and in what order
- Each provider's response time, HTTP status code, and result summary
- Which provider's result was selected and why
- The final output sent to the client
- Total processing time for the full cascade
This data feeds performance monitoring, cost tracking, provider evaluation, and debugging. When a customer reports a bad result, you need to trace back through the cascade and identify which provider returned the incorrect data and why the system accepted it.
Aggregate logging data also drives cascade optimization. If Provider 3 in the cascade is consistently the one finding results that Provider 1 and 2 miss, you might move it to position 1 and save two unnecessary API calls per contact.
Scaling Considerations
For platforms processing hundreds of thousands or millions of enrichment requests, horizontal scaling is essential. The cascade processing is naturally parallelizable: each request is independent and can be processed on any available worker. A message queue (like RabbitMQ, SQS, or Kafka) distributes incoming requests across a pool of worker processes that execute the cascade logic.
The bottleneck is usually external API rate limits. Most enrichment providers limit requests to 1,000-5,000 per hour. When processing 100,000 contacts, you need to respect these limits while maintaining reasonable throughput. Rate limiters per provider, request queuing, and intelligent batching (grouping requests by provider to maximize batch API efficiency) are standard solutions.
At enterprise scale (100K+ records per batch), the cost management layer also needs sophistication. Budget alerts, per-client spending caps, and real-time cost tracking prevent a single large batch from consuming an unexpected amount of provider credits. At $0.02-0.15 per record across 100,000 contacts, a single batch can cost $2,000-15,000, making cost visibility critical.