Crabill LeadGen — Pipeline Map

Pipeline Overview

—

Total Leads

—

With Email

—

Verified

—

Premium Tier

Conversion Funnel

Connect to EDDIE to see live data...

Last 24 Hours

—

Leads Scraped

—

Contacted

—

Replies

Conversion Rates

—

Contacted → Replied

—

Replied → Booked

—

Booked → Show

—

Show → Closed

Top Verticals

Connect to EDDIE to see live data...

Runs daily at 7am — 9 stages, fully automated

Scrape Leads

Pull new businesses from OpenStreetMap across all 48 cities

osm-scraper.ts --all
Free, unlimited. Queries Overpass API. Maps 18 verticals to OSM tags. Deduplicates on (business_name, city, state).

Extract Emails + LinkedIn

Crawl business websites for contact info

scrape-emails.ts --limit 500
Fetches homepage + /contact, /about pages. Extracts emails + LinkedIn URLs. Zero API cost.

Score Leads

100-point rules engine → premium / standard / skip

score-deterministic.ts
Vertical: 30pts • Reviews: 20pts • Rating: 15pts • Website: 10pts • Email: 10pts • DM name: 10pts • Phone: 5pts
Premium: 70-100 • Standard: 50-69 • Skip: 0-49

Generate Icebreakers

Personalized opening lines from 24 template variants

generate-icebreakers-template.ts
6 scenarios × 4 variants. Deterministic selection by lead ID.

Verify Emails

MX record check, syntax validation, disposable filter

verify-emails.ts
Uses dig for MX lookups (free). Filters disposable domains (26 blocked) and role addresses. Invalid emails auto-suppressed.

Export Premium → Instantly

Push top-tier verified leads to premium email campaign

export-instantly.ts --tier premium
Campaign 5dbd3854. Filters: verified, not bounced, not suppressed, not contacted. Batches of 500.

Export Standard → Instantly

Push mid-tier leads to standard email campaign

export-instantly.ts --tier standard
Campaign bc52b241. Same filters, separate email sequence.

Pipeline Hygiene

Auto-correct status mismatches across the database

pipeline-hygiene.ts
Bounced→lost, stale 30d→lost, positive reply 7d→qualified, meeting done→show, orphaned booked_at→booked.

Process Touchpoints

Fire drip sequences for inbound calculator leads

process-touchpoints.ts
6-stage drip for statement uploaders (AI results → confirmation → progress → value nugget → anticipation → quote ready).
4-stage followup for non-uploaders.

Lead Lifecycle

new

→

scraped

→

enriched

→

scored

→

contacted

→

replied

→

qualified

→

booked

→

show

→

closed

Any stage can also →

lost

(bounce, no_response, negative reply)

Free — Runs Daily (Unlimited)

OpenStreetMap / Overpass APIFree • Unlimited

Primary scraper. 48 cities × 18 verticals. No rate limits, no API key needed. Runs every morning.

osm-scraper.ts
Queries Overpass QL with bounding boxes (~15km per city). Maps verticals to OSM tags. This is the workhorse.

Free Tiers Available — Not Yet Activated

These APIs have generous free tiers we should be using alongside OSM for better data coverage.

Google Maps Places$200/mo Free Credit

Best data quality — ratings, reviews, photos, hours. ~10K free calls/month via Google Cloud credit.

gmaps-scraper.ts — needs GOOGLE_API_KEY
After free credit: ~$5/1K calls. Budget-gated via api-tracker. Already built and tested.

Yelp Fusion5,000/day Free

Rich review data, business categories, Yelp URLs. 5,000 API calls per day for free.

yelp-scraper.ts — needs YELP_API_KEY
Free tier is very generous for our volume. Captures yelp_url and yelp_id for enrichment.

HERE Maps250K/mo Free

Nearby search with business data. 250,000 free transactions per month — more than enough.

here-scraper.ts — needs HERE_API_KEY
Free tier is massive. Sign up at developer.here.com.

Foursquare PlacesFree Tier

Venue data by category. Free developer tier available — good for restaurant/retail verticals.

foursquare-scraper.ts — needs FOURSQUARE_API_KEY
Sign up at developer.foursquare.com for free tier.

Bing MapsFree Tier

Alternative search source via Azure Maps. Basic plan available with free transactions.

bing-scraper.ts — needs BING_MAPS_KEY
Via Azure portal. Good fallback when Google quota is exhausted.

State Business RegistryFree

Secretary of State databases. Verifies official business registration. Currently FL, TX, TN only.

state-registry-scraper.ts
Free public data. Limited to 3 states — needs expansion.

Inbound Leads

crabillchamp.com CalculatorInbound Webhook

Leads submit their processing volume → get savings estimate → enter our pipeline. Auto-links to existing leads. Triggers touchpoint drip.

calculator-webhook.ts
Captures: email, industry, monthly volume, savings estimate, contact preference, phone, has_statement.
If has_statement → immediate 6-stage drip. Otherwise → 24h followup.

Always listening — event-driven, real-time

Instantly — Email EventsReal-time

Tracks every email interaction. Auto-classifies reply sentiment. Telegram alert on positive responses. Auto-suppresses negative.

instantly-webhook.ts

Bounced → Mark bounce + suppress
Replied → Classify sentiment (Haiku), Telegram alert if positive, suppress if negative
Opened/Clicked → Record timestamp
Unsubscribed/Complained → Suppress

All events logged to lead_communications.

Cal.com — BookingsReal-time

Meeting booked, cancelled, or rescheduled → update lead status, link FKs, Telegram alert.

calcom-webhook.ts
Creates cal_booking record, sets lead to "booked", backfills FK columns.

Calculator — InboundReal-time

New lead from savings calculator → create record, link to existing leads, start drip.

calculator-webhook.ts
Creates calculator_lead, auto-links if email matches crabill_leads.

Search Leads

Search for leads or click Search to browse all...

System Health Checks

Click "Run Check" or connect to EDDIE to see system health...

Scheduled Reports (via Telegram)

Morning Pipeline Review8am Daily

Hot leads, today's meetings, new inbound, at-risk leads, funnel stats, conversion rates.

pipeline-review.ts
Queries all lead tables. Sent to Telegram at 8am.

Call Prep Briefs8am Daily

Per-meeting brief: company overview, score, savings, interaction history, talking points.

call-prep.ts
One Telegram message per Cal.com booking today.

Win/Loss AnalysisMonthly

Win rate by vertical/city/source/tier, loss reasons, pipeline velocity, stage drop-off.

win-loss-analysis.ts
Runs 1st of month at 9am. Telegram + markdown report in Brain Vault.

On-Demand CLI

Mine Lost LeadsFree

Find stale/lost leads worth re-engaging. Generate follow-up lines. Reset for re-campaign.

mine-lost-leads.ts
Flags: --dry-run, --export-csv, --limit N

Sync Instantly EmailsFree

Pull sent email bodies from Instantly into communication log for full thread context.

sync-instantly-emails.ts
Flags: --dry-run, --since YYYY-MM-DD

Suppression ListFree

Manage global email blocklist — add, remove, import bulk, check, stats.

suppress.ts

API Budget TrackerFree

View API usage and enforce spending limits per service.

api-tracker.ts

Core Tables

crabill_leadsPrimary lead table — 50+ columns, full lifecycle—

calculator_leadsInbound from crabillchamp.com calculator—

cal_bookingsCal.com meeting records—

suppression_listBounce, unsubscribe, complaint blocklist—

CRM Intelligence

lead_communicationsEvery email sent/received per lead—

lead_activitiesNotes, research, call prep snapshots0

lead_stage_historyAuto-tracked status transitions—

Config

leadgen-config.tsConfig

48 cities, 18 verticals, 2 Instantly campaign profiles

Active in campaigns: Dental, Medical, Legal, Home Services, Chiropractor, CPA, Physical Therapy, Financial Advisor, Auto Repair, Restaurants, Retail
Scraped but not in campaign: Salon, Spa, Veterinary, Gym, Real Estate, Optometry, Insurance

Quick Wins — Free, Just Need API Keys

Activate Free API TiersUpgrade

Sign up for free tiers on Google Maps ($200/mo credit), Yelp (5K/day), HERE (250K/mo), Foursquare, Bing. Scripts are already built.

All scrapers are built, tested, and budget-gated. Just need API keys added to .env.baremetal on homelabx. Could 5-10x our data quality overnight.

Add 7 Verticals to CampaignUpgrade

Salon, Spa, Vet, Gym, Real Estate, Optometry, Insurance — already scraped and scored, just not exported to Instantly yet.

Add to crabill campaign profile in leadgen-config.ts. These leads are sitting in the DB ready to go.

Fixes Needed

Hardcoded dan@crabillchamp.com Fix

Sender email hardcoded in 3 files. Should be an env var for reusability.

Files: instantly-webhook.ts, touchpoint-scheduler.ts, non-uploader-followup.ts
Move to LEADGEN_FROM_EMAIL env var.

GMaps Scraper Overwrites Data Fix

Uses bare EXCLUDED.website instead of COALESCE — can overwrite enriched data with nulls.

gmaps-scraper.ts ON CONFLICT should use COALESCE to preserve existing data, like other scrapers do.

lead_activities Never Written To Fix

Table exists but no code writes to it. Call prep, research, and notes should populate this.

Need: call-prep to write a summary after generating, research scripts to log findings, manual note-taking via Telegram.

Code Cleanup

Deduplicate CITY_COORDSCleanup

48-city coordinate map duplicated in 3 scraper files. Should be in leadgen-config.ts.

Files: osm-scraper.ts, bing-scraper.ts, here-scraper.ts all have identical city coordinate maps.

mine-lost-leads Resets to "new"Cleanup

Re-engaged leads go back to "new" and get fully reprocessed. Should use a "re-engage" status instead.

Add 're-engage' to status CHECK constraint, skip scoring/icebreaker stages for these leads, export directly.

Batch DB UpdatesCleanup

Several scripts update leads one-at-a-time. Batch UPDATE would be much faster at scale.

Affects: score-deterministic.ts, generate-icebreakers-template.ts, verify-emails.ts

Future Upgrades (Paid)

LinkedIn Profile EnrichmentPaid Upgrade

Enrich premium leads with LinkedIn profile data (title, company size, about section). ~$0.005-0.01/call.

Options: Netrows ($0.005/call), Proxycurl ($0.01/call). For premium-tier leads only — maybe 200/month = $1-2/mo.

LinkedIn OutreachPaid Upgrade

Add LinkedIn as an outreach channel alongside email. Connect requests + DMs.

Options: PhantomBuster ($69/mo), Unibox. Adds a second touch channel.

AI Scoring + AI IcebreakersPaid Upgrade

Use Claude API for richer scoring and more personalized icebreakers on premium leads.

Scripts already built: score-leads.ts and generate-icebreakers.ts. Just need API credits.

Dashboard Read EndpointsEnhancement

Add /api/leads, /api/funnel, /api/pipeline-stats to EDDIE's web dashboard for browser-based monitoring.

Currently all visibility is through CLI scripts and Telegram. A web dashboard would make this accessible to Dan.