Building a B2B Prospecting Agent with CrewAI and Web Metadata Extractor

Description and use

A guide to building a B2B prospecting agent using CrewAI and a web metadata extraction API. The solution separates data gathering from decision-making into two agents to improve accuracy and scalability.

Problem

Manual B2B prospecting does not scale, and custom scrapers require handling SSRF safety, HTML parsing, and rate limits before implementing agent logic.

Outcome

A functional two-agent system that separates data extraction from judgment, producing a ranked list of B2B leads with specific technical justifications.

Workflow

• Install crewai and crewai-webmetadata-extractor via pip.
• Obtain a RapidAPI key for the Web Metadata & Contact Extractor API.
• Set the WEBMETADATA_API_KEY environment variable.
• Define a 'Researcher' agent equipped with WebContactsTool and WebMetadataExtractTool to gather raw signals.
• Define a 'Qualifier' agent to analyze the researcher's output and rank companies based on fit.
• Create a sequential Crew process linking the research task to the qualification task.
• Run the crew kickoff to generate a ranked list with justifications.

Evidence

This walks through building a two-agent CrewAI crew that takes a list of company URLs and produces a prioritized, reasoned shortlist

Source

The researcher's tool calls return structured JSON, not free text — which is what lets the qualifier reason about specific fields instead of vibes.

Source