A Model Context Protocol (MCP) server implementation that integrates with FireCrawl for advanced web scraping capabilities.

Features

• Web scraping with JavaScript rendering
• Batch scraping with parallel processing and queuing
• URL discovery and crawling
• Web search with content extraction
• Automatic retries with exponential backoff
• Credit usage monitoring for cloud API
• Comprehensive logging system
• Support for cloud and self-hosted FireCrawl instances
• Mobile/Desktop viewport support
• Smart content filtering with tag inclusion/exclusion

Configuration

Environment Variables

• FIRE_CRAWL_API_KEY: Your FireCrawl API key
  • Required when using cloud API (default)
  • Optional when using self-hosted instance with FIRE_CRAWL_API_URL
• FIRE_CRAWL_API_URL (Optional): Custom API endpoint for self-hosted instances
  • Example: https://firecrawl.your-domain.com
  • If not provided, the cloud API will be used (requires API key)

Usage with Claude Desktop

Add this to your claude_desktop_config.json:

Available Tools

1. Scrape Tool (fire_crawl_scrape)

Scrape content from a single URL with advanced options.

2. Batch Scrape Tool (fire_crawl_batch_scrape)

Scrape multiple URLs with parallel processing and queuing.

Response includes operation ID for status checking:

3. Check Batch Status (fire_crawl_check_batch_status)

Check the status of a batch operation.

4. Search Tool (fire_crawl_search)

Search the web and optionally extract content from search results.

5. Crawl Tool (fire_crawl_crawl)

Start an asynchronous crawl with advanced options.

Logging System

The server includes comprehensive logging:

• Operation status and progress
• Performance metrics
• Credit usage monitoring
• Rate limit tracking
• Error conditions

Example log messages:

[INFO] FireCrawl MCP Server initialized successfully [INFO] Starting scrape for URL: https://example.com [INFO] Batch operation queued with ID: batch_1 [WARNING] Credit usage has reached warning threshold [ERROR] Rate limit exceeded, retrying in 2s...

Error Handling

The server provides robust error handling:

• Automatic retries for transient errors
• Rate limit handling with backoff
• Detailed error messages
• Credit usage warnings
• Network resilience

Example error response: