Firecrawl Crawl
firecrawl_crawlStarts a crawl job on a website and extracts content from all pages. **Best for:** Extracting content from multiple related pages, when you need comprehensive coverage. **Not recommended for:** Extracting content from a single page (use scrape); when token limits are a concern (use map + batch_scrape); when you need fast results (crawling can be slow). **Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + batch_scrape for better control. **Common mistakes:** Setting limit or maxDiscoveryDepth too high (causes token overflow) or too low (causes missing pages); using crawl for a single page (use scrape instead). Using a /* wildcard is not recommended. **Prompt Example:** "Get all blog posts from the first two levels of example.com/blog." **Usage Example:** ```json { "name": "firecrawl_crawl", "arguments": { "url": "https://example.com/blog/*", "maxDiscoveryDepth": 5, "limit": 20, "allowExternalLinks": false, "deduplicateSimilarURLs": true, "sitemap": "include" } } ``` **Returns:** Operation ID for status checking; use firecrawl_check_crawl_status to check progress. **Safe Mode:** Read-only crawling. Webhooks and interactive actions are disabled for security.
When to Use
Use this tool when you need to starts a crawl job on a website and extracts content from all pages.
best for: extracting content from multiple related pages, when you need comprehensive coverage. not recommended for: extracting content from a single page (use scrape); when token limits are a concern (use map + batch_scrape); when you need fast results (crawling can be slow). warning: crawl responses can be very large and may exceed token limits. limit the crawl depth and number of pages, or use map + batch_scrape for better control. common mistakes: setting limit or maxdiscoverydepth too high (causes token overflow) or too low (causes missing pages); using crawl for a single page (use scrape instead). using a /* wildcard is not recommended. prompt example: "get all blog posts from the first two levels of example.com/blog." usage example:
{
"name": "firecrawl_crawl",
"arguments": {
"url": "https://example.com/blog/*",
"maxdiscoverydepth": 5,
"limit": 20,
"allowexternallinks": false,
"deduplicatesimilarurls": true,
"sitemap": "include"
}
}
returns: operation id for status checking; use firecrawl_check_crawl_status to check progress. safe mode: read-only crawling. webhooks and interactive actions are disabled for security. . This is part of the Firecrawl provider on xpay✦.
MCP Connection
Connect to xpay✦ to access this tool (and 10+ others):
{
"mcpServers": {
"xpay": {
"url": "https://mcp.xpay.sh/mcp?key=YOUR_API_KEY"
}
}
}
For Claude Code:
claude mcp add --transport http xpay "https://mcp.xpay.sh/mcp?key=YOUR_API_KEY"
How to Execute
Use the xpay✦ meta-tools to run this tool:
xpay_details— Get full input schema:xpay_details("firecrawl/firecrawl_crawl")xpay_run— Execute:xpay_run("firecrawl/firecrawl_crawl", { ...inputs })
Input Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
excludePaths | array | No | |
crawlEntireDomain | boolean | No | |
includePaths | array | No | |
scrapeOptions | object | No | |
deduplicateSimilarURLs | boolean | No | |
allowExternalLinks | boolean | No | |
url | string | Yes | |
maxConcurrency | number | No | |
delay | number | No | |
maxDiscoveryDepth | number | No | |
limit | number | No | |
allowSubdomains | boolean | No | |
ignoreQueryParameters | boolean | No | |
prompt | string | No | |
sitemap | string | No |
Pricing
- Cost: $0.05/call
- Balance check: Use
xpay_balanceto check remaining credits - Get your API key at xpay.tools — $5 free credits included
Related Skills
- Firecrawl (all tools) — 11 tools
- Firecrawl Scrape — $0.03/call
- Firecrawl Map — $0.02/call
- Firecrawl Search — $0.03/call
- Firecrawl Check Crawl Status — $0.03/call
- Firecrawl Extract — $0.05/call
Links
- Tool page: https://xpay.tools/firecrawl/firecrawl-crawl/
- Provider: https://xpay.tools/firecrawl/
- All tools: https://xpay.tools/explore
How to Execute
Input Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
excludePaths | array | No | - |
crawlEntireDomain | boolean | No | - |
includePaths | array | No | - |
scrapeOptions | object | No | - |
deduplicateSimilarURLs | boolean | No | - |
allowExternalLinks | boolean | No | - |
url | string | Yes | - |
maxConcurrency | number | No | - |
delay | number | No | - |
maxDiscoveryDepth | number | No | - |
limit | number | No | - |
allowSubdomains | boolean | No | - |
ignoreQueryParameters | boolean | No | - |
prompt | string | No | - |
sitemap | string | No | - |
Install Skill
Pricing
Cost
$0.05/call
Model
Flat rate
Provider
Firecrawl

