---
name: "xpay-tavily-tavily-crawl"
description: "Crawl a website starting from a URL. Extracts content from pages with configurable depth and breadth."
version: "1.0.0"
price: "$0.02/call"
provider: "Tavily Search"
---

# Tavily Crawl

Crawl a website starting from a URL. Extracts content from pages with configurable depth and breadth.

**Provider:** Tavily Search
**Price:** $0.02/call
**Pricing model:** Flat rate per call

## When to Use

Use this tool when you need to crawl a website starting from a url. extracts content from pages with configurable depth and breadth.. This is part of the Tavily Search provider on xpay✦.

## MCP Connection

Connect to xpay✦ to access this tool (and 5+ others):

```json
{
  "mcpServers": {
    "xpay": {
      "url": "https://mcp.xpay.sh/mcp?key=YOUR_API_KEY"
    }
  }
}
```

For Claude Code:
```bash
claude mcp add --transport http xpay "https://mcp.xpay.sh/mcp?key=YOUR_API_KEY"
```

## How to Execute

Use the xpay✦ meta-tools to run this tool:

1. **`xpay_details`** — Get full input schema: `xpay_details("tavily/tavily_crawl")`
2. **`xpay_run`** — Execute: `xpay_run("tavily/tavily_crawl", { ...inputs })`

## Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `instructions` | string | No | Natural language instructions for the crawler. Instructions specify which types of pages the crawler should return. |
| `select_domains` | array | No | Regex patterns to restrict crawling to specific domains or subdomains (e.g., ^docs\.example\.com$) |
| `include_favicon` | boolean | No | Whether to include the favicon URL for each result |
| `select_paths` | array | No | Regex patterns to select only URLs with specific path patterns (e.g., /docs/.*, /api/v1.*) |
| `max_depth` | integer | No | Max depth of the crawl. Defines how far from the base URL the crawler can explore. |
| `limit` | integer | No | Total number of links the crawler will process before stopping |
| `format` | string | No | The format of the extracted web page content. markdown returns content in markdown format. text returns plain text and may increase latency. |
| `extract_depth` | string | No | Advanced extraction retrieves more data, including tables and embedded content, with higher success but may increase latency |
| `allow_external` | boolean | No | Whether to return external links in the final response |
| `url` | string | Yes | The root URL to begin the crawl |
| `max_breadth` | integer | No | Max number of links to follow per level of the tree (i.e., per page) |

## Pricing

- **Cost:** $0.02/call
- **Balance check:** Use `xpay_balance` to check remaining credits
- Get your API key at [xpay.tools](https://xpay.tools/account/settings/api-keys/) — $5 free credits included

## Related Skills

- [Tavily Search (all tools)](https://xpay.tools/skills/tavily/SKILL.md) — 6 tools
- [Tavily Search](https://xpay.tools/skills/tavily/tavily-search/SKILL.md) — $0.02/call
- [Tavily Extract](https://xpay.tools/skills/tavily/tavily-extract/SKILL.md) — $0.02/call
- [Tavily Map](https://xpay.tools/skills/tavily/tavily-map/SKILL.md) — $0.02/call
- [Tavily Research](https://xpay.tools/skills/tavily/tavily-research/SKILL.md) — $0.02/call
- [Tavily Skill](https://xpay.tools/skills/tavily/tavily-skill/SKILL.md) — $0.02/call

## Links

- Tool page: https://xpay.tools/tavily/tavily-crawl/
- Provider: https://xpay.tools/tavily/
- All tools: https://xpay.tools/explore