Olostep Start Crawl
olostep_start_crawlStarts a new crawl. You receive a `id` to track the progress. The operation may take 1-10 mins depending upon the site and depth and pages parameters.
When to Use
Use this tool when you need to starts a new crawl. you receive a id to track the progress. the operation may take 1-10 mins depending upon the site and depth and pages parameters.. This is part of the Olostep API provider on xpay✦.
MCP Connection
Connect to xpay✦ to access this tool (and 11+ others):
{
"mcpServers": {
"xpay": {
"url": "https://mcp.xpay.sh/mcp?key=YOUR_API_KEY"
}
}
}
For Claude Code:
claude mcp add --transport http xpay "https://mcp.xpay.sh/mcp?key=YOUR_API_KEY"
How to Execute
Use the xpay✦ meta-tools to run this tool:
xpay_details— Get full input schema:xpay_details("olostep/olostep_start_crawl")xpay_run— Execute:xpay_run("olostep/olostep_start_crawl", { ...inputs })
Input Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
include_urls | array | No | URL path patterns to include in the crawl using glob syntax. Defaults to /** which includes all URLs. Use patterns like /blog/** to crawl specific sections (e.g., only blog pages), /products/*.html for product pages, or multiple patterns for different sections. Supports standard glob features like * (any characters) and ** (recursive matching). |
top_n | number | No | An optional number to only crawl the top N most relevant links on every page as per search query. |
webhook_url | string | No | An optional POST request endpoint called when this crawl is completed. The body of the request will be same as the response of this v1/crawls/{crawl_id} endpoint. |
max_depth | number | No | Maximum depth of the crawl. Useful to extract only up to n-degree of links. |
include_external | boolean | No | Crawl first-degree external links. |
start_url | string | Yes | The starting point of the crawl. |
include_subdomain | boolean | No | Include subdomains of the website. false by default. |
exclude_urls | array | No | URL path names in glob pattern to exclude. For example: /careers/**. Excluded URLs will supersede included URLs. |
max_pages | number | Yes | Maximum number of pages to crawl. Recommended for most use cases like crawling an entire website. |
search_query | string | No | An optional search query to find specific links and also sort the results by relevance. |
timeout | number | No | End the crawl after n seconds with the pages completed until then. May take ~10s extra from provided timeout. |
Pricing
- Cost: $0.10/call
- Balance check: Use
xpay_balanceto check remaining credits - Get your API key at xpay.tools — $5 free credits included
Related Skills
- Olostep API (all tools) — 12 tools
- Olostep Create Scrape — $0.02/call
- Olostep Create Answer — $0.10/call
- Olostep Maps — $0.02/call
- Olostep Start Batch — $0.02/call
- Olostep Batch Items — $0.02/call
Links
- Tool page: https://xpay.tools/olostep/olostep-start-crawl/
- Provider: https://xpay.tools/olostep/
- All tools: https://xpay.tools/explore
How to Execute
Input Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
include_urls | array | No | URL path patterns to include in the crawl using glob syntax. Defaults to `/**` which includes all URLs. Use patterns like `/blog/**` to crawl specific sections (e.g., only blog pages), `/products/*.html` for product pages, or multiple patterns for different sections. Supports standard glob features like * (any characters) and ** (recursive matching). |
top_n | number | No | An optional number to only crawl the top N most relevant links on every page as per search query. |
webhook_url | string | No | An optional POST request endpoint called when this crawl is completed. The body of the request will be same as the response of this [`v1/crawls/{crawl_id}`](./info#response-created) endpoint. |
max_depth | number | No | Maximum depth of the crawl. Useful to extract only up to n-degree of links. |
include_external | boolean | No | Crawl first-degree external links. |
start_url | string | Yes | The starting point of the crawl. |
include_subdomain | boolean | No | Include subdomains of the website. `false` by default. |
exclude_urls | array | No | URL path names in glob pattern to exclude. For example: `/careers/**`. Excluded URLs will supersede included URLs. |
max_pages | number | Yes | Maximum number of pages to crawl. Recommended for most use cases like crawling an entire website. |
search_query | string | No | An optional search query to find specific links and also sort the results by relevance. |
timeout | number | No | End the crawl after n seconds with the pages completed until then. May take ~10s extra from provided timeout. |
Install Skill
Pricing
Cost
$0.10/call
Model
Flat rate
Provider
Olostep API