WebScraping.AI
WebScraping.AI provides URL-to-HTML and AI question endpoints with optional JavaScript rendering, proxy selection, custom headers, and plain-text answers for webpage questions.
Connect this API to an AI agent Browse all specs
What you can do with WebScraping.AI via MCP
Every endpoint below becomes a governed tool your AI agent (Claude, ChatGPT, or a customer-facing VX agent) can call — with scoped permissions, sandbox testing, and audit logs.
| Method | Path | Operation | Description |
|---|---|---|---|
| GET | /html |
getHtml | Fetch HTML Method: GET Path: /html IMPORTANT: This function has 2 REQUIRED parameter(s) and 4 OPTIONAL parameter(s) REQUIRED parameters MUST be provided OPTIONAL parameters can be omitted if not needed Parameters: Query Parameters: REQUIRED: - api_key: WebScraping.AI API key. - url: Target URL to fetch. OPTIONAL: - js: Enable JavaScript rendering. - proxy: Proxy type such as datacenter, residential, or stealth. - timeout: Request timeout in milliseconds. - headers: Custom request headers using the documented parameter encoding. |
| GET | /ai/question |
getAiQuestion | Ask a question about a URL Method: GET Path: /ai/question IMPORTANT: This function has 3 REQUIRED parameter(s) and 4 OPTIONAL parameter(s) REQUIRED parameters MUST be provided OPTIONAL parameters can be omitted if not needed Parameters: Query Parameters: REQUIRED: - api_key: WebScraping.AI API key. - url: Target URL to fetch. - question: Question to answer from the target webpage. OPTIONAL: - js: Enable JavaScript rendering. - proxy: Proxy type such as datacenter, residential, or stealth. - timeout: Request timeout in milliseconds. - headers: Custom request headers using the documented parameter encoding. |
Usage guide
## Authentication Create a WebScraping.AI API key at https://webscraping.ai. Pass it as the required `api_key` query parameter. Keep the key in a backend secret store; do not expose it in browser code or commit it to source control. ## Operations - `GET /html` fetches a target URL and returns HTML, which can be converted to Markdown or text by the caller. - `GET /ai/question` fetches a target URL and returns a plain-text answer to a question about that page. ## Example ```bash curl -G "https://api.webscraping.ai/html" \ --data-urlencode "api_key=$WEB_SCRAPING_AI_KEY" \ --data-urlencode "url=https://example.com" \ --data-urlencode "js=true" curl -G "https://api.webscraping.ai/ai/question" \ --data-urlencode "api_key=$WEB_SCRAPING_AI_KEY" \ --data-urlencode "url=https://example.com" \ --data-urlencode "question=What is this page about?" ``` Official documentation: https://webscraping.ai/docs
How to use this spec
- Create a free VX Agents account (sandbox tools included).
- Open the marketplace and connect “WebScraping.AI” to an agent.
- Expose it as an MCP server for Claude/ChatGPT, or chat with it directly.