Docs
→
Getting Started
→
API Reference

API Reference

Complete API documentation for all HeadlessX endpoints

HeadlessX provides RESTful APIs for website scraping and search result extraction. All endpoints require API key authentication.

Authentication

Include your API key in the X-API-Key header:

curl -H "X-API-Key: your-api-key-here" \
  http://localhost:3001/api/...

Generate API Key

  1. Open Dashboard at http://localhost:3000
  2. Navigate to Settings → API Keys
  3. Click Generate New Key
  4. Copy and store securely

Website Scraping APIs

Get Raw HTML

Extract HTML without JavaScript execution (fastest method).

Endpoint: POST /api/website/html

Request Body:

{
  "url": "https://example.com",
  "stealth": true,
  "proxy": "http://proxy:port" // Optional
}

Response:

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "title": "Example Domain",
    "html": "<!DOCTYPE html>...",
    "statusCode": 200,
    "timing": {
      "total": 1234,
      "navigation": 890
    }
  }
}

Get HTML with JavaScript

Extract HTML after JavaScript execution and rendering.

Endpoint: POST /api/website/html-js

Request Body:

{
  "url": "https://example.com",
  "stealth": true,
  "waitFor": "networkidle", // Optional: "load" | "domcontentloaded" | "networkidle"
  "timeout": 60000, // Optional: milliseconds
  "proxy": "http://proxy:port" // Optional
}

Response:

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "title": "Example Domain",
    "html": "<!DOCTYPE html>...",
    "statusCode": 200,
    "timing": {
      "total": 2345,
      "navigation": 1200,
      "javascript": 1145
    }
  }
}

Get Markdown Content

Extract clean markdown content (great for LLMs and RAG).

Endpoint: POST /api/website/content

Request Body:

{
  "url": "https://example.com",
  "stealth": true,
  "cleanMarkdown": true // Remove ads, navigation, etc.
}

Response:

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "title": "Example Domain",
    "markdown": "# Example Domain\n\nThis domain is for use in...",
    "text": "Example Domain This domain is for use in...",
    "wordCount": 127,
    "statusCode": 200
  }
}

Capture Screenshot

Take full-page or viewport screenshots.

Endpoint: POST /api/website/screenshot

Request Body:

{
  "url": "https://example.com",
  "stealth": true,
  "fullPage": true, // false for viewport only
  "format": "png", // "png" | "jpeg"
  "quality": 90 // JPEG quality (0-100)
}

Response:

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "screenshot": "base64-encoded-image-data",
    "format": "png",
    "width": 1920,
    "height": 1080
  }
}

Real-time Stream (SSE)

Stream scraping progress in real-time via Server-Sent Events.

Endpoint: POST /api/website/stream

Request Body:

{
  "url": "https://example.com",
  "stealth": true
}

Response (SSE Stream):

event: progress
data: {"status":"initializing","message":"Starting browser..."}

event: progress
data: {"status":"navigating","message":"Loading page..."}

event: progress
data: {"status":"rendering","message":"Executing JavaScript..."}

event: complete
data: {"success":true,"html":"<!DOCTYPE html>..."}

JavaScript Example:

const eventSource = new EventSource('/api/website/stream', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-API-Key': 'your-api-key'
  },
  body: JSON.stringify({ url: 'https://example.com' })
});

eventSource.addEventListener('progress', (e) => {
  const data = JSON.parse(e.data);
  console.log(data.message);
});

eventSource.addEventListener('complete', (e) => {
  const data = JSON.parse(e.data);
  console.log('Done!', data);
  eventSource.close();
});

Google SERP APIs

Extract Search Results

Extract Google search results with zero detection.

Endpoint: POST /api/google-serp/search

Request Body:

{
  "query": "best headless browser",
  "location": "United States", // Optional
  "language": "en", // Optional
  "numResults": 10, // Optional: 1-100
  "stealth": true
}

Response:

{
  "success": true,
  "data": {
    "query": "best headless browser",
    "totalResults": "About 12,400,000 results",
    "searchTime": "0.42 seconds",
    "organicResults": [
      {
        "position": 1,
        "title": "Best Headless Browsers in 2026",
        "link": "https://example.com/article",
        "displayedLink": "example.com › article",
        "snippet": "Discover the top headless browsers...",
        "date": "Jan 15, 2026"
      }
    ],
    "featuredSnippet": {
      "title": "What is a headless browser?",
      "snippet": "A headless browser is...",
      "link": "https://example.com"
    },
    "peopleAlsoAsk": [
      {
        "question": "What is the best headless browser?",
        "answer": "The best headless browser depends on..."
      }
    ],
    "relatedSearches": [
      "puppeteer vs playwright",
      "headless chrome"
    ]
  }
}

Real-time SERP Stream

Stream Google SERP scraping progress.

Endpoint: GET /api/google-serp/stream?query=your+query

Query Parameters:

  • query (required): Search query
  • location: Location (e.g., "United States")
  • language: Language code (e.g., "en")
  • numResults: Number of results (1-100)

Response (SSE Stream):

event: progress
data: {"status":"searching","message":"Opening Google..."}

event: progress
data: {"status":"extracting","message":"Parsing results..."}

event: complete
data: {"success":true,"organicResults":[...]}

Configuration API

Get Configuration

Endpoint: GET /api/config

Response:

{
  "browserHeadless": true,
  "browserTimeout": 60000,
  "maxConcurrency": 5,
  "camoufoxBlockWebrtc": true,
  "camoufoxGeoip": true,
  "camoufoxEnableCache": false
}

Update Configuration

Endpoint: PUT /api/config

Request Body:

{
  "browserHeadless": true,
  "browserTimeout": 120000,
  "maxConcurrency": 10
}

Error Responses

All endpoints return consistent error format:

{
  "success": false,
  "error": {
    "code": "TIMEOUT_ERROR",
    "message": "Page load timeout after 60000ms",
    "details": {
      "url": "https://example.com",
      "timeout": 60000
    }
  }
}

Common Error Codes

CodeDescription
INVALID_URLURL format is invalid
TIMEOUT_ERRORRequest exceeded timeout
NETWORK_ERRORNetwork connection failed
AUTH_ERRORInvalid or missing API key
RATE_LIMITToo many requests
CAPTCHA_DETECTEDCAPTCHA blocking access
SERVER_ERRORInternal server error

Rate Limits

PlanRequests/HourConcurrent
Free1003
Pro1,00010
EnterpriseUnlimited50+

Next Steps