Overview
Overview of the HeadlessX backend API surface, auth model, operator-first route families, and transport patterns.
by @saifyxpro
This section documents the current backend HTTP surface for the HeadlessX API service.
API shape
The backend exposes three transport patterns:
- standard REST routes under
/api/* - Server-Sent Events for long-running or progressive workflows
- a remote MCP endpoint at
/mcp
Route families
| Area | Base path | Purpose |
|---|---|---|
| Health | /api/health | public health and route summary |
| System | /api/config, /api/dashboard, /api/logs, /api/keys, /api/proxies | configuration and operations |
| Operators | /api/operators/status | operator catalog and availability |
| Website | /api/operators/website/* | scraping, map, crawl, content, screenshot, streams |
| Search | /api/operators/google/ai-search/*, /api/operators/tavily/*, /api/operators/exa/* | search and research workflows |
| YouTube | /api/operators/youtube/* | extract, formats, subtitles, save, download |
| Jobs | /api/jobs/* | queue-backed workflows and streaming reconnect |
| MCP | /mcp | remote MCP endpoint over HTTP |
Authentication
GET /api/healthis public- every other
/api/*route requiresx-api-key - dashboard proxy requests can use
DASHBOARD_INTERNAL_API_KEY - MCP only accepts normal dashboard-created API keys
SSE conventions
Common event names:
startprogressresulterrordone
Google AI Search currently ends with end instead of done.
Dependency model
| Dependency | Needed for |
|---|---|
| PostgreSQL | API keys, logs, settings, proxies, persisted state |
| Redis | queue-backed jobs and crawl |
| Worker | processing queued jobs |
HTML_TO_MARKDOWN_SERVICE_URL | markdown extraction when configured |
| Google cookie bootstrap | Google AI Search before the first real search |
YT_ENGINE_URL | YouTube routes |
TAVILY_API_KEY | Tavily routes |
EXA_API_KEY | Exa routes |
Only /api/operators/website/crawl requires Redis in the website operator family. The other website operator endpoints do not depend on Redis.
Google AI Search now also requires the shared browser profile to be bootstrapped once through the cookie flow before normal searches are allowed.