Introduction
What HeadlessX is, what it currently ships, and how the stack is organized
by saifyxpro
HeadlessX is a full-stack scraping workspace built around Camoufox on the backend and a Next.js dashboard on the frontend. It gives you two ways to work with the same system:
- A browser-based dashboard for playground usage, settings, profiles, proxies, logs, and API key management
- An authenticated HTTP API for website scraping, Google SERP extraction, and operational endpoints
This documentation is written against the current HeadlessX V2 codebase and updated for the 2.0.3 release line.
What HeadlessX currently does
| Area | Current capability |
|---|---|
| Website scraping | Raw HTML, JS-rendered HTML, markdown extraction, screenshots, and SSE progress streaming |
| Google SERP | Standard JSON search results and streaming progress/results |
| Browser sessions | Persistent profiles, reusable sessions, and profile launch/stop control |
| Network control | Saved proxies, active proxy testing, and proxy assignment to profiles |
| Operations | API keys, request logs, dashboard metrics, and runtime configuration |
Product shape
Backend API
The API is an Express service in apps/api. It exposes authenticated routes under /api/*, including:
/api/website/*/api/google-serp/*/api/config/api/keys/api/logs/api/profiles/api/proxies/api/jobs/api/dashboard/stats
Dashboard
The dashboard lives in apps/web and runs on a separate Next.js app. As of 2.0.3, the dashboard does not ship its internal API secret to the browser anymore. It now proxies dashboard requests through Next.js server routes.
Storage and state
HeadlessX uses PostgreSQL for application state and stores runtime browser/profile data on disk. Important persistent data includes:
- API keys
- runtime settings
- request logs
- browser profiles
- saved proxies
What changed recently
The 2.0.3 line includes the major behavior and docs changes that affect how you should use the product today:
- Dashboard-to-API traffic now requires
DASHBOARD_INTERNAL_API_KEY - Stored proxy/profile passwords are encrypted at rest
- Public dashboard fallback auth was removed
- Website PDF scraping/export docs are obsolete because the PDF flow was removed
- REST and streaming HTML flows are now closer in behavior on JS-heavy sites
- Job cancellation now stops active streaming work instead of only updating UI state
See the Changelog for the linked issues and commit trail.
Important constraints
HeadlessX is an anti-detection scraping stack, not a guarantee of access. Current behavior is more conservative than older marketing copy suggested:
- Some targets will still challenge or block automation
- Cloudflare challenge pages are now surfaced explicitly instead of being returned as successful content
- The dashboard is an operator UI, not a full multi-user auth system
- The screenshot endpoint returns raw JPEG bytes, not JSON metadata
- The old PDF endpoint is no longer part of the public API surface