Docs
→
Getting Started
→
Changelog

Changelog

Release history and updates for HeadlessX

All notable changes to HeadlessX will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[2.0.0] - 2026-01-28 🦊 CAMOUFOX RELEASE

🎉 Major Release: Complete Browser Engine Rewrite

HeadlessX V2.0 represents a complete rewrite of the browser engine, achieving 0% detection rate across all major anti-bot systems by replacing Chromium-based stealth with Camoufox (Firefox with C++ level fingerprint spoofing).

✨ Added

Browser Engine

  • Camoufox Browser Engine - Firefox with C++ level fingerprint spoofing
  • 0% Detection Rate - Passes CreepJS, Sannysoft, BrowserScan, and all major bot detection
  • WebRTC Protection - Built-in IP leak prevention
  • Binary-Level Stealth - Canvas, WebGL, and AudioContext spoofing at C++ level
  • Persistent Context Pooling - 3x faster browser launches

New Scrapers

  • Google SERP Scraper - Extract search results with zero detection
    • Organic results with position tracking
    • Featured snippets extraction
    • People Also Ask questions
    • Related searches
  • Website Scraper - Full-featured web scraping
    • Raw HTML extraction (fast)
    • JavaScript-rendered content
    • Markdown conversion for LLMs
    • Full-page screenshots (PNG/JPEG)
    • Real-time SSE streaming

API Endpoints

  • /api/website/html - Fast HTML extraction without JavaScript
  • /api/website/html-js - JavaScript rendering with wait options
  • /api/website/content - Clean markdown conversion
  • /api/website/screenshot - Screenshot capture with quality control
  • /api/website/stream - Real-time SSE stream with progress
  • /api/google-serp/search - Google SERP extraction
  • /api/google-serp/stream - Real-time SERP streaming

Documentation

  • Complete API Reference - All endpoints documented with curl examples
  • N8N Integration Guide - Workflow automation with examples
  • Zapier Integration Guide - No-code automation patterns
  • Make.com Integration Guide - Visual automation scenarios
  • Configuration Guide - Dashboard and environment settings
  • Installation Guide - Platform-specific setup (Windows, Linux, macOS)

Dashboard Features

  • Modern UI - Next.js 16 with React 19 and Turbopack
  • Live Configuration - Change settings without restart via Dashboard
  • Request Logs - Full history with screenshots and timing
  • Playground - Real-time testing interface for both scrapers
  • Profile Management - Persistent browser sessions and cookies
  • Proxy Management - Automatic rotation and validation

🚀 Improved

  • Performance - 3x faster browser initialization with context pooling
  • Stealth - From 67% detection (V1.3) to 0% detection (V2.0)
  • Reliability - Better error handling and automatic recovery
  • Documentation - Comprehensive guides with visual examples
  • Developer Experience - Simplified installation with one-command setup

🗑️ Removed

  • playwright-extra - No longer needed with Camoufox native stealth
  • puppeteer-extra-plugin-stealth - Replaced by C++ level patches
  • PDF endpoint (/api/website/pdf) - Removed for optimization
  • Generic component previews from docs - Focused on documentation clarity
  • All JavaScript-based fingerprint protection - Replaced by binary-level

🔧 Changed

  • Browser Engine - Migrated from Chromium to Firefox (Camoufox)
  • Config Management - Moved most settings from .env to Dashboard UI
  • Frontend - Upgraded to Next.js 16 with Turbopack and React 19
  • TypeScript - Upgraded to 5.9+ with stricter type checking
  • Node.js - Now requires Node.js 22+ for optimal performance

🐛 Fixed

  • Hydration mismatch in mobile navigation components
  • Invalid SVG attributes (fill-rule → fillRule) in icon components
  • Component preview build errors after library removal
  • Missing module imports after Badtz-UI refactoring
  • File system access errors in component library stubs
  • Emoji in heading IDs causing invalid CSS selectors

📦 Dependencies

Added:

  • camoufox - Stealth browser engine with Firefox
  • playwright 1.58+ - Browser automation framework

Updated:

  • next → 16.0.7 (with Turbopack)
  • react → 19.x
  • typescript → 5.9+
  • node → 22+
  • pnpm → 9+

Removed:

  • playwright-extra and all stealth plugins
  • puppeteer-extra-plugin-stealth
  • Legacy fingerprinting libraries

🚨 Breaking Changes

  1. Browser Engine: Chromium → Firefox (Camoufox)
  2. API Response Format: All responses wrapped in { success, data, error } structure
  3. PDF Endpoint Removed: /api/website/pdf no longer available
  4. Configuration: Most settings moved from .env to Dashboard UI
  5. Node.js Version: Requires Node.js 22+ (was 18+)

🔄 Migration Guide from V1.3

1. Update Dependencies

cd HeadlessX
pnpm install

2. Download Camoufox

pnpm camoufox:fetch

3. Update Environment Variables

# Remove old stealth configs
# Add new required vars
NEXT_PUBLIC_API_URL="http://localhost:3001"

4. Update API Integrations

// Old format
const html = response.html

// New format (V2.0)
const html = response.data.html

5. Remove PDF Endpoint Usage

// ❌ No longer available
POST /api/website/pdf

// ✅ Use screenshot instead
POST /api/website/screenshot

No Database Migration Required:

  • Database schema unchanged
  • Existing profiles remain compatible
  • API keys remain valid

[1.3.0] - 2025-09-23 🛡️ ADVANCED ANTI-DETECTION & SECURITY RELEASE

🛡️ Major Security & Privacy Features

  • Advanced Fingerprinting Protection: Comprehensive canvas, WebGL, and audio fingerprinting mitigation
  • Behavioral Simulation Engine: Human-like mouse movement with Bezier curves and natural timing
  • Hardware Emulation: Realistic device profiling with consistent hardware characteristics
  • WAF Bypass Capabilities: Cloudflare and DataDome evasion with TLS fingerprint masking
  • Enhanced Authentication: Multi-token support with admin and profile management tokens
  • Comprehensive Security Audit: Full codebase security review with vulnerability fixes

🚀 Anti-Detection Technologies

Fingerprinting Protection

  • Canvas Noise Injection: Dynamic noise with consistent seeding for reproducible fingerprints
  • WebGL Spoofing: GPU vendor/renderer masking with hardware-specific profiles
  • Audio Fingerprint Control: Hardware audio database with realistic device simulation
  • WebRTC Leak Protection: ICE candidate filtering and media device enumeration control
  • Hardware Noise: CPU timing, memory allocation, and performance API manipulation
  • Timezone Intelligence: Automatic timezone alignment with IP geolocation

Behavioral Simulation

  • Natural Mouse Movement: Bezier curve path generation with acceleration modeling
  • Keyboard Dynamics: Dwell time randomization and typing rhythm simulation
  • Scroll Patterns: Natural scroll behavior with reader/scanner/browser profiles
  • Attention Modeling: User attention simulation with realistic interaction patterns
  • Micro-movements: Subtle mouse adjustments and human-like timing variations

WAF & Bot Detection Bypass

  • Cloudflare Bypass: Challenge solver with TLS fingerprint masking
  • DataDome Evasion: Resource blocking and behavioral pattern bypasses
  • Generic WAF Bypass: Signature detection and response analysis
  • Detection Monitoring: Real-time tracking of bot detection encounters

🏗️ Enhanced Architecture

  • Modular Anti-Detection Services: Organized fingerprinting, behavioral, and evasion modules
  • Profile Management System: Device profile creation, validation, and rotation
  • Testing Framework: Comprehensive anti-detection testing against major services
  • Development Tools: Interactive fingerprint testing and profile benchmarking
  • Performance Monitoring: Real-time success rate analytics and optimization

📊 New API Endpoints

  • POST /api/render/stealth - Maximum stealth rendering with all anti-detection features
  • GET /api/test-fingerprint - Interactive fingerprinting effectiveness testing
  • POST /api/profiles - Device profile management and validation
  • GET /api/analytics/detection-rate - Real-time detection rate monitoring
  • POST /api/test/cloudflare - Cloudflare bypass testing and validation

🔧 Configuration Enhancements

  • Expanded Environment Variables: 50+ new configuration options for anti-detection
  • Profile Configuration: Custom device profiles with hardware specifications
  • Stealth Mode Settings: Basic, advanced, and maximum stealth levels
  • Behavioral Tuning: Configurable human behavior simulation parameters
  • Monitoring Controls: Audit trails, performance tracking, and detection analytics

🔒 Security Improvements

  • Authentication Timing Attack Fix: Secure token comparison using crypto.timingSafeEqual
  • Log Sanitization: Token exposure prevention in application logs
  • Input Validation: SSRF protection and comprehensive URL validation
  • Rate Limiting: Token-based and IP-based request throttling
  • Security Headers: CSP, HSTS, and anti-clickjacking protection

🚨 Breaking Changes

  • New Required Environment Variables: FINGERPRINT_PROFILE, STEALTH_MODE
  • API Response Changes: Enhanced error responses with security context
  • Browser Configuration: New profile-based browser initialization

🐛 Bug Fixes

  • Fixed browser resource leaks in concurrent operations
  • Resolved timing inconsistencies in behavioral simulation
  • Corrected WebGL context isolation issues
  • Fixed audio fingerprint noise distribution
  • Resolved profile validation edge cases

📈 Performance Improvements

  • 30% faster rendering with optimized browser pooling
  • 50% reduction in memory usage through enhanced cleanup
  • 90% improvement in detection evasion success rates
  • Real-time monitoring with minimal performance impact

[1.2.0] - 2025-09-15 🏗️ MAJOR MODULAR ARCHITECTURE REFACTOR

🚀 Revolutionary Changes

  • Complete Modular Refactor: Transformed 3079-line monolithic server.js into 20+ focused modules
  • Separation of Concerns: Clean architecture with distinct layers for configuration, services, controllers, and middleware
  • Enhanced Maintainability: Each module has a single responsibility for better code organization
  • Production-Ready: Enterprise-grade error handling, logging, and monitoring capabilities
  • Developer Experience: Improved development workflow with clear module boundaries

🏗️ New Modular Architecture

src/
├── config/         # Configuration management
├── utils/          # Utilities (errors, logging, helpers)
├── services/       # Business logic (browser, stealth, rendering)
├── middleware/     # Express middleware (auth, errors)
├── controllers/    # Request handlers by feature
├── routes/         # Route definitions and mappings
├── app.js          # Main application setup
└── server.js       # Entry point for PM2

✨ Major Features Added

  • Enhanced Error Handling: Structured error responses with correlation IDs for debugging
  • Advanced Rate Limiting: Intelligent rate limiting with memory-based storage and cleanup
  • Improved Logging: Structured logging with request correlation and detailed context
  • Better Browser Management: Optimized browser lifecycle with resource monitoring
  • Security Enhancements: Improved authentication middleware and request validation
  • Performance Optimization: Better resource utilization and memory management

🔧 Configuration Updates

  • Environment Variables: TOKEN → AUTH_TOKEN (breaking change)
  • PM2 Configuration: Moved from config/ecosystem.config.js to root ecosystem.config.js
  • Enhanced .env: More configuration options with validation and defaults
  • Docker Optimization: Updated Docker configuration for modular structure

💥 Breaking Changes

  • Environment Variable: TOKEN renamed to AUTH_TOKEN
  • File Structure: PM2 configuration moved from config/ to root directory
  • Import Paths: Internal imports updated for modular structure
  • Script Updates: Setup scripts updated to work with new architecture

🔄 Migration Guide

# Update environment variables
sed -i 's/TOKEN=/AUTH_TOKEN=/g' .env

# Update PM2 configuration path
mv config/ecosystem.config.js ./ecosystem.config.js

# Restart services
npm run pm2:restart

[1.1.0] - 2024-12-19 🌐 UNIFIED ARCHITECTURE RELEASE

🚀 Major Features Added

  • Unified Architecture: Single Node.js server now serves both website and API
  • Integrated Website: Complete Next.js website served at root path (/)
  • Enhanced API: All API endpoints now available under /api/* prefix
  • Environment Variables: Complete .env file support for all configurations
  • Domain Integration: Automatic subdomain and domain configuration from environment

🌐 Website Integration

  • Next.js Website: Modern React-based website with Tailwind CSS
  • API Documentation: Interactive documentation and examples
  • Live Testing: Built-in API testing interface
  • Responsive Design: Mobile-first design with dark/light theme support
  • TypeScript Support: Full TypeScript integration for better development

⚙️ Infrastructure Improvements

  • Simplified Nginx: Single proxy configuration for all routes
  • Unified Server: Website and API served from same Node.js process
  • Better Routing: Intelligent routing between static files and API endpoints
  • Performance: Improved caching and static file serving
  • Security: Enhanced security headers and token validation

🐳 Docker & Deployment

  • Docker Support: Multi-stage build with optimized containers
  • Docker Compose: Complete stack deployment with one command
  • PM2 Integration: Production process management
  • SSL Support: Ready for Let's Encrypt certificates
  • Health Checks: Automatic service monitoring

💔 Breaking Changes

  • API endpoints moved from root to /api/* prefix
  • Configuration now requires .env file setup
  • Nginx configuration changed to proxy-only
  • Docker deployment process updated

🔄 Migration Guide

# Update API URLs
# Old: https://headlessx.yourdomain.com/render
# New: https://headlessx.yourdomain.com/api/render

# Environment setup required
cp .env.example .env
# Edit .env with your configuration

# New build process
npm run build:full
npm run deploy

[1.0.0] - 2024-12-01 🎉 INITIAL RELEASE

Core API

  • Complete web scraping API with Playwright
  • Screenshot generation with high quality
  • PDF generation from webpages
  • HTML extraction (clean and raw)
  • Text content extraction
  • Batch processing for multiple URLs

🌐 API Endpoints

  • GET /health - Health check endpoint
  • GET /status - Server status with authentication
  • POST /render - Full page rendering with options
  • GET /html - HTML content extraction
  • GET /content - Text content extraction
  • GET /screenshot - Screenshot generation
  • GET /pdf - PDF generation
  • POST /batch - Batch URL processing

🔧 Features

  • Playwright Integration: Chrome, Firefox, Safari browser support
  • Human Behavior: Realistic scrolling, mouse movements, typing
  • Responsive Design: Mobile and desktop viewport simulation
  • Custom Headers: Support for authentication and custom headers
  • Proxy Support: Route requests through proxy servers
  • Timeout Handling: Configurable request timeouts
  • Error Handling: Comprehensive error responses

🚀 Deployment

  • PM2 Support: Production process management
  • Nginx Configuration: Reverse proxy setup
  • Docker Support: Container deployment
  • Environment Configuration: Flexible environment setup

Development Roadmap

🔮 Planned Features (V2.1 - Q1 2026)

  • Docker Support - One-command deployment with Docker Compose
  • Amazon Scraper - Product listings and reviews extraction
  • LinkedIn Scraper - Job listings and profile data
  • Twitter Scraper - Tweets, trends, and profile information
  • Instagram Scraper - Posts, stories, and profile data

🎯 Future Enhancements (V2.2 - Q2 2026)

  • GraphQL API - Alternative GraphQL interface alongside REST
  • Python SDK - Official Python client library
  • Node.js SDK - Official Node.js client library
  • Bulk Scraping - Optimized concurrent URL processing
  • Advanced Scheduling - Cron-based scheduled jobs

🚀 Major Features (V3.0 - Q3 2026)

  • AI-Powered Extraction - LLM-based intelligent data parsing
  • Visual Testing - Screenshot comparison and diff detection
  • CAPTCHA Auto-Solving - Built-in AI-powered CAPTCHA solver
  • Cloud Deployment - Managed hosting with auto-scaling

Contributing

We welcome contributions! Please see our Contributing Guide for details on how to:

  • 🐛 Report bugs
  • 💡 Suggest features
  • 🔀 Submit pull requests
  • 📖 Improve documentation
  • 🧪 Add tests

Security

If you discover a security vulnerability, please read our Security Policy or email security issues directly. All security vulnerabilities will be promptly addressed.

Support

License

This project is licensed under the MIT License.

Acknowledgments

  • Camoufox Team - For the revolutionary stealth browser engine
  • Playwright Team - For the excellent browser automation framework
  • Next.js Team - For the amazing React framework
  • Community Contributors - For suggestions, bug reports, and improvements

HeadlessX v2.0.0 - The world's most advanced anti-detection scraping platform 🦊