Docs
→
Getting Started
→
Zapier Integration

Zapier Integration

Complete guide to integrating HeadlessX with Zapier automation

Automate web scraping with Zapier and HeadlessX. Connect to 5,000+ apps without code.

Quick Setup

Step 1: Add Webhooks by Zapier

  1. Create a new Zap
  2. Choose trigger app
  3. Add Webhooks by Zapier action
  4. Select POST method

Step 2: Configure Webhook

URL: http://your-headlessx-server:3001/api/website/html

Headers:

Content-Type: application/json
X-API-Key: your-api-key-here

Data (JSON):

{
  "url": "{{url}}",
  "stealth": true
}

Step 3: Test & Enable

  1. Click Test & Continue
  2. Review the response
  3. Enable your Zap

Complete Zap Examples

Example 1: New RSS Item → Scrape → Save to Google Sheets

Trigger: RSS by Zapier

  • Feed URL: https://example.com/feed

Action 1: Webhooks by Zapier (POST)

  • URL: http://localhost:3001/api/website/content
  • Headers: 
    Content-Type: application/json
X-API-Key: your-api-key
  • Data: 
    {
  "url": "{{link}}",
  "stealth": true
}

Action 2: Google Sheets - Create Spreadsheet Row

  • Spreadsheet: My Scraped Articles
  • Worksheet: Sheet1
  • Title: {{title}}
  • Content: {{data__markdown}}
  • URL: {{link}}
  • Date: {{pubDate}}

Example 2: Scheduled Price Monitoring

Trigger: Schedule by Zapier

  • Frequency: Every 6 hours

Action 1: Code by Zapier (JavaScript)

  • Input Data: productUrls = ["url1", "url2", "url3"]
  • Code: 
    const urls = inputData.productUrls;
return urls.map(url => ({ url }));

Action 2: Looping by Zapier

  • Loop through URLs

Action 3: Webhooks by Zapier (POST)

  • URL: http://localhost:3001/api/website/html
  • Data: 
    {
  "url": "{{url}}",
  "stealth": true
}

Action 4: Code by Zapier

  • Extract price from HTML
  • Compare with previous

Action 5: Gmail - Send Email (if price dropped)

Example 3: Lead Generation from LinkedIn

Trigger: LinkedIn Lead Gen Forms

Action 1: Webhooks by Zapier

  • Scrape company website from form

Action 2: Code by Zapier

  • Extract contact info using regex

Action 3: HubSpot - Create Contact

  • Add to CRM

Custom Zapier App

For easier integration, use our custom Zapier app (coming soon):

Installation

  1. Visit Zapier App Directory
  2. Search "HeadlessX"
  3. Click Add to Zapier
  4. Authenticate with API key

Authentication

  • API URL: http://your-server:3001
  • API Key: Your HeadlessX API key

Triggers

  • New Scrape Completed - When scraping finishes
  • New Screenshot Taken - When screenshot captured
  • CAPTCHA Detected - When CAPTCHA found

Actions

  1. Scrape HTML

    • Input: URL, Stealth (yes/no)
    • Output: HTML, Title, Status Code

  2. Scrape with JavaScript

    • Input: URL, Wait For, Timeout
    • Output: Rendered HTML

  3. Get Markdown Content

    • Input: URL
    • Output: Markdown, Plain Text

  4. Take Screenshot

    • Input: URL, Full Page (yes/no)
    • Output: Image URL (Base64)

  5. Google Search

    • Input: Query, Location, Results Count
    • Output: Organic Results, Featured Snippet

Searches

  • Find Scrape by URL - Look up cached scrape
  • Find Screenshot - Get existing screenshot

Advanced Workflows

Workflow 1: Content Republishing

RSS Feed → Scrape Article → Extract Images → 
Upload to S3 → Reformat with AI → Publish to WordPress

Tools Used:

  • RSS by Zapier
  • HeadlessX (Webhooks)
  • Amazon S3
  • OpenAI
  • WordPress

Workflow 2: Competitor Analysis

Schedule → Scrape Competitors → Extract Pricing → 
Compare with Yours → Update Spreadsheet → Alert if Cheaper

Tools Used:

  • Schedule by Zapier
  • HeadlessX (Webhooks)
  • Code by Zapier
  • Google Sheets
  • Slack

Workflow 3: Job Board Aggregator

Trigger URLs → Scrape Job Listings → Parse with AI → 
Filter by Criteria → Post to Your Site → Tweet

Tools Used:

  • Webhook (trigger)
  • HeadlessX
  • OpenAI (extract structured data)
  • Filter
  • WordPress
  • Twitter

Multi-Step Zaps

Step 1: Trigger

  • New email in Gmail
  • Contains "scrape"

Step 2: Extract URL

Code by Zapier:

const body = inputData.body;
const urlMatch = body.match(/https?:\/\/[^\s]+/);
return { url: urlMatch ? urlMatch[0] : null };

Step 3: Scrape

Webhooks by Zapier:

{
  "url": "{{url}}",
  "stealth": true
}

Step 4: Process with AI

OpenAI:

  • Extract key information from HTML

Step 5: Save

Airtable - Create Record

Step 6: Reply

Gmail - Send Email

  • Attach results

Error Handling

Filter Bad Responses

Add Filter by Zapier after webhook:

Condition:

  • success equals true

Retry on Failure

Use Error Handler path:

  1. If webhook fails
  2. Delay 30 seconds
  3. Retry once
  4. If still fails → notify admin

Validate URLs

Code by Zapier before scraping:

function isValidUrl(string) {
  try {
    new URL(string);
    return true;
  } catch (_) {
    return false;  
  }
}

const url = inputData.url;
if (!isValidUrl(url)) {
  throw new Error('Invalid URL');
}

return { url };

Formatter Integration

Extract Data with Formatter

Use Formatter by Zapier to parse HTML:

Action: Text - Extract Pattern

  • Input: {{data__html}}
  • Pattern: <title>(.*?)</title>
  • Output: Page title

Clean Markdown

Action: Text - Replace

  • Input: {{data__markdown}}
  • Find: \n\n+
  • Replace: \n\n

Path Rules

Create different paths based on scrape results:

Path A: Success

  • success equals true
  • Save to Notion

Path B: CAPTCHA

  • error__code contains CAPTCHA
  • Send alert to Slack

Path C: Timeout

  • error__code equals TIMEOUT
  • Retry with longer timeout

Storage Integration

Save Screenshots

  1. HeadlessX - Take Screenshot
  2. Code by Zapier - Convert Base64
  3. Google Drive - Upload File

Code:

const base64 = inputData.screenshot;
const buffer = Buffer.from(base64, 'base64');
return { fileData: buffer };

Cache Results

  1. HeadlessX - Scrape
  2. Storage by Zapier - Set Value
    • Key: URL hash
    • Value: HTML (expires in 1 hour)

Best Practices

  1. Use Filters: Don't run actions on failed scrapes
  2. Add Delays: If scraping multiple URLs
  3. Error Paths: Always handle failures
  4. Format Data: Use Formatter to clean output
  5. Test First: Always test before enabling

Limits & Tips

Zapier Limits

PlanTasks/MonthZapsUpdate Time
Free100515 min
Starter7502015 min
Professional2,000+Unlimited2 min

Tips

  • Combine Steps: Use Code instead of multiple actions
  • Batch Processing: Loop through arrays
  • Caching: Store frequently scraped pages
  • Webhooks: For real-time triggers

Troubleshooting

Zap Not Triggering

  1. Check trigger setup
  2. Verify test data exists
  3. Review Zap history logs

Webhook Returns Error

  1. Verify API key in headers
  2. Check server is accessible
  3. Test URL manually with curl

Data Not Mapping

  1. Test step and view output
  2. Check field names match
  3. Use Code to transform if needed

Example Apps

Connect HeadlessX with popular apps:

AppUse Case
SlackSend scrape alerts
Google SheetsLog all scrapes
AirtableStore structured data
NotionCreate knowledge base
HubSpotEnrich contacts
Monday.comTrack scraping tasks
MailchimpUpdate email lists

Next Steps