# Introduction

Obscura is an open-source headless browser engine written in Rust. It runs JavaScript via V8, speaks the Chrome DevTools Protocol, and works as a drop-in replacement for headless Chrome with Puppeteer and Playwright.

### Versus headless Chrome

| Metric      | Obscura  | Headless Chrome |
| ----------- | -------- | --------------- |
| Memory      | 30 MB    | 200+ MB         |
| Binary size | 70 MB    | 300+ MB         |
| Startup     | Instant  | \~2s            |
| Page load   | 85 ms    | \~500 ms        |
| Anti-detect | Built-in | None            |
| Puppeteer   | Yes      | Yes             |
| Playwright  | Yes      | Yes             |

Rendering and stealth are both first-class capabilities. Release builds support screenshots, scroll-aware layout, activity-driven CDP screencasting, and raster PDF export; stealth builds retain all of those surfaces while adding the wreq/BoringSSL transport and browser-identity protections.

### Quickstart

* [Installation](/quickstart/installation)
* [Your first fetch](/quickstart/your-first-fetch)
* [Extract data](/quickstart/extract-data)
* [Connect Puppeteer or Playwright](/quickstart/connect-puppeteer-or-playwright)

### Guides

* [Build from source](/guides/build-from-source)
* [Configure stealth and proxies](/guides/configure-stealth-and-proxies)
* [Markdown extraction](/guides/markdown-extraction)
* [Use with Puppeteer](/guides/use-with-puppeteer)
* [Use with Playwright](/guides/use-with-playwright)
* [Use the MCP server](/guides/use-the-mcp-server)
* [Use as a Rust library](/guides/use-as-a-rust-library)
* [Persist cookies and storage](/guides/persist-cookies-and-storage)
* [Intercept and modify requests](/guides/intercept-and-modify-requests)
* [Run in production at scale](/guides/run-in-production-at-scale)

### Reference

* [CLI reference](/reference/cli-reference)
* [Environment variables](/reference/environment-variables)

### Contributing

* [Architecture overview](/contributing/architecture-overview)
* [Adding a CDP method or Web API](/contributing/adding-a-cdp-method-or-web-api)
* [Testing and debugging](/contributing/testing-and-debugging)

### Links

* Source: <https://github.com/h4ckf0r0day/obscura>
* Releases: <https://github.com/h4ckf0r0day/obscura/releases>
* Issues: <https://github.com/h4ckf0r0day/obscura/issues>

License: Apache-2.0.


# Installation

### Linux x86\_64

```bash
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz
tar xzf obscura-x86_64-linux.tar.gz
./obscura --version
```

### Linux ARM64

```bash
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-aarch64-linux.tar.gz
tar xzf obscura-aarch64-linux.tar.gz
./obscura --version
```

Linux builds target Ubuntu 22.04 and require glibc 2.35+.

### macOS Apple Silicon

```bash
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-aarch64-macos.tar.gz
tar xzf obscura-aarch64-macos.tar.gz
./obscura --version
```

### macOS Intel

```bash
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-macos.tar.gz
tar xzf obscura-x86_64-macos.tar.gz
./obscura --version
```

### Windows

Download the `.zip` from [Releases](https://github.com/h4ckf0r0day/obscura/releases), extract, run `obscura.exe --version`.

### Arch Linux (AUR)

```bash
yay -S obscura-browser
```

### Docker

```bash
docker run -d --name obscura -p 127.0.0.1:9222:9222 h4ckf0r0day/obscura
```

Image: [h4ckf0r0day/obscura](https://hub.docker.com/r/h4ckf0r0day/obscura). Built on `distroless/cc:nonroot`, with no shell or package manager in the runtime image, running as uid 65532. Note the `-p 127.0.0.1:...` above: it publishes the port to host loopback only. A mounted `--storage-dir` must be writable by uid 65532 — see [Run in production at scale](/guides/run-in-production-at-scale#the-container-does-not-run-as-root).

Official archives and the Docker image include the rendering engine. Source builders must pass `--features render`; see [Build from source](/guides/build-from-source).

### From source

See [Build from source](/guides/build-from-source).

### What's in the archive

* `obscura`: CLI and CDP server.
* `obscura-worker`: helper for the parallel `scrape` command. Keep both in the same directory.

Archive suffixes identify the feature set: no suffix includes rendering, `-stealth` includes rendering and stealth, `-no-render` includes neither, and `-no-render-stealth` includes stealth without rendering.

### Smoke test

```bash
./obscura fetch https://example.com --eval "document.title"
./obscura fetch https://example.com --screenshot smoke.png
```

Expected output: `"Example Domain"`, followed by a nonempty PNG at `smoke.png`.

### Troubleshooting

`cannot execute binary file`: wrong arch. Check `uname -m`.

`GLIBC_2.35 not found`: distro is older than Ubuntu 22.04. Use Docker or build from source.

macOS Gatekeeper warning: `xattr -d com.apple.quarantine ./obscura`.


# Your first fetch

`obscura fetch` loads a URL, runs its JavaScript, and prints the result.

### Load a page

```bash
obscura fetch https://example.com
```

Prints the rendered HTML.

### Run JavaScript with `--eval`

```bash
obscura fetch https://example.com --eval "document.title"
```

```
"Example Domain"
```

Returns JSON:

```bash
obscura fetch https://news.ycombinator.com \
  --eval "Array.from(document.querySelectorAll('.titleline a')).slice(0, 5).map(a => a.textContent)"
```

### Multi-statement eval

`--eval` evaluates one expression. For multiple statements, wrap in an IIFE:

```bash
obscura fetch https://example.com --eval "(function(){
  const links = document.querySelectorAll('a');
  return Array.from(links).map(a => a.href);
})()"
```

A bare block starting with `const` or `let` returns `null` because V8 gives top-level declarations an empty completion value.

### Wait for the right moment

CLI default is `load`. For faster returns on slow sites:

```bash
obscura fetch https://my-spa.example --wait-until domcontentloaded --eval "document.title"
```

| Level              | Returns when                            |
| ------------------ | --------------------------------------- |
| `domcontentloaded` | HTML parsed, scripts ran                |
| `load`             | All subresources finished (default)     |
| `networkidle2`     | ≤2 network connections active for 500ms |
| `networkidle0`     | 0 network connections active for 500ms  |

(When driving obscura via Puppeteer or Playwright the default is `domcontentloaded` to match client expectations.)

### Common flags

```
--user-agent "..."        Override the User-Agent
--timeout 30                Navigation timeout in seconds (default 30)
--wait 5                    Extra wait after the page settles, in seconds (default 5)
--selector ".main"          CSS selector to narrow output to
--proxy http://host:port    Route through a proxy
--stealth                   Stealth client (TLS fingerprint, tracker blocking)
-o, --output file.html      Write output to a file
-q, --quiet                 Suppress info logging
```

Full list: [CLI reference](/reference/cli-reference).


# Extract data

`--dump` formats the page output without writing JavaScript.

```bash
obscura fetch https://example.com --dump html
obscura fetch https://example.com --dump text
obscura fetch https://example.com --dump markdown
obscura fetch https://example.com --dump links
obscura fetch https://example.com --dump assets
obscura fetch https://example.com --dump original
obscura fetch https://example.com --dump cookies
```

### `html`

Rendered HTML after JavaScript runs. Default.

```bash
obscura fetch https://news.ycombinator.com --dump html > hn.html
```

### `text`

Plain text. No markup.

```bash
obscura fetch https://en.wikipedia.org/wiki/Rust_(programming_language) --dump text
```

### `markdown`

Markdown conversion: headings, lists, links, code blocks, images.

```bash
obscura fetch https://docs.example.com/page --dump markdown > page.md
```

### `links`

Every `<a href>` on the page, one per line.

```bash
obscura fetch https://example.com --dump links
```

### `assets`

Every external resource (stylesheets, scripts, images, fonts, iframes), plus the URLs the page requested through `fetch()`/XHR, one JSON object per line.

```bash
obscura fetch https://example.com --dump assets
```

### `original`

The raw HTML the server sent, before JavaScript ran.

```bash
obscura fetch https://my-spa.example --dump original > before.html
obscura fetch https://my-spa.example --dump html     > after.html
diff before.html after.html
```

### `cookies`

Every cookie in the jar as a JSON array, including HttpOnly cookies that `document.cookie` cannot see. Useful for capturing session tokens set by anti-bot challenges.

```bash
obscura fetch https://example.com --dump cookies
```

### With `--wait-until`

`--dump` runs after the wait condition:

```bash
obscura fetch https://my-spa.example --wait-until load --dump markdown
```

### Pipe and redirect

```bash
obscura fetch https://example.com --dump markdown > example.md
obscura fetch https://example.com --dump text --quiet | wc -w
```


# Connect Puppeteer or Playwright

Obscura speaks the Chrome DevTools Protocol over WebSocket. Puppeteer and Playwright can connect to its CDP endpoint for the supported workflows below.

### Start the server

```bash
obscura serve --port 9222
```

```
obscura listening on ws://127.0.0.1:9222
```

### Puppeteer

```bash
npm install puppeteer-core
```

```js
const puppeteer = require('puppeteer-core');

const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222',
});

const page = await browser.newPage();
await page.goto('https://example.com');
console.log(await page.title()); // "Example Domain"

await browser.disconnect();
```

Use `puppeteer-core`, not `puppeteer`. The `puppeteer` package bundles a Chrome download.

### Playwright

```bash
npm install playwright
```

```js
const { chromium } = require('playwright');

const browser = await chromium.connectOverCDP('ws://127.0.0.1:9222');
const context = browser.contexts()[0] || await browser.newContext();
const page = await context.newPage();

await page.goto('https://example.com');
console.log(await page.title());

await browser.close();
```

Use `connectOverCDP`, not `connect`. Playwright's `connect` speaks Playwright's own protocol, which obscura does not implement.

### `waitUntil`

Default is `domcontentloaded`. For full subresource load:

```js
await page.goto('https://example.com', { waitUntil: 'load' });
```

| Value              | Returns when                            |
| ------------------ | --------------------------------------- |
| `domcontentloaded` | HTML parsed, scripts ran (default)      |
| `load`             | All subresources finished               |
| `networkidle2`     | ≤2 network connections active for 500ms |
| `networkidle0`     | 0 network connections active for 500ms  |

### Supported

* `page.goto`, `page.reload`, `page.goBack`, `page.goForward`
* `page.evaluate`, `page.evaluateHandle`
* `page.click`, `page.type`, `page.fill`, `page.focus`
* `page.waitForSelector`, `page.waitForFunction`, `page.waitForNavigation`
* `page.cookies`, `page.setCookie`, `context.cookies`
* `page.setRequestInterception`, block / modify
* `page.exposeFunction`
* `page.content`, `page.title`, `page.url`
* `page.screenshot` for viewport, clipped, and full-page capture
* `page.pdf` for raster-backed print output
* raw CDP `Page.startScreencast` with frame acknowledgements (`page.createCDPSession()` in Puppeteer; `context.newCDPSession(page)` in Playwright)

DOM-agent frameworks such as browser-use also connect: obscura implements `DOMSnapshot.captureSnapshot` and `Target.targetInfoChanged` for perception, and `DOM.focus` so a focused field receives `Input.dispatchKeyEvent` keystrokes.

### Capture example

```js
await page.setViewport({ width: 1440, height: 1000 });
await page.screenshot({ path: 'viewport.png' });
await page.screenshot({ path: 'full-page.png', fullPage: true });
await page.pdf({ path: 'page.pdf', format: 'A4', printBackground: true });
```

Rendering is included in official binaries and requires `--features render` for source builds. The client-specific guides cover scrolling, raw CDP screencasting, and current output limits.

### Current limits

* Pages share one V8 isolate. CPU-bound JavaScript on one page can delay others.
* PDF output is raster-backed; text is not selectable and tagged PDF, headers/footers, outlines, and full CSS paged media are not implemented.
* Service workers, native media playback, some Web APIs, and long-tail CSS or compositor effects are still incomplete relative to Chromium.


# Build from source

### Requirements

* Rust 1.75+ ([rustup.rs](https://rustup.rs))
* C compiler (gcc or clang)
* \~5 GB free disk space (V8 compiles from source on first build)

First build takes about 5 minutes. Incremental builds are seconds.

### Build

```bash
git clone https://github.com/h4ckf0r0day/obscura.git
cd obscura
cargo build --release -p obscura-cli --bins --features render
```

Binary is at `./target/release/obscura`.

This produces the release binary with geometry, screenshots, screencasting, and PDF export.

### Rendering and stealth

```bash
cargo build --release -p obscura-cli --bins --features render,stealth
```

This is the complete rendering build with the stealth wreq/BoringSSL transport, TLS fingerprint randomization, browser-identity protections, and tracker blocklist. See [Configure stealth and proxies](/guides/configure-stealth-and-proxies).

### Without rendering

```bash
cargo build --release -p obscura-cli --bins --no-default-features
cargo build --release -p obscura-cli --bins --no-default-features --features stealth
```

The second command keeps stealth while excluding layout, screenshots, screencasting, and PDF export.

The stealth feature builds BoringSSL and generates Rust bindings. In addition to the default requirements, install CMake, Clang, and the libclang/LLVM development libraries. On Ubuntu/Debian:

```bash
sudo apt-get install build-essential cmake clang libclang-dev llvm-dev
```

On macOS, install the Xcode Command Line Tools and CMake. On Windows, install the Visual Studio C++ Build Tools, CMake, and LLVM/Clang. Ensure the directory containing `libclang` is available through `LIBCLANG_PATH` if bindgen cannot locate it automatically.

On macOS 26 with the standalone Command Line Tools, Apple Clang may not find libc++ while compiling BoringSSL. Use the active SDK for that build:

```bash
SDK_PATH="$(xcrun --show-sdk-path)"
SDKROOT="$SDK_PATH" CXXFLAGS="-isystem $SDK_PATH/usr/include/c++/v1" \
  cargo build --release -p obscura-cli --bins --features render,stealth
```

### OpenSSL on older systems

If the build fails on the vendored OpenSSL with an AVX-512 assembler error (common on older VPS hosts):

```bash
OPENSSL_NO_VENDOR=1 cargo build --release -p obscura-cli --bins --features render
```

Uses the system OpenSSL instead.

### Run from the build

```bash
./target/release/obscura --version
./target/release/obscura fetch https://example.com --eval "document.title"
```

Install system-wide:

```bash
cargo install --path crates/obscura-cli --features render
```

### Tests

```bash
cargo nextest run --release --features render --no-fail-fast
```

Integration suite:

```bash
python3 tests/test_all.py
```

Use `cargo nextest`, not `cargo test`: runtime tests require process isolation because the engine owns a single V8 isolate per process.


# Configure stealth and proxies

### Stealth mode

```bash
obscura fetch https://example.com --stealth
obscura serve --stealth
obscura scrape url1 url2 --stealth
obscura mcp --stealth
```

`--stealth` is a global flag, so it works before or after the subcommand and applies to `fetch`, `serve`, `scrape`, and `mcp`. In a `scrape` run each worker inherits it.

What `--stealth` changes:

* Uses the wreq HTTP client with browser-matching TLS fingerprints (ClientHello, ALPN, cipher order).
* Loads a tracker blocklist that drops requests to known analytics and fingerprinting endpoints.
* Bundles webpki roots instead of relying on the system store.

Requires a build that includes the stealth feature. Use a `-stealth` archive with rendering or a `-no-render-stealth` archive without it. To build the rendering variant yourself:

```bash
cargo build --release -p obscura-cli --bins --features render,stealth
```

Omit rendering with `cargo build --release -p obscura-cli --bins --no-default-features --features stealth`.

### What stealth handles

* Basic bot detection that checks TLS fingerprint or User-Agent.
* Sites that rely on third-party analytics being reachable.

### What stealth does not handle

* Cloudflare interactive challenges.
* Datadome and Akamai bot manager active challenges.
* CAPTCHAs.
* IP-based rate limiting (use proxies).

### Proxies

HTTP proxy:

```bash
obscura fetch https://example.com --proxy http://proxy.example.com:8080
obscura serve --proxy http://proxy.example.com:8080
```

With auth:

```bash
obscura fetch https://example.com --proxy http://user:pass@proxy.example.com:8080
```

SOCKS5:

```bash
obscura fetch https://example.com --proxy socks5://proxy.example.com:1080
```

### Custom User-Agent

```bash
obscura fetch https://example.com --user-agent "Mozilla/5.0 (...) ..."
obscura serve --user-agent "Mozilla/5.0 (...) ..."
```

Default UA matches a recent Chrome on the build platform.

### Browser profile, timezone, and geolocation

The engine presents one of a built-in pool of realistic browser profiles (a mix of Windows and macOS, recent Chrome versions). Each profile keeps `navigator.platform`, `navigator.userAgentData` (platform and platform version), and the UA string internally consistent, so the surfaces a site fingerprints agree with each other. There is no GPU renderer among them: `canvas.getContext('webgl')` returns `null`, so a page cannot read a renderer string at all.

A single stable profile is used by default. One IP cycling through different identities is itself a signal, so rotation is opt-in:

```bash
OBSCURA_PROFILE=2 obscura serve          # pin a specific profile by index
OBSCURA_ROTATE_PROFILE=1 obscura serve   # random profile per browser context
```

Timezone is driven by the process zone so `Date` (`getTimezoneOffset`, `toString`) and `Intl.DateTimeFormat` report the same region. Default is `Europe/Berlin`; set it to match the exit IP:

```bash
OBSCURA_TIMEZONE=America/New_York obscura serve
```

`navigator.geolocation` reports configurable coordinates. Set them as `lat,lon` and keep them consistent with the timezone and proxy region:

```bash
OBSCURA_GEOLOCATION="40.7128,-74.0060" obscura serve
```

Keep these aligned. A rotated or mismatched profile carries no matching TLS or timezone fingerprint, so when you pin a proxy region or TLS fingerprint, leave rotation off and set the timezone and geolocation to the same region. See [Environment variables](/reference/environment-variables) for the full list.

### Combine

```bash
obscura serve \
  --stealth \
  --proxy http://user:pass@proxy.example.com:8080 \
  --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 ..."
```


# Markdown extraction

`--dump markdown` converts the rendered page to markdown.

```bash
obscura fetch https://example.com --dump markdown
```

### What gets converted

* Headings (`<h1>` through `<h6>`)
* Paragraphs, line breaks
* Bold, italic, code spans
* Links (with `href`)
* Images (with `src` and `alt`)
* Ordered and unordered lists
* Block quotes
* Code blocks (`<pre>`, `<code>`)
* Tables

### What gets stripped

* `<script>`, `<style>`, `<noscript>`
* Inline styles
* ARIA attributes
* Tracking pixels and beacons

### Save to file

```bash
obscura fetch https://docs.example.com/page --dump markdown -o page.md
```

### For RAG / LLM context

```bash
obscura fetch https://docs.example.com/page --dump markdown --quiet
```

`--quiet` strips info logging so the output is just markdown.

### Wait for SPA content

For pages that render content client-side:

```bash
obscura fetch https://my-spa.example --wait-until load --dump markdown
```

### Narrow to a region

`--selector` restricts the conversion to a CSS selector:

```bash
obscura fetch https://example.com --selector "main" --dump markdown
obscura fetch https://example.com --selector "article.post" --dump markdown
```

Useful for skipping nav, sidebars, and footers.


# Use with Puppeteer

### Setup

```bash
obscura serve --port 9222
npm install puppeteer-core
```

### Connect

```js
const puppeteer = require('puppeteer-core');

const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222',
});
```

Use `puppeteer-core`, not `puppeteer`. The `puppeteer` package bundles a Chrome download.

### Navigate

```js
const page = await browser.newPage();
await page.goto('https://example.com');
await page.goto('https://example.com', { waitUntil: 'load' });
await page.goto('https://example.com', { waitUntil: 'networkidle0', timeout: 60000 });
```

Default `waitUntil` is `domcontentloaded`. Other values: `load`, `networkidle2`, `networkidle0`.

### Evaluate

```js
const title = await page.evaluate(() => document.title);

const items = await page.evaluate(() => {
  return Array.from(document.querySelectorAll('.item')).map(el => ({
    text: el.textContent,
    href: el.querySelector('a')?.href,
  }));
});
```

### Interact

```js
await page.click('#login-button');
await page.type('#username', 'alice');
await page.fill('#password', 'secret');  // alias of .type for compat

await page.waitForSelector('#dashboard');
await page.waitForFunction(() => window.appReady === true);
```

### Cookies

```js
await page.setCookie({
  name: 'session',
  value: 'abc123',
  domain: 'example.com',
  path: '/',
  httpOnly: true,
  secure: true,
});

const cookies = await page.cookies();
```

For session persistence across runs see [Persist cookies and storage](/guides/persist-cookies-and-storage).

### Intercept requests

```js
await page.setRequestInterception(true);

page.on('request', req => {
  if (req.resourceType() === 'image') {
    req.abort();
  } else {
    req.continue();
  }
});
```

See [Intercept and modify requests](/guides/intercept-and-modify-requests).

### Expose a Node callback

```js
await page.exposeFunction('logFromPage', (msg) => {
  console.log('page:', msg);
});

await page.evaluate(() => {
  window.logFromPage('hello from the browser');
});
```

### Multiple pages

```js
const page1 = await browser.newPage();
const page2 = await browser.newPage();

await Promise.all([
  page1.goto('https://a.example.com'),
  page2.goto('https://b.example.com'),
]);
```

Pages share one V8 isolate. Concurrent JS execution serializes through a lock. CPU-bound JS on one page blocks the others.

### Screenshots, scrolling, and PDF

```js
await page.setViewport({ width: 1440, height: 1000, deviceScaleFactor: 1 });
await page.screenshot({ path: 'viewport.png' });

await page.evaluate(() => window.scrollTo(0, 1200));
await page.screenshot({ path: 'scrolled.png' });

await page.screenshot({ path: 'full-page.png', fullPage: true });
await page.pdf({ path: 'page.pdf', format: 'A4', printBackground: true });
```

A normal screenshot captures the live viewport and scroll position; `fullPage: true` captures document space. PDF output is raster-backed.

### Screencasting

Attach a raw CDP session to the page, acknowledge every frame, and detach it when finished:

```js
const client = await page.createCDPSession();

client.on('Page.screencastFrame', async ({ data, sessionId }) => {
  const jpeg = Buffer.from(data, 'base64');
  // Consume or forward `jpeg` here.
  await client.send('Page.screencastFrameAck', { sessionId });
});

await client.send('Page.startScreencast', {
  format: 'jpeg',
  quality: 80,
  maxWidth: 1280,
  maxHeight: 720,
});

// ...navigate, scroll, and interact...

await client.send('Page.stopScreencast');
await client.detach();
```

Frames are activity-driven page captures, not fixed-rate desktop video.

### Disconnect

```js
await browser.disconnect();  // leaves obscura serve running
```

### Current limits

* Some device emulation, service-worker, native media, long-tail CSS, and compositor behavior remains incomplete relative to Chromium.
* Pages share one V8 isolate; CPU-bound JavaScript serializes across pages.
* PDF text is not selectable/searchable and tagged PDF is not yet available.


# Use with Playwright

### Setup

```bash
obscura serve --port 9222
npm install playwright
```

### Connect

```js
const { chromium } = require('playwright');

const browser = await chromium.connectOverCDP('ws://127.0.0.1:9222');
const context = browser.contexts()[0] || await browser.newContext();
const page = await context.newPage();
```

Use `connectOverCDP`, not `connect`. Playwright's `connect` speaks Playwright's own protocol.

### Navigate

```js
await page.goto('https://example.com');
await page.goto('https://example.com', { waitUntil: 'load' });
await page.goto('https://example.com', { waitUntil: 'networkidle' });
```

Default is `domcontentloaded`. Other values: `load`, `networkidle`.

### Evaluate

```js
const title = await page.evaluate(() => document.title);

const items = await page.$$eval('.item', els => els.map(el => ({
  text: el.textContent,
  href: el.querySelector('a')?.href,
})));
```

### Interact

```js
await page.click('#login-button');
await page.fill('#username', 'alice');
await page.fill('#password', 'secret');

await page.waitForSelector('#dashboard');
await page.waitForFunction(() => window.appReady === true);
```

### Locators

```js
await page.locator('button.submit').click();
await page.getByRole('button', { name: 'Submit' }).click();
await page.getByLabel('Email').fill('alice@example.com');
```

### Cookies

```js
await context.addCookies([{
  name: 'session',
  value: 'abc123',
  domain: 'example.com',
  path: '/',
}]);

const cookies = await context.cookies();
```

### Intercept requests

```js
await page.route('**/*', route => {
  if (route.request().resourceType() === 'image') {
    route.abort();
  } else {
    route.continue();
  }
});
```

### Multiple pages

```js
const page1 = await context.newPage();
const page2 = await context.newPage();

await Promise.all([
  page1.goto('https://a.example.com'),
  page2.goto('https://b.example.com'),
]);
```

Pages share one V8 isolate. CPU-bound JS on one page blocks the others.

### Screenshots, scrolling, and PDF

```js
await page.setViewportSize({ width: 1440, height: 1000 });
await page.screenshot({ path: 'viewport.png' });

await page.evaluate(() => window.scrollTo(0, 1200));
await page.screenshot({ path: 'scrolled.png' });

await page.screenshot({ path: 'full-page.png', fullPage: true });
await page.pdf({ path: 'page.pdf', format: 'A4', printBackground: true });
```

A normal screenshot captures the live viewport and scroll position; `fullPage: true` captures document space. PDF output is raster-backed.

### Screencasting

Playwright does not expose CDP screencasting as a page method. Attach a raw CDP session to the page, acknowledge every frame, and detach it when finished:

```js
const client = await context.newCDPSession(page);

client.on('Page.screencastFrame', async ({ data, sessionId }) => {
  const jpeg = Buffer.from(data, 'base64');
  // Consume or forward `jpeg` here.
  await client.send('Page.screencastFrameAck', { sessionId });
});

await client.send('Page.startScreencast', {
  format: 'jpeg',
  quality: 80,
  maxWidth: 1280,
  maxHeight: 720,
});

// ...navigate, scroll, and interact...

await client.send('Page.stopScreencast');
await client.detach();
```

Frames are activity-driven page captures, not fixed-rate desktop video.

### Disconnect

```js
await browser.close();  // closes the CDP connection, leaves obscura serve running
```

### Current limits

* Playwright `page.video()` and tracing artifacts that require desktop capture are not implemented. Use the raw CDP flow above for page frames.
* `BrowserContext` storage-state save/restore remains limited; use `--storage-dir` on `obscura serve`, as described in [Persist cookies and storage](/guides/persist-cookies-and-storage).
* Service workers, native media, some Web APIs, long-tail CSS, and compositor behavior remain incomplete relative to Chromium.
* PDF text is not selectable/searchable and tagged PDF is not yet available.


# Use the MCP server

`obscura mcp` exposes obscura as a Model Context Protocol server so MCP-capable clients (Claude Desktop, Claude Code, etc.) can drive it.

### Run

Stdio (default, for direct client integration):

```bash
obscura mcp
```

HTTP (for remote or shared use):

```bash
obscura mcp --http --port 3000
```

The HTTP transport binds `127.0.0.1` by default. Bind all interfaces with `--host` for a container or sidecar deployment:

```bash
obscura mcp --http --host 0.0.0.0 --port 3000
```

With stealth and proxy:

```bash
obscura mcp --stealth --proxy http://proxy.example.com:8080
```

### Security

The HTTP transport has no built-in auth, so anyone who can reach the port can drive the browser. Two guards ship for the HTTP transport:

* **Origin allowlist.** Set `OBSCURA_MCP_ALLOWED_ORIGINS` to a comma-separated list of allowed `Origin` values. When set, a browser request from an unlisted origin is refused with `403` before it can drive the server, which blocks a malicious page from POSTing to a loopback MCP port. Native, non-browser clients send no `Origin` and are always allowed. Unset (the default) keeps the permissive behavior.
* **Body cap.** A single request body is capped at 16 MiB, so an unauthenticated caller cannot force a large allocation with an oversized `Content-Length`.

```bash
OBSCURA_MCP_ALLOWED_ORIGINS="https://app.example.com" obscura mcp --http --host 0.0.0.0
```

When you expose the HTTP transport beyond loopback, set the allowlist and put it behind a reverse proxy or network isolation that enforces auth.

### Tools exposed

The server keeps a live browser session, so tools operate on the current page rather than taking a URL each call. Navigate first, then read or act.

Navigation and lifecycle:

* `browser_navigate`, `browser_back`, `browser_forward`, `browser_reload`, `browser_close`

Read the page:

* `browser_snapshot`: current URL, title, readable body text, and interactive element references. Optional `max_chars` limits the returned text.
* `browser_markdown`, `browser_links`, `browser_extract`: page as markdown, link list, or structured content.
* `browser_interactive_elements`, `browser_detect_forms`: actionable elements and form fields.
* `browser_get_attribute`, `browser_count`, `browser_search`: read an attribute, count matches, find text.

Interact:

* `browser_click`, `browser_fill`, `browser_fill_form`, `browser_type`, `browser_press_key`, `browser_select_option`, `browser_scroll`

Wait and run JS:

* `browser_wait_for`, `browser_wait_for_text`, `browser_evaluate`

Diagnostics:

* `browser_network_requests`, `browser_console_messages`

Visual output (render-enabled builds):

* `browser_screenshot`: current viewport as an MCP `image/png` content block.
* `browser_pdf`: current page as an embedded `application/pdf` resource.

`browser_screenshot` accepts optional positive `width` and `height` values in CSS pixels and enforces a bounded capture size. `browser_pdf` accepts `landscape`, `print_background`, `scale`, paper width/height, and top, bottom, left, and right margins. Paper dimensions and margins are measured in inches.

Cookies and storage:

* `browser_get_cookies`, `browser_set_cookie`, `browser_clear_cookies`, `browser_storage_state`, `browser_set_storage_state`

Tabs:

* `browser_tab_new`, `browser_tab_list`, `browser_tab_switch`, `browser_tab_close`

Element references describe the current rendered page state and can become stale after navigation, interaction, scrolling, or a framework rerender. Take a fresh snapshot or interactive-element listing before acting again.

MCP exposes still-image and PDF output. It does not stream video frames; use CDP `Page.startScreencast` for activity-driven screencasting.

### Claude Desktop

Edit `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows):

```json
{
  "mcpServers": {
    "obscura": {
      "command": "/path/to/obscura",
      "args": ["mcp"]
    }
  }
}
```

Restart Claude Desktop. The obscura tools appear in the tool list.

### Claude Code

```bash
claude mcp add obscura /path/to/obscura mcp
```

### With stealth in config

```json
{
  "mcpServers": {
    "obscura": {
      "command": "/path/to/obscura",
      "args": ["mcp", "--stealth"]
    }
  }
}
```


# Watch agent sessions live

Obscura is headless, so an agent driving it through MCP, Puppeteer, or Playwright works invisibly. The CDP screencast surface lets you stream what the browser sees to a local tab while the agent works, which helps when supervising long tasks or debugging what an agent clicked.

This guide builds a small live viewer on top of `obscura serve`. It uses only Node's built-in modules and its native WebSocket client (Node 21+).

## How it works

1. Start the CDP server:

```bash
obscura serve --port 9222
```

2. Run the viewer script below:

```bash
node watch.mjs
```

3. Open <http://localhost:8080> in a browser. Every page the agent navigates to appears there, updating as the page paints.

The script attaches to the CDP endpoint, captures the current page twice a second with `Page.captureScreenshot`, and forwards JPEG frames to your browser over Server-Sent Events.

```js
// watch.mjs
// Usage: node watch.mjs [cdpPort] [httpPort]
import http from "node:http";

const cdpPort = process.argv[2] ?? 9222;
const httpPort = process.argv[3] ?? 8080;

const clients = new Set();
let latest = null;
let sessionId = null;
let ws = null;

const page = `
<!doctype html>
<html>
<head><meta charset="utf-8"><title>Obscura live</title>
<style>body{margin:0;background:#111;display:grid;place-items:center;height:100vh}
img{max-width:100%;max-height:100%}</style></head>
<body><img id="s" alt="live page">
<script>
const img = document.getElementById("s");
let old = null;
const es = new EventSource("/events");
es.onmessage = (e) => {
  const bytes = Uint8Array.from(atob(e.data), (c) => c.charCodeAt(0));
  const url = URL.createObjectURL(new Blob([bytes], { type: "image/jpeg" }));
  img.src = url;
  if (old) URL.revokeObjectURL(old);
  old = url;
};
</script></body>
</html>`;

const server = http.createServer((req, res) => {
  if (req.url === "/events") {
    res.writeHead(200, {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-store",
      Connection: "keep-alive",
    });
    res.socket.setNoDelay(true);
    if (latest) res.write(`data:${latest}\n\n`);
    clients.add(res);
    req.on("close", () => clients.delete(res));
  } else {
    res.writeHead(200, { "Content-Type": "text/html" });
    res.end(page);
  }
});

server.listen(httpPort, "127.0.0.1", () => {
  console.log(`live view: http://localhost:${httpPort}`);
});

function broadcast(base64) {
  latest = base64;
  for (const res of clients) {
    if (res.writable) res.write(`data:${base64}\n\n`);
  }
}

async function connect() {
  ws = new WebSocket(`ws://127.0.0.1:${cdpPort}/devtools/browser`);
  let id = 0;
  const pending = new Map();
  const call = (method, params = {}, sess) =>
    new Promise((resolve, reject) => {
      const mid = ++id;
      pending.set(mid, { resolve, reject });
      const msg = { id: mid, method, params };
      if (sess) msg.sessionId = sess;
      ws.send(JSON.stringify(msg));
    });

  ws.addEventListener("message", (ev) => {
    const msg = JSON.parse(ev.data);
    if (msg.id && pending.has(msg.id)) {
      const p = pending.get(msg.id);
      pending.delete(msg.id);
      msg.error ? p.reject(new Error(msg.error.message)) : p.resolve(msg.result);
    }
  });
  ws.addEventListener("close", () => {
    sessionId = null;
    setTimeout(() => connect().catch(retry), 2000);
  });
  ws.addEventListener("error", () => ws.close());

  await new Promise((resolve) => ws.addEventListener("open", resolve));

  // reuse the page target an agent session may already have created
  const { targetInfos } = await call("Target.getTargets");
  const existing = targetInfos.find((t) => t.type === "page");
  if (existing) {
    // pre-existing target: attach explicitly and use the returned session id
    const attached = await call("Target.attachToTarget", {
      targetId: existing.targetId,
      flatten: true,
    });
    sessionId = attached.sessionId;
  } else {
    // a freshly created target is auto-attached under a managed session id
    const created = await call("Target.createTarget", { url: "about:blank" });
    sessionId = `${created.targetId}-session`;
  }
  await call("Page.enable", {}, sessionId);

  // capture on an interval instead of Page.startScreencast: screencast
  // frames stream to the session that drives the page, so a passive
  // viewer stops receiving them once an agent takes over. An explicit
  // capture always reflects the current page state.
  const capture = async () => {
    if (ws.readyState !== WebSocket.OPEN || !sessionId) return;
    try {
      const shot = await call(
        "Page.captureScreenshot",
        { format: "jpeg", quality: 70 },
        sessionId
      );
      if (shot.data && shot.data.length > 100) broadcast(shot.data);
    } catch {
      // transient failures during navigation are normal, retry next tick
    }
    setTimeout(capture, 500);
  };
  capture();
}

function retry(err) {
  console.error(err.message);
  sessionId = null;
  setTimeout(() => connect().catch(retry), 2000);
}
connect().catch(retry);
```

> For a packaged version with adaptive pacing (fast while painting, idle back-off) and multi-page handling, see `tools/live-view.mjs`.

## Things worth knowing

* **Why captureScreenshot polling instead of screencast.** `Page.startScreencast` frames stream to the CDP session that drives the page, so once an agent session takes over, a passive viewer session stops receiving frames. It also only produces frames when the page paints something new. An explicit `Page.captureScreenshot` always reflects the current page state regardless of which session navigated, which makes it the reliable choice for watching someone else's session.
* **Acknowledge every frame.** Without `Page.screencastFrameAck`, frame delivery stops.
* **One client per page socket.** If an agent's CDP client already holds the page's `webSocketDebuggerUrl`, connecting to it again fails with `503`. Attach through the browser endpoint instead (`ws://127.0.0.1:9222/devtools/browser`) using `Target.attachToTarget` with `flatten: true`, then pass the returned session id as `sessionId` on every command.
* **Still images vs continuous view.** The MCP server exposes `browser_screenshot` for one-shot captures. Screencast is the right tool when you want to watch continuously; it stays CDP-only by design.

## Verifying

With `obscura serve` running and the viewer open:

1. Navigate from another client, for example `obscura fetch https://example.com` through a separate worker, or any Puppeteer/Playwright/MCP session connected to port 9222.
2. The tab shows the page within a second of it painting.
3. Closing the viewer tab and reopening it resumes from the most recent frame.


# Use as a Rust library

The `obscura` crate embeds the engine in a Rust program with a `Browser` / `Page` / `Element` API plus a cookie store, no CDP round-trips. It builds V8 from source, so it is a git dependency rather than a crates.io release.

### Add the dependency

```toml
[dependencies]
obscura = { git = "https://github.com/h4ckf0r0day/obscura" }
tokio = { version = "1", features = ["rt", "macros"] }
anyhow = "1"
```

The first build compiles V8 from source, so it is slow and needs the same build tools as [Build from source](/guides/build-from-source). Pin a tag for reproducible builds:

```toml
obscura = { git = "https://github.com/h4ckf0r0day/obscura", tag = "v0.1.7" }
```

### Quickstart

```rust
use obscura::Browser;
use std::time::Duration;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let browser = Browser::builder()
        .stealth(true)
        .build()?;

    let mut page = browser.new_page().await?;
    page.goto("https://example.com").await?;

    println!("URL: {}", page.url());
    println!("HTML bytes: {}", page.content().len());

    let el = page.wait_for_selector("h1", Duration::from_secs(5)).await?;
    println!("Heading: {}", el.text());

    let title = page.evaluate("document.title");
    println!("Title: {}", title);

    Ok(())
}
```

### API surface

`Browser::builder()` configures the engine: `.stealth(bool)`, `.proxy(url)`, `.user_agent(ua)`, `.storage_dir(dir)`, then `.build()`. `Browser::new()` uses defaults.

`Page`:

* `goto(url).await` navigate and wait for load
* `content()` rendered HTML
* `url()` current URL
* `evaluate(js)` run JavaScript, returns a `serde_json::Value`
* `query_selector(css)` first match as an `Element`, or `None`
* `wait_for_selector(css, Duration).await` poll until present
* `settle(max_ms).await` drive the event loop so async work (`fetch`, timers) completes
* `on_request(cb)` / `on_response(cb)` passive callbacks for every request and response
* `enable_interception()` channel to block, mock, or rewrite requests
* `add_preload_script(js)` run a script before the page's own scripts

`Element`: `text()`, `attribute(name)`, `click()`.

`CookieStore`: `set`, `get_all`, `get_for_url`, `save_to_file`, `load_from_file`.

### Intercept requests

The interception API observes, blocks, mocks, and rewrites the requests a page makes, including JavaScript `fetch()` and XHR. Use it to capture API payloads while crawling, block trackers, or mock responses in tests.

#### Passive callbacks

`on_request` and `on_response` fire for every request and response (navigation and JS `fetch()`/XHR) and are non-blocking. `on_response` is the main path for capturing the JSON an SPA loads asynchronously. Both return a stable id; pass it to `off_request` / `off_response` to detach the callback when a crawl phase is done. Callbacks are scoped to the page that registered them: they never fire for another page's requests and are dropped with the page.

```rust
use obscura::{Browser, ResourceType};
use std::sync::Arc;

let browser = Browser::new()?;
let mut page = browser.new_page().await?;

page.on_response(Arc::new(|info, resp| {
    if info.resource_type == ResourceType::Fetch {
        println!("{} -> {} bytes", info.url, resp.body.len());
    }
}));

page.goto("https://example.com").await?;
page.settle(2000).await;   // let in-page fetch() calls resolve
```

#### Active interception

`enable_interception()` returns a channel of every JS `fetch()`/XHR request. Resolve each through its `resolver` to pass, block, mock, or rewrite it.

```rust
use obscura::{Browser, InterceptResolution};

let mut page = browser.new_page().await?;
let mut rx = page.enable_interception();

tokio::spawn(async move {
    while let Some(req) = rx.recv().await {
        let action = if req.url.contains("/ads") {
            InterceptResolution::Fail { reason: "blocked".into() }
        } else if req.url.ends_with("/api/flags") {
            InterceptResolution::Fulfill {
                status: 200,
                headers: Default::default(),
                body: r#"{"newDashboard":true}"#.into(),
            }
        } else {
            // Pass through, or rewrite by setting url/method/headers/body.
            InterceptResolution::Continue { url: None, method: None, headers: None, body: None }
        };
        let _ = req.resolver.send(action);
    }
});

page.goto("https://example.com").await?;
page.settle(2000).await;
```

A `Continue` with `url: Some(...)` rewrites the target. The new URL is re-checked against the SSRF / private-network gate, so a rewrite cannot reach an internal address that would otherwise need `--allow-private-network`.

#### Preload scripts

`add_preload_script` runs a script before any of the page's own `<script>` tags (the CDP `Page.addScriptToEvaluateOnNewDocument` contract), so it can install hooks before the page bootstraps. Call it before `goto`.

```rust
let mut page = browser.new_page().await?;
page.add_preload_script("window.__patched = true;");
page.goto("https://example.com").await?;
```

`resource_type` reports `Fetch` for JS-initiated requests and does not yet split `Xhr` from `Fetch`.

### When to use which interface

* Embedding the engine in a Rust service: this crate.
* Driving from Node/Python with existing Puppeteer/Playwright code: the [CDP server](/quickstart/connect-puppeteer-or-playwright).
* Giving an AI agent browser tools: the [MCP server](/guides/use-the-mcp-server).
* One-off fetches and scraping from the shell: the [CLI](/reference/cli-reference).


# Persist cookies and storage

`--storage-dir` persists cookies and localStorage to disk so they survive across runs.

### CLI

```bash
obscura fetch https://example.com --storage-dir ./obscura-data
obscura fetch https://example.com --storage-dir ./obscura-data
```

The second invocation starts with the cookies and localStorage left by the first.

### Server

```bash
obscura serve --storage-dir ./obscura-data
```

All CDP sessions read and write to the same directory. Run separate `obscura serve` processes with different `--storage-dir` paths for isolated profiles.

### Layout

Inside `./obscura-data`:

* `cookies.json`: cookie jar in a stable format with `same_site`, `expires`, `http_only`, `secure`.
* `localStorage/<origin>.json`: one file per origin.

The format is stable. Inspect with `jq`:

```bash
jq '.[] | select(.domain == "example.com")' ./obscura-data/cookies.json
```

### When state is written

* On clean process exit (Ctrl-C, SIGTERM).
* After every navigation completes (CDP `Page.navigate`).
* Manually via CDP `Network.setCookie` and `Network.deleteCookies`.

### Login once, scrape many

```bash
obscura serve --storage-dir ./session-1
```

Drive a login flow once via Puppeteer or Playwright. Stop the server. Subsequent runs against the same `--storage-dir` start logged in.

### Multiple identities

```bash
obscura serve --port 9222 --storage-dir ./identity-a
obscura serve --port 9223 --storage-dir ./identity-b
```

### Clear state

```bash
rm -rf ./obscura-data
```


# Intercept and modify requests

CDP `Fetch.enable` lets you inspect, block, or modify every request the page makes.

### Block by resource type

Puppeteer:

```js
await page.setRequestInterception(true);

page.on('request', req => {
  if (['image', 'media', 'font'].includes(req.resourceType())) {
    req.abort();
  } else {
    req.continue();
  }
});
```

Playwright:

```js
await page.route('**/*', route => {
  if (['image', 'media', 'font'].includes(route.request().resourceType())) {
    route.abort();
  } else {
    route.continue();
  }
});
```

### Block by URL pattern

```js
// Puppeteer
page.on('request', req => {
  const url = req.url();
  if (url.includes('google-analytics.com') || url.includes('doubleclick.net')) {
    req.abort();
  } else {
    req.continue();
  }
});
```

```js
// Playwright
await page.route(/google-analytics\.com|doubleclick\.net/, route => route.abort());
```

### Modify headers

```js
// Puppeteer
page.on('request', req => {
  req.continue({
    headers: { ...req.headers(), 'X-Custom': 'value' },
  });
});
```

```js
// Playwright
await page.route('**/*', route => {
  route.continue({
    headers: { ...route.request().headers(), 'X-Custom': 'value' },
  });
});
```

### Return a fake response

```js
// Puppeteer
page.on('request', req => {
  if (req.url().endsWith('/api/feature-flags')) {
    req.respond({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify({ newDashboard: true }),
    });
  } else {
    req.continue();
  }
});
```

```js
// Playwright
await page.route('**/api/feature-flags', route => {
  route.fulfill({
    status: 200,
    contentType: 'application/json',
    body: JSON.stringify({ newDashboard: true }),
  });
});
```

### Strip analytics in production scrapes

```js
const BLOCK = [
  'google-analytics.com',
  'googletagmanager.com',
  'doubleclick.net',
  'facebook.net',
  'segment.io',
  'mixpanel.com',
  'hotjar.com',
];

page.on('request', req => {
  if (BLOCK.some(host => req.url().includes(host))) {
    req.abort();
  } else {
    req.continue();
  }
});
```

Built-in: `--stealth` ships with a tracker blocklist that handles most of these without per-script setup. See [Configure stealth and proxies](/guides/configure-stealth-and-proxies).

### From the Rust library

The patterns above drive interception over CDP from Puppeteer or Playwright. If you embed the engine with the `obscura` crate, the same capability is a native API on `Page`: `on_request` / `on_response` callbacks, an `enable_interception()` channel that can block, mock, or rewrite requests, and `add_preload_script` to run code before the page's own scripts. See [Use as a Rust library](/guides/use-as-a-rust-library#intercept-requests).


# Run in production at scale

### Docker

```bash
# The container runs as uid 65532, so a mounted storage dir must be writable
# by it. Without this the cookie jar silently fails to persist.
sudo install -d -o 65532 -g 65532 /srv/obscura/data

docker run -d \
  --name obscura \
  --restart unless-stopped \
  -p 127.0.0.1:9222:9222 \
  -v /srv/obscura/data:/data \
  h4ckf0r0day/obscura \
  serve --host 0.0.0.0 --storage-dir /data --stealth
```

The image runs `obscura serve` by default. Override with arguments after the image name.

#### The container does not run as root

The image is built on `gcr.io/distroless/cc-debian12:nonroot` and runs as uid/gid **65532**. Obscura executes untrusted page JavaScript in-process through V8, so a V8 exploit lands with the process's privileges — there is no reason for those to be root's.

Two consequences worth knowing:

* **A mounted `--storage-dir` must be writable by uid 65532**, as above. This is the one thing that breaks quietly rather than loudly. Verified: with an unwritable storage dir Obscura completes the run, exits `0`, and prints no warning — the cookie jar simply never persists. Check that `{storage-dir}/cookies.json` exists after your first run rather than assuming it does.
* **Nothing in the image needs a privileged operation.** It binds an unprivileged port, reads the CA bundle, and writes only to the storage dir and a temp dir. Verified in the non-root image: an HTTPS fetch succeeds, so the CA bundle is readable, and a writable storage dir is populated.

#### Why the in-container bind is `0.0.0.0`

A container-loopback bind is unreachable through `-p`, so `--host 0.0.0.0` is required for the published port to work at all. Publish to **host loopback** (`-p 127.0.0.1:9222:9222`, as above) rather than `-p 9222:9222`: the latter exposes the port on every host interface, and Docker's iptables rules bypass most host firewalls.

The CDP control plane has no authentication of its own: anything that can reach the port can drive the browser. See [Authentication](#authentication) for the controls that actually gate it.

### Systemd

`/etc/systemd/system/obscura.service`:

```ini
[Unit]
Description=Obscura headless browser
After=network.target

[Service]
ExecStart=/usr/local/bin/obscura serve --port 9222 --stealth --storage-dir /var/lib/obscura
Restart=always
RestartSec=5
User=obscura
Group=obscura
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

```bash
systemctl enable --now obscura
journalctl -fu obscura
```

### Workers

`obscura serve --workers N` runs N CDP server workers behind the listener.

```bash
obscura serve --workers 4
```

Use one worker per CPU core. Each worker handles its own pool of pages. Sessions are sticky to a worker.

### V8 heap

Default V8 heap is 4 GB on 64-bit systems. The defaults also cap the young generation (`--max-semi-space-size=4`) and pass `--optimize-for-size` to hold RSS down. Override:

```bash
obscura serve --v8-flags "--max-old-space-size=2048"
```

Flags you pass are appended after the defaults, and V8 uses the last value for a repeated flag, so your `--max-old-space-size` wins while the memory-tuning defaults stay in effect. Lower for memory-constrained hosts, raise for heavy SPAs.

### Parallel scrape

`obscura scrape` fans out a list of URLs across worker processes:

```bash
obscura scrape \
  --concurrency 20 \
  --format json \
  --timeout 60 \
  url1 url2 url3 ...
```

Reads URLs from stdin:

```bash
cat urls.txt | obscura scrape --concurrency 20 -
```

Requires `obscura-worker` next to `obscura` in `PATH`.

### Resource limits

Per-process memory cap with systemd:

```ini
[Service]
MemoryMax=4G
MemoryHigh=3G
```

Per-container with Docker:

```bash
docker run --memory=4g --cpus=2 ...
```

### Reverse proxy

Expose obscura on TLS through nginx or caddy:

```nginx
location /obscura/ {
  proxy_pass http://127.0.0.1:9222/;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
  proxy_read_timeout 86400;
}
```

CDP needs WebSocket upgrade and long read timeouts.

### Authentication

Obscura's CDP server has no built-in auth. Anyone who can reach the port can drive the browser. Options:

* Bind to `127.0.0.1` and require SSH for access (default).
* Put it behind a reverse proxy that enforces auth.
* Use Docker network isolation.

Never bind `0.0.0.0` on a public IP without one of the above.

### MCP HTTP transport

`obscura mcp --http` binds `127.0.0.1` by default. To reach it from another container, bind with `--host 0.0.0.0` and set an `Origin` allowlist so a browser page cannot drive it cross-origin:

```bash
OBSCURA_MCP_ALLOWED_ORIGINS="https://app.example.com" \
  obscura mcp --http --host 0.0.0.0 --port 3000
```

Request bodies are capped at 16 MiB. Like the CDP server it has no built-in auth, so keep it on an internal network or behind an authenticating proxy. See [Use the MCP server](/guides/use-the-mcp-server).

### Observability

```bash
obscura serve --verbose
RUST_LOG=obscura=debug obscura serve
```

`--verbose` enables info-level logs. `RUST_LOG=obscura=debug` enables debug-level. Logs go to stderr.

### Reliability and timeouts

The engine is hardened so one page cannot hang, crash, or wedge a worker. A V8 watchdog terminates runaway scripts and microtask storms, DOM ops are panic-safe, cyclic DOM mutations are rejected, and the CDP server terminates any single command that overruns its budget so a hung session cannot stall the others. Scripted `fetch()`/XHR and navigation are timeout-bounded. You can point the server at arbitrary or heavy pages without a stuck worker.

Tune the bounds with environment variables (see [Environment variables](/reference/environment-variables)):

```bash
OBSCURA_NAV_TIMEOUT_MS=60000 \
OBSCURA_SCRIPT_DEADLINE_MS=45000 \
OBSCURA_MODULE_BUDGET_MS=10000 \
OBSCURA_CDP_COMMAND_TIMEOUT_MS=70000 \
OBSCURA_FETCH_TIMEOUT_MS=20000 \
  obscura serve
```

`OBSCURA_NAV_TIMEOUT_MS` is the per-navigation ceiling (default 30000). `OBSCURA_SCRIPT_DEADLINE_MS` bounds the complete script phase (default 30000), while `OBSCURA_MODULE_BUDGET_MS` bounds each enhancement module's graph loading and evaluation (default 3000; unmounted SPA shells use the full script deadline). `OBSCURA_CDP_COMMAND_TIMEOUT_MS` is the outer per-CDP-command V8 deadline (default 60000, `0` disables), so keep it above the navigation ceiling. `OBSCURA_FETCH_TIMEOUT_MS` bounds scripted fetch/XHR and module network requests (default 30000).


# CLI reference

### `obscura`

Top-level flags apply to every subcommand.

```
-v, --verbose                Enable info logging
-p, --port <PORT>            CDP port (default 9222)
    --proxy <URL>            HTTP or SOCKS5 proxy
    --stealth                Consistent browser fingerprint + tracker blocking
    --obey-robots            Respect robots.txt
    --user-agent <UA>        Override the User-Agent
    --storage-dir <DIR>      Persistent cookies and localStorage
    --allow-private-network  Permit loopback / RFC1918 / link-local
    --v8-flags <FLAGS>       Raw V8 flags, applied at startup
-h, --help                   Help
-V, --version                Version
```

### `obscura fetch <URL>`

Load a URL and print its content or an evaluated expression.

```
    --dump <FORMAT>          html | text | links | markdown | original | assets | cookies
                             (default html)
    --selector <CSS>         Narrow output to a CSS selector
    --wait <SECONDS>         Fixed post-load delay; omitted uses adaptive settle (5s cap)
    --timeout <SECONDS>      Navigation timeout (default 30)
    --wait-until <LEVEL>     domcontentloaded | load | networkidle2 | networkidle0
                             (default load)
    --user-agent <UA>        Override the User-Agent
    --proxy <URL>            HTTP or SOCKS5 proxy
    --stealth                Consistent browser fingerprint + tracker blocking (global)
-e, --eval <JS>              Evaluate JS, print the result as JSON
-o, --output <FILE>          Write to a file instead of stdout
-s, --screenshot <FILE>      Capture the settled page as PNG (single URL)
-q, --quiet                  Suppress info logging
-v, --verbose                Enable verbose logging
```

`--screenshot` requires a render-enabled build. It uses a 1280×720 viewport by default and may be combined with `--eval`; the expression runs before capture, which is useful for scrolling or preparing page state. It is not available in `--file` batch mode.

When `--wait` is omitted, Obscura drives timers and async work until the page becomes quiescent, with a five-second ceiling. Supplying `--wait N` instead requests a fixed `N`-second delay. `--timeout` separately bounds navigation.

`--dump` values:

| Value      | Output                                                                                            |
| ---------- | ------------------------------------------------------------------------------------------------- |
| `html`     | Rendered HTML (default)                                                                           |
| `text`     | Plain text                                                                                        |
| `markdown` | Markdown conversion                                                                               |
| `links`    | Every `<a href>`, one URL per line                                                                |
| `assets`   | Every external resource, one JSON object per line (DOM assets plus `fetch()`/XHR requests)        |
| `original` | Raw HTTP response body (binary-safe, bypasses the engine)                                         |
| `cookies`  | All cookies in the jar as a JSON array, including HttpOnly cookies invisible to `document.cookie` |

### `obscura serve`

Run the CDP server. Puppeteer and Playwright connect over WebSocket.

```
-p, --port <PORT>            CDP port (default 9222)
    --host <HOST>            Bind host (default 127.0.0.1)
    --proxy <URL>            HTTP or SOCKS5 proxy
    --user-agent <UA>        Override the User-Agent
    --stealth                Consistent browser fingerprint + tracker blocking (global)
    --workers <N>            Worker processes (default 1)
    --allow-file-access      Permit CDP clients to navigate to file:// URLs
    --storage-dir <DIR>      Persistent cookies and localStorage
    --allow-private-network  Permit loopback / RFC1918 / link-local
-q, --quiet                  Suppress info logging
-v, --verbose                Enable info logging
```

Default endpoint is `ws://127.0.0.1:9222`.

### `obscura scrape [URLS]...`

Run a JS expression across many URLs in parallel.

```
-e, --eval <JS>              JS to run on each page
    --concurrency <N>        Parallel pages (default 10)
    --format <FORMAT>        Output format (default json)
    --timeout <SECONDS>      Per-URL timeout (default 60)
    --proxy <URL>            HTTP or SOCKS5 proxy
    --stealth                Consistent browser fingerprint + tracker blocking (global)
    --allow-private-network  Permit loopback / RFC1918 / link-local
-q, --quiet                  Suppress info logging
-v, --verbose                Enable verbose logging
```

`--stealth`, `--proxy`, and `--allow-private-network` are global flags: they work before or after any subcommand, so each worker in a `scrape` run inherits stealth too.

Read URLs from stdin with `-`:

```bash
cat urls.txt | obscura scrape - --eval "document.title" --concurrency 20
```

Requires `obscura-worker` next to `obscura` in `PATH`.

### `obscura mcp`

Run obscura as an MCP server.

```
    --http                   HTTP transport instead of stdio
    --host <HOST>            HTTP bind host (default 127.0.0.1)
    --port <PORT>            HTTP port (default 3000)
    --proxy <URL>            HTTP or SOCKS5 proxy
    --user-agent <UA>        Override the User-Agent
    --stealth                Consistent browser fingerprint + tracker blocking (global)
    --allow-private-network  Permit loopback / RFC1918 / link-local
-v, --verbose                Enable info logging
```

`--host` only applies with `--http`. The default `127.0.0.1` keeps the server loopback-only; set `0.0.0.0` to bind all interfaces (for example a Docker Compose sidecar) and pair it with `OBSCURA_MCP_ALLOWED_ORIGINS`.

Default transport is stdio. See [Use the MCP server](/guides/use-the-mcp-server).

Render-enabled builds add `browser_screenshot` and `browser_pdf` to the MCP tool list. Streaming screencasts are available through CDP rather than MCP.


# Environment variables

### Runtime

#### `OBSCURA_ALLOW_PRIVATE_NETWORK`

Allow fetches to loopback (`127.0.0.0/8`), RFC1918 (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`), and link-local (`169.254.0.0/16`, including the `169.254.169.254` cloud-metadata endpoint) addresses. The deny-set also covers the unspecified address (`0.0.0.0` / `::`), IPv6 unique-local (`fc00::/7`), and any IPv4-mapped form of the above. Off by default to block SSRF.

The guard validates at DNS-resolution time as well as on literal hosts, so a public hostname that resolves to a forbidden address is rejected at connect time (DNS-rebinding safe), not just hosts written as raw IPs.

Truthy values: `1`, `true`, `yes`, `on`.

```bash
OBSCURA_ALLOW_PRIVATE_NETWORK=1 obscura fetch http://localhost:8080
```

Per-process equivalent: `--allow-private-network` on any subcommand.

#### `OBSCURA_NAV_TIMEOUT_MS`

Hard ceiling on a single navigation. Default 30000 (30 seconds). Applies to `Page.navigate` and the CLI `fetch` command.

```bash
OBSCURA_NAV_TIMEOUT_MS=60000 obscura serve
```

#### `OBSCURA_NAV_CHAIN_LIMIT`

How many documents a navigation chain may load, the first navigation included. Default 10, which allows the requested document and nine navigations the page itself triggers via `location` assignments or form submissions. Raise the value for an endpoint that chains longer for good reasons, such as an SSO handover across several providers. The low default is what stops a page that resets `location` on every load.

A zero is raised to 1. This loads the requested document. If the page wants to chain further afterwards, the call reports an error, as at any other limit. A value the engine does not read as a number is replaced by the default. This also applies to a negative value and to a value with a trailing space.

The time budget is not tied to this limit. A longer chain usually also needs a higher `OBSCURA_NAV_TIMEOUT_MS`, because its default of 30 seconds applies to the whole chain and not to the individual document.

```bash
OBSCURA_NAV_CHAIN_LIMIT=20 obscura serve
```

#### `OBSCURA_SCRIPT_DEADLINE_MS`

Soft deadline for the complete page script-execution phase, including classic scripts and ES modules. Default 30000 (30 seconds). Raise it for a heavy SPA whose initial module is responsible for mounting an otherwise empty document. The engine also uses this value as a hard V8 watchdog budget, with a one-second grace period, so a synchronous script cannot run forever.

```bash
OBSCURA_SCRIPT_DEADLINE_MS=60000 obscura serve
```

#### `OBSCURA_MODULE_BUDGET_MS`

Per-module graph-loading and evaluation budget for modules that enhance an already-rendered page. Default 3000 (3 seconds). Raise it when a module such as the Vite HMR client legitimately needs longer to evaluate:

```bash
OBSCURA_MODULE_BUDGET_MS=10000 obscura serve
```

This shorter budget applies when the document body already contains more than 50 descendant nodes, where modules are normally progressive enhancement and should not delay navigation indefinitely. For an unmounted SPA shell, Obscura instead gives each module the full `OBSCURA_SCRIPT_DEADLINE_MS` budget so the app has time to mount. Module network requests remain independently bounded by `OBSCURA_FETCH_TIMEOUT_MS`.

#### `OBSCURA_CDP_COMMAND_TIMEOUT_MS`

Per-command deadline for the CDP server. A hung page (a runaway `Runtime.evaluate`, a synchronous DOM op) is terminated after this budget so one bad session cannot hold the shared V8 lock and stall the others. Default 60000 (60 seconds); `0` disables it. Navigation self-bounds via `OBSCURA_NAV_TIMEOUT_MS` well under this.

```bash
OBSCURA_CDP_COMMAND_TIMEOUT_MS=30000 obscura serve
```

#### `OBSCURA_FETCH_TIMEOUT_MS`

Request timeout for scripted `fetch()`, `XMLHttpRequest`, and ES-module loads. Without it a request to a server that accepts the connection but never responds (including a CORS preflight) hangs forever and the XHR is stuck with no completion event. Default 30000 (30 seconds).

```bash
OBSCURA_FETCH_TIMEOUT_MS=15000 obscura serve
```

#### `OBSCURA_PROXY`

Default proxy URL used by `obscura-worker` for the parallel `scrape` command when no `--proxy` flag is set.

```bash
OBSCURA_PROXY=http://proxy.example.com:8080 obscura scrape - < urls.txt
```

### Stealth and identity

These tune the browser identity the engine presents so it stays internally consistent. See [Configure stealth and proxies](/guides/configure-stealth-and-proxies) for the full picture.

#### `OBSCURA_TIMEZONE`

Pins the process timezone before V8/ICU reads it, so `Date` (`getTimezoneOffset`, `toString`) and `Intl.DateTimeFormat` report one consistent zone. Default `Europe/Berlin`. Set it to match the exit IP's region.

```bash
OBSCURA_TIMEZONE=America/New_York obscura serve
```

#### `OBSCURA_GEOLOCATION`

Override the coordinates the `navigator.geolocation` shim reports, as `lat,lon`. Without it the shim reports a fixed default. Keep it consistent with `OBSCURA_TIMEZONE` and the proxy region.

```bash
OBSCURA_GEOLOCATION="40.7128,-74.0060" obscura serve
```

#### `OBSCURA_PROFILE`

Pin a specific browser profile from the built-in pool by index (`0`-based). Each profile keeps `navigator.platform`, `userAgentData`, the UA string, and the GPU renderer internally consistent. Without it a single stable profile is used.

```bash
OBSCURA_PROFILE=2 obscura serve
```

#### `OBSCURA_ROTATE_PROFILE`

Opt into picking a random profile per browser context instead of the stable default. Leave it off when you pin a TLS fingerprint, proxy region, or timezone, since a rotated profile would no longer match those.

```bash
OBSCURA_ROTATE_PROFILE=1 obscura serve
```

### MCP

#### `OBSCURA_MCP_ALLOWED_ORIGINS`

Comma-separated `Origin` allowlist for the HTTP MCP transport (`obscura mcp --http`). Off by default, which keeps the permissive behavior. When set, a browser request whose `Origin` is not listed is refused with `403` before it can drive the server; native, non-browser MCP clients (which send no `Origin`) are always allowed. Use it to stop cross-origin pages from reaching a loopback MCP port.

```bash
OBSCURA_MCP_ALLOWED_ORIGINS="https://app.example.com" obscura mcp --http --host 0.0.0.0
```

### Logging

#### `RUST_LOG`

Standard `tracing` filter. Common settings:

```bash
RUST_LOG=obscura=info obscura serve
RUST_LOG=obscura=debug obscura serve
RUST_LOG=obscura_cdp=trace,obscura_browser=debug obscura serve
```

`--verbose` on the CLI is equivalent to `RUST_LOG=obscura=info`.

### Build

#### `OPENSSL_NO_VENDOR`

Forces `cargo build` to use the system OpenSSL instead of compiling the vendored copy. Set to `1` on hosts where the vendored OpenSSL fails (older VPS with AVX-512 issues).

```bash
OPENSSL_NO_VENDOR=1 cargo build --release --features render
```

### V8

V8 flags are passed via `--v8-flags`, not environment variables:

```bash
obscura serve --v8-flags "--max-old-space-size=2048 --expose-gc"
```

Defaults are `--max-old-space-size=4096 --max-semi-space-size=4 --optimize-for-size` on 64-bit systems (a 4 GB old-space ceiling, a capped young generation, and codegen tuned for a smaller footprint to cut RSS). Anything you pass with `--v8-flags` is appended after these, and V8 uses the last value for a repeated flag, so your value wins for that flag while the other defaults stay in effect.

### HTTP proxy environment

Obscura does not honor `HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY`. Use `--proxy` or `OBSCURA_PROXY`.


# Architecture overview

Obscura is a workspace of nine crates.

```
obscura-cli       CLI entry point. fetch, serve, scrape, mcp.
obscura-cdp       Chrome DevTools Protocol server. WebSocket, dispatch, domain handlers.
obscura-browser   Page type, navigation, lifecycle events.
obscura-js        V8 runtime via deno_core. bootstrap.js + Rust ops.
obscura-dom       DOM tree implementation.
obscura-net       HTTP client, stealth client, cookie jar, robots cache, tracker blocklist.
obscura-mcp       Model Context Protocol server.
obscura-render    CSS cascade, retained layout, text shaping, and CPU paint.
obscura           Embeddable Rust library API (Browser, Page, Element, CookieStore).
```

### Request flow

A `Page.navigate` from a CDP client:

```
CDP client (Puppeteer)
        │ WebSocket frame
        ▼
obscura-cdp/server.rs           accept, route by sessionId
        │
        ▼
obscura-cdp/dispatch.rs         method router, acquires v8_lock
        │
        ▼
obscura-cdp/domains/page.rs     Page.navigate handler
        │
        ▼
obscura-browser/page.rs         navigate_with_wait
        │
        ├──► obscura-net/client.rs        HTTP fetch
        │
        ├──► obscura-dom/tree.rs          parse HTML into the tree
        │
        └──► obscura-js/runtime.rs        run inline scripts
                  │
                  └──► bootstrap.js + ops.rs    DOM bindings
```

The dispatcher emits CDP events (`Network.requestWillBeSent`, `Page.frameNavigated`, `Page.lifecycleEvent`) back to the client through the same WebSocket.

### Rendering flow

`obscura-render` consumes the shared DOM and computed style state. Taffy provides the flex/grid foundation; Obscura adds browser formatting behavior, text shaping, intrinsic replaced-element sizing, retained geometry, scrolling, and CPU-backed paint. `obscura-js` exposes renderer-owned geometry to DOM APIs, `obscura-browser` prepares resources and owns capture, and `obscura-cdp` maps screenshots, screencast frames, and raster PDF output onto CDP.

Layout is retained between captures and invalidated by relevant DOM, style, viewport, scroll, animation, font, and resource changes. The same geometry therefore drives browser APIs and paint instead of maintaining separate measurement and screenshot models.

### Single V8 isolate

All pages in a process share one V8 isolate. The isolate is single-threaded by design.

`obscura_js::v8_lock::global()` is a `tokio::sync::Mutex` that serializes V8 work. A handler that wants to run JS must acquire the lock first:

```rust
let _guard = obscura_js::v8_lock::global().lock().await;
page.evaluate(expr).await
```

The dispatcher routes long-running operations (navigation, eval) through `process_with_interception` in `server.rs`, which spawns the work onto the tokio `LocalSet` and releases the dispatcher to keep handling other CDP messages.

This is why `Target.createTarget` from many concurrent clients works: each `newPage` returns immediately while the actual navigation runs in a spawned task.

### Robustness

One page cannot hang or crash the process. `obscura-js/runtime.rs` provides a V8 termination watchdog (`arm_watchdog`, `run_event_loop_bounded`) that terminates the isolate from a separate thread when synchronous work overruns a budget, because `tokio::time::timeout` cannot preempt synchronous V8. It bounds the post-load settle, the navigation event-loop pumps, and `--eval`. The complete script phase is bounded by `OBSCURA_SCRIPT_DEADLINE_MS`; enhancement modules have a shorter per-module graph-loading/evaluation budget controlled by `OBSCURA_MODULE_BUDGET_MS`, while modules mounting an empty SPA shell receive the full script deadline. `obscura-js/cdp_watchdog.rs` is a single shared watchdog the dispatcher arms around every CDP command, so a runaway page cannot hold the V8 lock and wedge other sessions (tunable via `OBSCURA_CDP_COMMAND_TIMEOUT_MS`). `op_dom` is wrapped in `catch_unwind` so a DOM-op panic degrades to a null result instead of aborting the process through V8's FFI frame, and `obscura-dom/tree.rs` rejects cyclic reparenting that would make tree walks loop forever. Scripted `fetch()`/XHR and module network requests are timeout-bounded (`OBSCURA_FETCH_TIMEOUT_MS`), and the one-shot `fetch` CLI has a process-level hard deadline as a final backstop.

### JS bridge

`obscura-js/js/bootstrap.js` provides the browser globals: `document`, `window`, `navigator`, `location`, observers, fetch, indexedDB, etc.

`obscura-js/src/ops.rs` registers Rust ops that the bootstrap calls into:

```js
Deno.core.ops.op_dom('insert_before', parentNid, refNid, newNid);
```

Adding a Web API usually means:

1. JS shim in `bootstrap.js` that exposes the API surface.
2. Rust op in `ops.rs` that performs the side effect (DOM mutation, fetch, crypto).
3. Register the op in `build_extension()`.

Worked example: [Adding a CDP method or Web API](/contributing/adding-a-cdp-method-or-web-api).

### Classic Web Workers

The JavaScript shim executes each classic Worker source once and retains its message handlers and lexical state. Bare `onmessage` assignments target the worker scope, and messages posted before the source loads are queued until initialization finishes. Terminating a worker discards pending messages.

Workers remain emulated within the page runtime, not separate V8 isolates or OS threads. This is not a complete WorkerGlobalScope implementation.

### CDP session model

Each CDP client connection gets attached to one or more targets. Session IDs are `"{targetId}-session"`. The dispatcher routes by `sessionId` in the incoming frame to the right `Page`.

Targets are created by `Target.createTarget`. Closing the WebSocket detaches all sessions but leaves the pages running.

### Lifecycle

Lifecycle events are emitted by `obscura-browser/lifecycle.rs` as the page transitions:

```
init → commit → domcontentloaded → load → networkidle2 → networkidle0
```

`waitUntil` on `Page.navigate` blocks until the requested level is reached. The Puppeteer / Playwright `goto` resolves on the matching `Page.lifecycleEvent` client-side.

### Storage

`--storage-dir` persists cookies (`cookies.json`) and localStorage (`localStorage/<origin>.json`). Reads on process start, writes on every navigation and on graceful shutdown.

### Stealth

`--stealth` swaps the default `reqwest` client for `obscura-net/wreq_client.rs`, which presents a real browser's TLS ClientHello, ALPN, and cipher order (a consistent Chrome fingerprint, not a randomized one) so the TLS layer matches the User-Agent and JS surfaces. It also applies the bundled tracker blocklist before any request leaves the process. Scripted `fetch()`/XHR go through the same stealth client, so subresource requests carry the same fingerprint as the navigation. `--stealth` is a global CLI flag that applies to `fetch`, `serve`, `scrape`, and `mcp`.

### Workspace conventions

* One crate per layer. Cross-crate calls go through the layer above, not sideways.
* All async is `tokio` with a `LocalSet` because V8 is `!Send`.
* All DOM ops go through `op_dom` to keep the JS/Rust boundary narrow.


# Adding a CDP method or Web API

Two recipes for the two most common extensions: a new CDP method, and a new JS Web API.

### Adding a CDP method

Worked example: `MyDomain.doThing` that takes `{ name }` and returns `{ ok }`.

#### 1. Add the handler

Create or edit a file under `crates/obscura-cdp/src/domains/`:

```rust
// crates/obscura-cdp/src/domains/my_domain.rs
use serde_json::{json, Value};
use crate::dispatch::CdpContext;

pub async fn do_thing(
    params: &Value,
    _ctx: &mut CdpContext,
    _session_id: &Option<String>,
) -> Result<Value, String> {
    let name = params.get("name")
        .and_then(|v| v.as_str())
        .ok_or("missing name")?;

    // do the work

    Ok(json!({ "ok": true, "name": name }))
}
```

#### 2. Register in the dispatcher

In `crates/obscura-cdp/src/dispatch.rs`, add a match arm:

```rust
"MyDomain.doThing" => domains::my_domain::do_thing(&req.params, ctx, &req.session_id).await,
```

#### 3. Test it

`crates/obscura-cdp/tests/cdp_my_domain.rs`:

```rust
use obscura_cdp::dispatch::{dispatch, CdpContext};
use obscura_cdp::types::CdpRequest;
use serde_json::json;

#[tokio::test(flavor = "current_thread")]
async fn my_domain_do_thing_returns_ok() {
    let mut ctx = CdpContext::new();
    let resp = dispatch(&CdpRequest {
        id: 1,
        method: "MyDomain.doThing".into(),
        params: json!({ "name": "test" }),
        session_id: None,
    }, &mut ctx).await;

    assert!(resp.error.is_none());
    assert_eq!(resp.result.unwrap()["ok"], true);
}
```

Run:

```bash
cargo nextest run --release --features render -p obscura-cdp my_domain
```

### Adding a Web API

Worked example: `crypto.subtle.digest`, real implementation backed by a Rust hash op.

#### 1. Add the Rust op

In `crates/obscura-js/src/ops.rs`:

```rust
#[op2]
#[buffer]
fn op_subtle_digest(#[string] algorithm: &str, #[buffer] data: &[u8]) -> Vec<u8> {
    use sha1::Digest as _;
    match algorithm.to_ascii_uppercase().as_str() {
        "SHA-1"   => sha1::Sha1::digest(data).to_vec(),
        "SHA-256" => sha2::Sha256::digest(data).to_vec(),
        "SHA-384" => sha2::Sha384::digest(data).to_vec(),
        "SHA-512" => sha2::Sha512::digest(data).to_vec(),
        _         => sha2::Sha256::digest(data).to_vec(),
    }
}
```

#### 2. Register the op

In the same file, `build_extension()`:

```rust
ops: std::borrow::Cow::Owned(vec![
    op_dom(),
    op_console_msg(),
    // ...
    op_subtle_digest(),
]),
```

#### 3. Add the JS shim

In `crates/obscura-js/js/bootstrap.js`:

```js
globalThis.crypto = globalThis.crypto || {};
globalThis.crypto.subtle = globalThis.crypto.subtle || {};
globalThis.crypto.subtle.digest = function digest(algorithm, data) {
  const algName = typeof algorithm === 'string' ? algorithm : algorithm.name;
  const bytes = data instanceof ArrayBuffer
    ? new Uint8Array(data)
    : new Uint8Array(data.buffer, data.byteOffset, data.byteLength);
  const out = Deno.core.ops.op_subtle_digest(algName, bytes);
  return Promise.resolve(out.buffer);
};
```

#### 4. Add a dependency if needed

`crates/obscura-js/Cargo.toml`:

```toml
sha1 = "0.10"
sha2 = "0.10"
```

#### 5. Smoke test

```bash
cargo build --release --features render
./target/release/obscura fetch https://example.com --eval "
  crypto.subtle.digest('SHA-256', new TextEncoder().encode('hi'))
    .then(buf => Array.from(new Uint8Array(buf)).map(b => b.toString(16).padStart(2, '0')).join(''))
"
```

### Tips

* Keep the JS shim thin. All side effects go through ops.
* Use `Promise.resolve` to keep async-shaped APIs callable from sync ops.
* Match the spec: Web API names and shapes are checked by Puppeteer / Playwright wrappers.
* DOM mutations go through `op_dom`, not new ops.
* For events that need to fire across handlers, use the existing `_makeListenerBox` helper in `bootstrap.js`.

### Worked examples in the tree

* CDP method with intercept: `crates/obscura-cdp/src/domains/page.rs` `do_navigate`.
* Web API with op + JS shim: `crypto.subtle.digest` (above).
* Web API in pure JS (no op): `DOMParser` in `bootstrap.js`.
* Web API with async event firing: `WebSocket`, `IntersectionObserver` in `bootstrap.js`.


# Testing and debugging

### Test suites

#### Rust unit and integration

```bash
cargo nextest run --release --features render --no-fail-fast
```

Crate-scoped:

```bash
cargo nextest run --release --features render -p obscura-cdp
cargo nextest run --release --features render -p obscura-browser
```

By name:

```bash
cargo nextest run --release --features render runtime_click_submit_prevent_default
```

Use `cargo nextest`, not `cargo test`. Runtime tests require process isolation because the engine owns one V8 isolate per process. Render tests must run in release mode; debug builds are not a fidelity or performance gate.

#### CDP parity tests

`crates/obscura-cdp/tests/cdp_*.rs` exercise CDP methods end-to-end with a real `dispatch` call and an in-process HTTP server.

Pattern:

```rust
#[tokio::test(flavor = "current_thread")]
async fn my_test() {
    std::env::set_var("OBSCURA_ALLOW_PRIVATE_NETWORK", "1");
    let url = serve_once().await;
    let mut ctx = CdpContext::new();
    let page_id = ctx.create_page();
    let session_id = "session-1";
    ctx.sessions.insert(session_id.to_string(), page_id.clone());

    cdp(&mut ctx, 1, "Page.navigate", json!({"url": url}), session_id).await;
    // assertions
}
```

`serve_once` and `cdp` helpers are copied across the parity tests; reuse them.

### Logging

```bash
RUST_LOG=obscura=info  obscura serve
RUST_LOG=obscura=debug obscura serve
RUST_LOG=obscura_cdp=trace,obscura_browser=debug obscura serve
```

Logs go to stderr.

`--verbose` on any subcommand is equivalent to `RUST_LOG=obscura=info`.

### Driving the CDP server manually

```bash
obscura serve --port 9222 --verbose
```

In another shell:

```bash
wscat -c ws://127.0.0.1:9222
> {"id":1,"method":"Target.createTarget","params":{"url":"about:blank"}}
> {"id":2,"method":"Target.attachToTarget","params":{"targetId":"...","flatten":true}}
> {"id":3,"sessionId":"...-session","method":"Page.navigate","params":{"url":"https://example.com"}}
> {"id":4,"sessionId":"...-session","method":"Runtime.evaluate","params":{"expression":"document.title"}}
```

Useful for reproducing what Puppeteer or Playwright is doing without their abstraction.

### Common failure modes

#### `Target.createTarget timed out`

Lock contention in the dispatcher. Should not happen on current main. If it does, run with `RUST_LOG=obscura_cdp=trace`, look for handlers that hold `v8_lock` across long awaits.

#### `page.goto()` returns `null` from Puppeteer

Means `Network.requestWillBeSent` for the main document did not arrive with `requestId == loaderId`. Check `do_navigate` in `crates/obscura-cdp/src/domains/page.rs`.

#### `Cannot find context with specified id`

Playwright's local context counter diverged from the server's `valid_context_ids`. Each navigation must allocate a fresh `executionContextId`. Check `ctx.next_isolated_context()` is called on every nav.

#### `V8_Fatal: heap->isolate() == Isolate::TryGetCurrent()`

Two pages tried to use V8 concurrently. The `v8_lock` was bypassed, or a handler suspended a JS runtime while another isolate was entered. Search for direct `JsRuntime` access outside the lock.

#### Test hangs

A handler is awaiting something that never resolves. Run with `RUST_LOG=obscura=trace` and check the last log line before the hang.

### Reproducing user bug reports

The integration suite in `tests/test_all.py` is the fastest path from a one-line repro to a regression test. Add the failing case as a new test function, get it failing, then fix.

For Puppeteer / Playwright bug reports, the user's repro script usually drops straight in. Save it as `tests/repro_<issue>.js`, run with `node`, fix until it passes.

#### Rendering regressions

Start with the committed deterministic fixtures, then use the representative real-site suite at both the top and bottom of pages:

```bash
RUN_ROOT="$(mktemp -d)"
OBSCURA_BIN=./target/release/obscura render-repros/run.sh "$RUN_ROOT/fixtures"
OBSCURA_BIN=./target/release/obscura render-repros/representative-suite/run.sh "$RUN_ROOT/top"
OBSCURA_BIN=./target/release/obscura render-repros/representative-suite/run.sh "$RUN_ROOT/bottom" bottom
```

Set `BASELINE_BIN` or `CHROMIUM_BIN` when producing paired captures. Keep the viewport, user agent, settle policy, scroll position, animation sample, and capture boundary identical. A pixel-distance score is a regression tripwire, not a verdict: verify both engines succeeded and produced nonblank output, then inspect missing resources, geometry, structural edges, and a reduced fixture. Do not add hostname-specific render branches.

### Profiling

CPU with `perf` and a flamegraph:

```bash
cargo build --release --features render
perf record -F 99 -g -- ./target/release/obscura fetch https://heavy-spa.example
perf script | flamegraph.pl > flame.svg
```

Memory with heaptrack:

```bash
heaptrack ./target/release/obscura serve
```

Tokio task inspection:

```bash
RUSTFLAGS="--cfg tokio_unstable" cargo build --release --features render
./target/release/obscura serve
# in another shell
tokio-console
```

Requires the workspace `tokio` dependency to be built with the `tracing` feature; not enabled by default, add it in the relevant `Cargo.toml` before profiling.


