Back to docs

Screen Capture

Record the screen to GIF, WebP or video and get an AI-readable frame manifest alongside it. (v18.70.0)

The Problem

Recording a browser session gives you a video, and a video is a dead end for an agent:

  • An AI agent cannot watch an MP4 — it can only read text
  • "The button flickered at some point" is not a timestamp you can act on
  • Console errors and network failures live in a different log with a different clock
  • Screenshots catch a moment; the bug lives in the transition between moments
  • A 500-frame recording has no index, so finding the one frame that matters means watching all of it

The Solution

CBrowser's screen capture:

  1. Records pixels over time via the CDP screencast
  2. Writes every frame to disk as a numbered JPEG
  3. Emits a manifest.json describing each frame — timestamp, crop, SSIM against the previous frame
  4. Marks where the page actually changed, at two sensitivity tiers
  5. Attaches the console messages and network requests that landed inside each frame's window
  6. Encodes the human artifact (GIF/WebP/WebM) from the same frames

One capture, two audiences. A human opens the GIF. An agent reads the manifest.

Quick Start

# Record the viewport for five seconds
npx cbrowser capture start https://example.com --duration 5s

# Record a fixed region of the page
npx cbrowser capture start https://example.com --region 0,0,640,480 --duration 5s

# Track an element as it moves
npx cbrowser capture start https://example.com --element "#cart-badge" --duration 5s

# Stop when a spinner disappears instead of after a fixed time
npx cbrowser capture start https://example.com --until-element-gone ".spinner"

# Record at a phone size, as WebP, with a contact sheet
npx cbrowser capture start https://example.com --device iphone-15 --format webp --contact-sheet --duration 5s

Reading the Manifest

This is the part that makes a recording queryable. Every capture writes manifest.json into the output directory next to the frames and the encoded artifact.

The top level describes the run:

{
  "version": 3,
  "slug": "demo1",
  "engine": "chromium",
  "capture_method": "cdp-screencast",
  "target_fps": 10,
  "actual_fps": 10.194015126603091,
  "viewport": { "width": 1280, "height": 800, "deviceScaleFactor": 1 },
  "target": { "kind": "viewport" },
  "start_trigger": null,
  "stop_trigger": "duration",
  "trigger_timeout": false,
  "started_at": "2026-07-19T20:47:43.648Z",
  "duration_ms": 3041,
  "output_dims": { "width": 1280, "height": 800 },
  "method": "changeScore-window-min",
  "frames": [],
  "change_points": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30],
  "key_frames": [4, 5, 9, 10, 14, 15, 19, 20, 24, 25, 29, 30],
  "change_points_saturated": true,
  "ssim_thresholds": { "change": 0.95, "key": 0.6 },
  "frame_gaps": [],
  "region_clamped": false,
  "artifacts": { "gif": "demo1.gif" }
}

(That example is a real capture of a progress-bar page: the bar advances every frame, so nearly every frame is a change point and the list is flagged saturated — while key_frames picks out only the frames where something notable happened.)

version is 3 on this build. Version 2 added the two-tier change signal — key_frames, change_points_saturated and ssim_thresholds; version 3 changed the change detector (see below), which rescaled every score. The tier fields are optional in the schema so older manifests keep parsing, but the engine always writes them. Scores are only comparable against the thresholds stored in the same file — a v2 file's scores live on a different scale than a v3 file's, which is exactly what the ssim_thresholds values and the method field are there to disambiguate.

The frame array

Every frame carries the same nine keys:

{
  "index": 9,
  "t_ms": 862,
  "path": "frames/0009.jpg",
  "crop": { "x": 0, "y": 0, "width": 1280, "height": 800 },
  "ssim_prev": 0.18855447401859635,
  "anomaly": true,
  "tracking": null,
  "console": [],
  "network": []
}
Field Description
index Position in the sequence, 0-based and dense
t_ms Milliseconds since started_at
path Frame image, relative to the manifest's own directory
crop Source-image rect this frame was cut from, after clamping
ssim_prev Change score against the previous frame (see the two-tier signal below); null on frame 0. Lower means more of the frame changed
anomaly Frame flagged as a visual anomaly by the change detector
tracking "ok" or "stale" while tracking an element; null otherwise
console Console messages that landed in this frame's window
network Network records that landed in this frame's window

An agent reads this instead of watching the video. "Which frame did the layout shift on" is ssim_prev. "What was on screen at 1.4 seconds" is a t_ms lookup returning a JPEG path. "Where did the element go" is the crop sequence.

The two-tier change signal

One threshold cannot answer both questions a reader asks, so the manifest reports two.

Field Threshold (v3) Question it answers
change_points score < 0.95 Did anything change at all
key_frames score < 0.6 Did something notable happen

The score is not a whole-frame average. Version 3 replaced global SSIM with minimum SSIM over a grid of windows ("method": "changeScore-window-min"): the frame is divided into windows and the score is the worst window. A whole-frame average is a quality metric — a small change (a counter ticking, a badge updating) gets diluted by the unchanged 99% of the page and becomes invisible at any threshold. Taking the minimum makes a localized change score like a change, whatever fraction of the frame it occupies.

That also makes tier membership stable across viewport sizes for any change large enough to be resolved at the signature size — form fills, clicks, navigations, content loads; real interactions. The boundary is honest: scoring happens on a 128px downsample of the frame, so a change occupying a tiny fraction of a very large viewport (a 7-character status word on a 4K screen) can be averaged away before the windows ever see it, and whether it survives depends on the viewport. If sub-pixel-fraction changes matter to your capture, they are the one case the default signature does not reliably classify.

On the window-min scale, an unchanged frame scores exactly 1.0, a small localized change lands around 0.87, a form fill around 0.21, and large-area changes go to 0 and below. The thresholds sit in the measured gaps: 0.95 catches everything that visibly changed while staying above the noise floor, and 0.6 separates "a pixel region ticked over" from "something happened that a person would notice".

Saturation is still possible — a progress bar advancing every frame makes nearly every frame a change point. change_points_saturated is true when more than 70% of frames qualify: read key_frames instead in that case. Because the key threshold is lower, key_frames is always a subset of change_points, and it is the same signal that sets each frame's anomaly flag, so the two agree by construction.

Frame gaps

Capture is event-driven, not a fixed-rate sampler. The screencast emits a frame when the page repaints, so --fps is a ceiling rather than a guarantee. A page that sits still emits nothing, and the manifest reports the still stretch rather than padding it out with duplicate frames:

"frame_gaps": [ { "after_index": 1, "gap_ms": 3015 } ]

The same three-second window on a continuously animating page produced 32 frames at 10.5 actual fps; on a static page it produced 4 frames at 1.3 actual fps, an empty change_points array and the gap above. Both are correct. actual_fps and frame_gaps are how you tell the difference between "nothing happened" and "the capture broke".

Correlated console and network events

Console messages and network records are attached to the frame whose window they landed in, so a visual symptom and its cause arrive together:

{
  "index": 2,
  "t_ms": 107,
  "path": "frames/0002.jpg",
  "crop": { "x": 0, "y": 0, "width": 1280, "height": 800 },
  "ssim_prev": 0.9639129118951039,
  "anomaly": false,
  "tracking": null,
  "console": [
    { "t_ms": 177, "type": "log", "text": "TICK interval wall=1784492334675" }
  ],
  "network": []
}

Console entries carry t_ms, type and text. Network entries carry t_ms, url and optionally method, status and failed. Both shapes are permissive — unknown keys survive parsing — so the recorder can attach engine-specific extras without breaking a reader.

The Three Targets

Viewport

The default. Records the whole visible page.

npx cbrowser capture start https://example.com --duration 3s --name checkout

The manifest records "target": { "kind": "viewport" }, and every frame's crop is the full viewport.

Region

A fixed rect, given as x,y,width,height. Useful when you know where the thing lives and everything else is noise.

npx cbrowser capture start https://example.com --region 0,0,640,480 --duration 3s

"target": { "kind": "region", "rect": { "x": 0, "y": 0, "width": 640, "height": 480 } },
"output_dims": { "width": 640, "height": 480 },
"region_clamped": false

A region larger than the viewport is clamped to fit, and region_clamped records that it happened.

Tracked element

A CSS selector. The crop follows the element as it moves.

npx cbrowser capture start https://example.com --element "#box" --element-padding 20 --duration 3s

"target": { "kind": "element", "selector": "#box", "padding": 20 },
"output_dims": { "width": 241, "height": 241 }

Each frame then carries its own fractional crop plus a tracking state:

{
  "index": 3,
  "t_ms": 198,
  "crop": { "x": 521.4876708984375, "y": 222.76394653320312,
            "width": 263.28460693359375, "height": 263.28460693359375 },
  "ssim_prev": 0.9055371971724897,
  "tracking": "ok"
}

This is a CLI-only behaviour. The element parameter on the capture_start MCP tool resolves the selector to a bounding box once at start and does not follow the element if it moves.

Bounding a Capture

A capture needs an end. You can bound it by time, by a DOM condition, or by visual stillness — and you can compose a start trigger with a stop trigger.

Option Description
--duration <5s> Auto-stop after this long (5s, 2m, 1500ms)
--after <event> Start on load, domcontentloaded or networkidle
--after-element <sel> Start when the element is present and visible
--after-delay <500ms> Extra delay after the start trigger
--until-element <sel> Stop when the element appears
--until-element-gone <sel> Stop when the element disappears
--until-idle <800ms> Stop after this long with no visual change
--timeout <30s> Trigger wait budget (default: 30s)

Composed, they cut the capture down to the interesting window:

# Start once the modal is visible, wait 200ms more, stop when the spinner clears
npx cbrowser capture start https://example.com \
  --after-element "#modal" --after-delay 200ms \
  --until-element-gone ".spinner"

Both triggers are recorded in the manifest, including how long the start trigger waited:

"start_trigger": "element:#box+200ms (waited 220ms)",
"stop_trigger": "duration"

A trigger that never fires is not a silent hang. --timeout bounds the wait; on expiry the capture stops, the manifest and the encoded artifacts are still written, and the process exits non-zero — so CI sees a failure while a human still gets the footage:

✗ Capture trigger timed out
  Stop trigger: element-appears:#never-appears-xyz (timed out after 4000ms)
  Frames: 42 (10.3 fps actual vs 10 requested)
  Duration: 4060ms
  Manifest: /path/to/out/manifest.json
  GIF: /path/to/out/demo4.gif

That run sets "trigger_timeout": true and records the timeout in stop_trigger.

Formats

Pass a comma-separated list to --format. The default is gif.

Format Status
gif ✅ Works out of the box
webp ✅ Works out of the box
webm ✅ Works out of the box
mp4 ⚠️ Needs a full ffmpeg

Playwright's bundled ffmpeg is a stripped WebM/VP8-only build, so mp4 does not encode against it. Install a full ffmpeg and point CBROWSER_FFMPEG_PATH at it, or use --format webm.

A partial format failure never loses the frames. The formats that could encode are still written and still listed in artifacts; only the exit code changes:

✓ Capture stopped
  Stop trigger: duration
  Frames: 22 (10.7 fps actual vs 10 requested)
  Duration: 2061ms
  Manifest: /path/to/out/manifest.json
  WEBM: /path/to/out/demo8.webm
  ✗ mp4: Cannot encode MP4: ~/.cache/ms-playwright/ffmpeg-1011/ffmpeg-linux is missing muxer 'mp4' and video encoder (need one of: libx264, libopenh264, h264_videotoolbox, mpeg4). Playwright's bundled ffmpeg is a stripped build (WebM/VP8 only). Install a full ffmpeg and set CBROWSER_FFMPEG_PATH, or re-run with --format webm.

--contact-sheet adds one JPEG summarising the whole capture as an evenly spaced visual sample. It lands in artifacts under contact-sheet.

Recording a Test Run

Both test-suite and cognitive-journey take the same four capture flags, so an existing run becomes a recording without rewriting it.

# Record a natural-language test suite
npx cbrowser test-suite login-flow.txt --capture --capture-out ./recordings

# Record an inline test at 8 fps
npx cbrowser test-suite --inline "go to https://example.com ; click login" --capture --capture-fps 8

# Record a cognitive journey as WebM
npx cbrowser cognitive-journey --capture --capture-format webm
Option Default Description
--capture false Record the run to GIF/WebP/video
--capture-fps 10 Capture frame rate
--capture-format gif Comma-separated: gif,webp,webm,mp4
--capture-out - Capture output directory

The capture summary prints after the test report:

🎥 Capture: 16 frames over 1666ms (9.6 fps)
   Manifest: /path/to/recordings/manifest.json
   GIF: /path/to/recordings/test-suite-inline-suite-1784492253937.gif

The manifest is the same schema. The run's own lifetime is the bound, so stop_trigger is "manual", and --capture-fps is honoured as target_fps.

Driving a Live Capture

A bounded capture runs fine in a single process. An open-ended one — no --duration, no stop trigger — needs the daemon, because otherwise the capture would live and die inside that one CLI process and capture stop from another shell would never reach it.

npx cbrowser daemon start

Then start the capture, drive the page from any shell, and stop it when you are done:

# Starts and returns immediately
npx cbrowser capture start https://example.com --name checkout-flow

# These drive the same page and land in the recording
npx cbrowser click "#add-to-cart"
npx cbrowser fill "#email" "[email protected]"
npx cbrowser navigate https://example.com/checkout

# Finish
npx cbrowser capture stop

capture start says so explicitly:

🎥 Capturing in the daemon (cdp-screencast)
  Name: demo10
  Output: /path/to/out

  The capture is live and this shell is free. Commands you run now drive
  the same page and appear in the recording:
    cbrowser navigate <url> / click <sel> / fill <sel> <value>
  Finish with: cbrowser capture stop

Without the daemon, an open-ended capture refuses rather than misleading you:

Error: an open-ended capture needs the daemon, which is not running.
  Without it the capture would live and die inside this one process, so
  'capture stop' from another shell and any later click/fill/navigate
  would never reach it.

  Start it:      cbrowser daemon start
  Or bound it:   --duration 5s | --until-element <sel> | --until-idle 800ms

In-process captures also carry a 10-minute safety cap, which applies only when no --duration was given.

Checking on a capture

capture status reports the running capture, or falls back to the most recent one:

🎥 Capture in progress (in the daemon)
  Name: demo10
  Frames so far: 14
  Elapsed: 4944ms
  Method: cdp-screencast
  Output: /path/to/out
  Stop with: cbrowser capture stop

No capture in progress. Most recent:
  Name: busy
  Started: 2026-07-19T20:15:45.620Z
  Frames: 2 over 45ms (44.4 fps)
  Capture: cdp-screencast
  Key frames: 0
  Change points: 0
  Artifacts: gif -> busy.gif
  Manifest: ~/.cbrowser/videos/e9fa8018/busy/manifest.json

capture stop with nothing running prints No active capture. A capture is stopped from the process that started it. and exits non-zero.

Capture Is Not Record

Two different features with similar names and no shared output.

capture record
Records Pixels over time User actions
Output GIF/WebP/WebM/MP4 + manifest Playwright test code
Use it to Show and query what happened Generate a test from a session

MCP Tools

Three tools, in the screen_capture category — tier 2, loaded on demand, pro pricing.

Tool Description
capture_start Start capturing the screen in the background
capture_stop Stop, encode and return the manifest path plus change points
capture_status Report the running or most recent capture

Capture runs in the background, so the pattern is: call capture_start, interact with the page through the other tools, then call capture_stop. Pass the _browserToken you received from a previous tool call, or you get a blank new browser instead of the page you were on.

capture_stop deliberately does not return the frame array — 500 frames of paths, crops, SSIM scores and per-frame records is megabytes of JSON that would blow the response budget. It returns frames_dir instead, and truncates the unbounded lists with their true totals alongside:

{
  "success": true,
  "slug": "checkout-flow",
  "manifest_path": "/path/to/out/manifest.json",
  "frames_dir": "/path/to/out/frames",
  "frames": 32,
  "target_fps": 10,
  "actual_fps": 10.45,
  "duration_ms": 3060,
  "change_points": [1, 2, 3],
  "change_points_total": 31,
  "frame_gaps": [],
  "frame_gaps_total": 0,
  "console_errors": [],
  "console_errors_total": 0,
  "network_requests_total": 14,
  "artifacts": { "gif": "checkout-flow.gif" }
}

The *_total counts are what let a model tell "3 change points" from "200 shown of 4,812". Frames and video are always returned as paths, never inlined; only the contact sheet is inlined, and it is size-budgeted.

Options

Option Default Description
--fps 10 Target frames per second
--duration - Auto-stop after this long (5s, 2m, 1500ms)
--viewport - Viewport size, e.g. 1280x720
--region - Capture a fixed region as x,y,w,h
--element - Track an element; crop follows it as it moves
--element-padding 0 Expand the element box by n CSS px per side
--device - Device preset, e.g. iphone-15
--format gif gif,webp,webm,mp4
--quality 80 JPEG quality of captured frames (1-100)
--max-frames 3000 Stop retaining frames after n
--contact-sheet false Also write one JPEG summarising the capture
--after - Start on load, domcontentloaded or networkidle
--after-element - Start when the element is present and visible
--after-delay - Extra delay after the start trigger
--until-element - Stop when the element appears
--until-element-gone - Stop when the element disappears
--until-idle - Stop after this long with no visual change
--timeout 30s Trigger wait budget
--out ~/.cbrowser/videos/<session>/<name> Output directory
--name - Capture name

Device presets

--device accepts: iphone-15, iphone-15-pro-max, pixel-8, pixel-8-pro, samsung-galaxy-s24, ipad-pro-12, ipad-air, desktop-1080p, desktop-1440p, mobile, tablet, desktop. An unknown name is rejected with the list.

Multiple viewports

--viewport accepts a comma-separated list, which runs sequential captures rather than one. Each size gets its own subdirectory and its own manifest, and the size is appended to the slug — sharing one output directory would let the second run overwrite the first run's frames.

npx cbrowser capture start https://example.com --viewport "800x600,375x667" --duration 2s --out ./caps --name demo

caps/800x600/manifest.json
caps/800x600/demo-800x600.gif
caps/375x667/manifest.json
caps/375x667/demo-375x667.gif

A multi-size --viewport with no --duration and no stop trigger is rejected outright, since each run has to be able to end on its own.

CI/CD Integration

# GitHub Actions
- name: Record the checkout suite
  run: |
    npx cbrowser test-suite checkout.txt --capture --capture-format webm --capture-out ./recordings

- name: Fail if the page never repainted
  run: |
    GAPS=$(cat ./recordings/manifest.json | jq '.frame_gaps | length')
    if [ "$GAPS" -gt "0" ]; then
      echo "Capture recorded $GAPS frame gap(s) - the page stopped repainting"
      exit 1
    fi

- name: Upload the recording
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: capture
    path: ./recordings

A trigger timeout exits non-zero on its own, so --until-element doubles as an assertion that the element ever showed up.

Best Practices

  1. Read key_frames before change_points - check change_points_saturated first; on an animating page the sensitive tier names almost every frame
  2. Bound every capture - --duration or a stop trigger keeps the run out of the daemon and out of the 10-minute cap
  3. Use --until-element-gone for loading states - it ends the recording at the moment the wait ended, instead of guessing a duration
  4. Prefer --element over cropping later - the tracked crop follows the element, so the subject stays centred as it moves
  5. Check actual_fps against target_fps - a large gap means the page was not repainting, not that the capture failed
  6. Ship --contact-sheet to reviewers - one JPEG reads faster than a GIF and survives a pull request comment
  7. Use webm rather than mp4 in CI - it encodes with the bundled ffmpeg and needs no extra install

Related

Copyright: (c) 2026 Alexa Eden.

License: MIT License

Contact: [email protected]

From the Blog