Back to docs

Tool Competitive Benchmark

See how your UX stacks up against competitors.

competitive_benchmark runs the same cognitive journey, with the same persona and goal, on your site and your competitors' sites, then ranks them. It shows which site reached the goal, which was faster, which needed fewer steps and which had less friction, with recommendations for the sites that lost.

Needs an Anthropic API key. Each site is one model-driven cognitive journey. Since 20.0 there is no keyless mode: the old heuristic simulation called a goal reached when goal words appeared on the page, so its rankings could not be trusted. Without a key the call fails with an error that names the missing key. On your own machine, set ANTHROPIC_API_KEY or run cbrowser config set-api-key <key>.

Quick Start

{
  "sites": [
    "https://example.com",
    "https://competitor-a.com",
    "https://competitor-b.com"
  ],
  "goal": "sign up for free trial"
}

What happens:

  1. A cognitive journey runs on each site with the same persona and goal, up to three sites at a time.
  2. Each site gets a score from its journey.
  3. Sites are ranked by score, with strengths and weaknesses relative to the average.
  4. Each site that did not win gets recommendations that point at the competitor doing better.

Parameters

Parameter Type Required Default Description
sites array Yes - URLs to compare
goal string Yes - Task goal, e.g. sign up for free trial
persona string No first-timer Persona for every journey
maxSteps number No 8 Max steps per site (keep it low to avoid timeouts)
maxTime number No 20 Max time per site, in seconds

Output

Field What it holds
goal, persona What was tested
ranking One entry per site: rank, site, score, strengths, weaknesses
comparison fastestSite, slowestSite, mostFriction, leastFriction, highestAbandonmentRisk, commonFrictionAcrossSites
recommendations Up to 5: site, improvement, competitorReference
duration Milliseconds for the whole benchmark

When two sites tie on a metric, comparison says tie (...) and names them, rather than picking one. With a single site there is nothing to compare, and the metric says so.

How a site is scored

Part Points
Goal reached 50
Speed Up to 30, less for slower journeys
Steps Up to 10, less for more steps
Friction Up to 10, less for more friction points

A journey that fails

If a site's journey throws an error, that site gets a failed result instead of a guess: goal not reached and abandonment risk 100. It ranks with the weakness "Failed to achieve goal", and the benchmark goes on with the other sites. The CLI's JSON report (--output) also lists each site's abandonment reason, which names the error. Before 20.0 a failed journey fell back to the heuristic simulation.

Use Cases

Conversion gap

Your signup converts at 15% and a competitor's at 25%. Run both with the same persona and goal and read the weaknesses and recommendations for your site.

{
  "sites": [
    "https://yoursite.com/signup",
    "https://competitor.com/signup"
  ],
  "goal": "create an account",
  "persona": "first-timer"
}

Checkout

{
  "sites": [
    "https://yourstore.com",
    "https://competitor.com"
  ],
  "goal": "buy a lamp",
  "maxSteps": 12
}

Tips

Choose fair comparisons

Compare the same journey on each site: signup to signup, not signup to checkout. Similar audiences and industries make the ranking easier to act on.

Try more than one persona

Different personas can rank the same sites in a different order. Run the benchmark once per persona, for example first-timer and elderly-user.

Cost

Each site is one journey, and each journey step is a model call, so the cost grows with the number of sites and with maxSteps.

CLI

cbrowser competitive-benchmark \
  --goal "sign up for free trial" \
  --html --sites \
  "https://a.com,https://b.com"

The CLI's defaults are longer: --max-steps 30 and --max-time 180 seconds per site. It also takes --persona, --output, --device and --viewport.

Last updated: v20.0.0

From the Blog