Dataset: UEyes (Jiang et al., CHI 2023) — the 495 web screenshots, 62 participants, Gazepoint fixation logs Method: Zero-shot saliency prediction scored against human fixations Original run: April 2026 · Re-run and corrected: July 29, 2026

Correction notice

This page previously reported that our saliency model beat a center-bias baseline (AUC-Judd 0.663 vs 0.615). That result does not reproduce, and the corrected result runs the other way.

Re-running the benchmark against the code we actually ship, scored against real fixation points, a plain center-bias blob beats every persona variant of our model on all three metrics and on 72-80% of individual pages.

The original number came from a Python reimplementation that had drifted from the shipped TypeScript, evaluated with a metric that was not measuring what its name says. Details in What went wrong. The superseded results are preserved at the bottom of this page.

We are publishing the negative result in full because a benchmark you only trust when it flatters you is not a benchmark.

Results

Table 1: Saliency prediction accuracy (N=495 web pages)

Ground truth is real fixation points from all 554 participant logs, filtered to a 7-second viewing horizon, deduplicated per fixation, invalid tracker samples dropped. Predictions come from analyzeAttention — the shipped entry point, not a research reimplementation.

Method	AUC-Judd	NSS	CC
elderly	0.5864 (0.102)	0.1894 (0.304)	0.0817 (0.119)
neurotypical	0.5854 (0.102)	0.2582 (0.353)	0.1089 (0.138)
dyslexic	0.5828 (0.102)	0.1650 (0.291)	0.0729 (0.113)
power user	0.5817 (0.102)	0.2470 (0.356)	0.1056 (0.139)
low vision	0.5817 (0.102)	0.1739 (0.310)	0.0763 (0.122)
ADHD	0.5817 (0.102)	0.2833 (0.349)	0.1205 (0.136)
Center bias	0.6580 (0.078)	0.5373 (0.282)	0.1872 (0.118)
Random	0.5103 (0.031)	0.0197 (0.108)	0.0069 (0.035)

Mean (SD) across 495 pages. Higher is better throughout. Chance AUC is 0.500.

Table 2: Center bias minus model, paired per page

Metric	Difference	t	Center bias wins on
AUC-Judd	+0.0727	14.6	78% of pages
NSS	+0.2791	15.3	80% of pages
CC	+0.0784	10.6	72% of pages

Each page contributes one paired difference, so variation between pages cancels. This is not a near-tie or an averaging artifact — the baseline wins on roughly three quarters of individual pages.

Key findings

A center-bias prior beats our bottom-up saliency model on web pages. By 0.073 AUC, 0.279 NSS, and 0.078 CC. This is the headline, and it is the opposite of what this page said before.

That is a strong baseline, not a trivial one. Center bias is the single strongest prior in saliency research (Tatler, 2007), and it is stronger on web stimuli than on natural photographs: page layouts put the logo, hero, and primary call-to-action top-and-center; viewers start at the center of a newly presented screenshot; and a screenshot cannot be scrolled, so nothing pulls fixations away. Any saliency claim about interfaces has to clear center bias first. Ours did not.

Persona differentiation is not detectable in this data. All six personas fall within 0.0047 AUC of each other (0.5817 to 0.5864). The previous version of this page reported ADHD separated from neurotypical by 0.058 and called that meaningful; the measured separation is 0.0037, below the 0.02 threshold that page proposed for itself.

The three metrics disagree about which persona is worst. ADHD is tied-lowest on AUC while being the best persona on NSS and CC. These metrics are reported together precisely because they catch different failure modes (Bylinskii et al., 2019). When they point in opposite directions, the honest conclusion is that no effect has been detected — not that two out of three agree.

The persona filters are doing something; it just is not this. The filters are not inert. On a single page, total attention mass ranges from 80.6 (dyslexic) to 367.1 (ADHD), a 4.5x spread, with per-cell differences up to 0.40 on a 0-1 scale. So the persona layer changes the map a great deal while changing its agreement with human fixations almost not at all. It moves attention around without moving it anywhere more correct.

What went wrong

Three defects, each independently sufficient to produce the original result.

1. The benchmark scored a port, not the product

The original harness used a Python file describing itself as a "vectorized port of CBrowser's computeLabSaliency()." The port had drifted from the shipped TypeScript: our implementation weights chrominance 3x over luminance; the port summed all three CIE-Lab channels equally. Both files were written the same day.

This is the defect that produced the inversion. We can isolate it: rerunning the shipped model under the original, flawed evaluation protocol still scores 0.585, not 0.663. The baseline, meanwhile, reproduces within 0.012. The published 0.663 was a property of the port. The product never scored it.

2. The "fixation map" was not a set of fixations

The original metric built its positive set as fixation_map > 0, where fixation_map is a Gaussian-smoothed density image. That marks every pixel the smoothing kernel ever touched — measured on this corpus, 18.8% of all pixels in the image. AUC-Judd is defined over discrete fixation points; scoring against a fifth of the image is a different quantity that happens to share a name.

The corpus ships 554 raw eye-tracker logs. The original harness even contained a function to parse them. It was never connected to the metric.

3. A tie-handling bug in the AUC implementation

The original assigned the i-th sorted fixation a true-positive rate of i / n_fixations, which is only correct if every fixation sits at a distinct saliency value. Combined with defect 2, the ties were enormous.

The same bug existed in our first TypeScript implementation, where it scored a completely flat map 0.25 instead of the analytically required 0.5. It was caught there because those tests were written to theoretical answers rather than to whatever the code happened to output.

How the harness changed

The benchmark now scores shipped code by construction:

Metrics live in src/visual/saliency-metrics.ts — part of the package, not a research script — and are unit-tested against analytic answers: a perfect predictor scores 1.0, a flat map 0.5/0/0, an inverted map CC of -1, AUC invariant under monotonic rescaling, NSS invariant under affine rescaling.
Corpus parsing lives in src/visual/fixation-corpus.ts, drops invalid tracker samples, and deduplicates per fixation so a long dwell counts once rather than becoming a vote multiplier.
Predictions come from analyzeAttention, the shipped path. There is no port to drift.

Before trusting any of the corrected numbers we checked the coordinate mapping: mean ground-truth density at the real fixation coordinates is 3.54x the global mean. A broken mapping would put that near 1.0.

What this does and does not mean

It does not mean the product is wrong. This benchmark measures one layer — bottom-up visual saliency — on static screenshots.

It cannot test the semantic layer at all. On a real page, analyzeAttention blends 35% bottom-up saliency with 65% DOM semantics: element type, persona priority, goal relevance. Screenshots have no DOM, so the majority of the production model never ran in this study. Every prediction here was generated in visual-only mode. Nothing in these results speaks to the semantic layer in either direction.

It does mean three previously published claims are withdrawn:

"COT outperforms center bias on all metrics" — reversed.
"The separation of 0.058 AUC exceeds the threshold for meaningful persona differentiation" — the separation is 0.0037.
"ADHD diverges as theoretically predicted" — ADHD is the best persona on two of three metrics. The original explained a divergence that is not in the data.

It does establish a useful result. Our bottom-up saliency layer does not beat a center-bias prior on web interfaces. Our separate ASD crossover study reached the same mechanistic conclusion from the opposite direction: fitting the persona parameters to two different populations converged on the identical parameter set, with exactly zero difference when cross-applied. Both results say the same thing about the four global parameters in the persona filter — exponent, bias, centerBias, threshold can make attention broader, tighter, or more central, but they cannot express what a population attends to. That is a structural limit, not a tuning problem.

The layer that could plausibly carry a population difference is the semantic one. It is untested, because no public corpus pairs clinical labels with interface stimuli.

Context

State-of-the-art deep saliency models (UMSI++, DeepGaze++) trained on millions of fixations reach AUC around 0.87 on UEyes. Center bias reaches 0.658 here. Our zero-shot model reaches 0.585. The gap to a learned model was always expected; the gap to the baseline was not.

Method

Saliency prediction

For each page, a persona-specific saliency map is produced by:

CIE-Lab color space conversion
Multi-scale center-surround contrast (Bures-Wasserstein distance at 3 spatial scales), weighting chrominance 3x over luminance
Persona attention filters (novelty weight, text weight, global integration, peripheral sensitivity, threshold)
Center-bias modulation scaled by global integration capacity
Sub-threshold suppression based on the persona's attention threshold

On a live page a sixth step blends this with DOM semantics at 35/65. That step does not run on static images.

Evaluation metrics

AUC-Judd — area under the ROC curve, treating real fixation points as positives. Chance is 0.5.
NSS — normalized scanpath saliency: mean z-scored predicted saliency at fixation points.
CC — Pearson correlation between the predicted map and the continuous fixation density map.

AUC and NSS take discrete fixation points. CC compares the two continuous maps.

Persona profiles

Persona	Novelty	Text	Global Integration	Peripheral	Threshold
Neurotypical	1.0	1.0	1.0	1.0	50th pct
ADHD	2.0	0.5	0.4	1.5	30th pct
Low Vision	0.3	0.8	0.6	0.3	70th pct
Elderly	0.7	1.3	0.8	0.6	55th pct
Dyslexic	1.0	1.5	0.9	0.9	50th pct
Power User	0.5	0.8	1.2	0.7	60th pct

Study 1b: Schwartz value modulation

Study 1b measured how motivational-value modulation changes Cognitive Transport Cost across the same 495 pages and 6 personas, reporting changes from -20.0% to +10.7%, all significant at p < 0.001.

Scope correction. Study 1b has no human ground truth. Its 2,970 "observations" are 495 pages x 6 personas evaluated by the model against itself, once with the modulation term and once without. Its hypothesis — that adding a non-zero term to a cost function changes the cost — is confirmed by arithmetic and could not have failed. The significance tests report that the modulation was applied, not that it is correct.

Study 1b is a sensitivity analysis: it shows which layers the value coefficients move and by how much. It is not evidence about how people with different values experience web pages, and earlier write-ups (including our blog post) stated it that way. Those have been corrected.

Table 3: CTC change with value modulation (N=495, model-internal)

Persona	Baseline CTC	Modulated CTC	Delta	Delta %
Neurotypical	0.870	0.738	-0.132	-14.3%
ADHD	1.209	1.346	+0.137	+10.7%
Power User	1.644	1.313	-0.331	-20.0%
Elderly	1.784	1.681	-0.102	-5.5%
Dyslexic	1.161	1.014	-0.146	-12.1%
Low Vision	1.385	1.285	-0.101	-7.1%

These are model outputs, not measurements of people.

Value modulation slightly reduced persona differentiation. Cross-persona CTC variance fell from 0.141 to 0.138, a ratio of 0.9826 — a 1.7% decrease. The modulation pulled the personas together rather than apart. (An earlier write-up reported 9.8%, from a superseded version of the model.)

On the live check. We previously reported running two personas against cbrowser.ai and finding different attention maps, described as confirming that "values change what people see." Two personas producing different maps on one page demonstrates that the value parameters are wired end-to-end and reach the saliency filter. It is a plumbing test with n=1 page and no human data, and it cannot show what people see. Presented as validation, that was an overstatement.

Superseded results (April 2026)

Preserved for reference. These numbers are not reproducible and should not be cited. They were produced by the drifted Python port under the flawed evaluation protocol described in What went wrong.

Method	AUC-Judd	NSS	CC
COT (neurotypical)	0.663	0.544	0.216
COT (color-blind)	0.660	0.536	0.212
COT (low vision)	0.660	0.489	0.191
COT (dyslexic)	0.659	0.531	0.210
COT (elderly)	0.659	0.521	0.205
COT (ADHD)	0.605	0.355	0.148
Center Bias	0.615	0.362	0.140
Random	0.500	0.001	0.000

A separate null result from the original run still stands: page demand computed from raw image features (edge density, pixel entropy, region count) did not predict fixation entropy (r = -0.062, p = 0.169) or dispersion (r = 0.059, p = 0.190). Image features conflate visual complexity with visual richness, and the full demand mapping needs DOM structure a screenshot cannot provide.

References

Jiang, Y., et al. (2023). UEyes: Understanding visual saliency across user interface types. Proc. CHI '23. ACM.
Bylinskii, Z., et al. (2019). What do different evaluation metrics tell us about saliency models? IEEE TPAMI, 41(3), 740-757.
Judd, T., Durand, F., & Torralba, A. (2012). A benchmark of computational models of saliency to predict human fixations. MIT CSAIL Technical Report.
Tatler, B.W. (2007). The central fixation bias in scene viewing. Journal of Vision, 7(14):4.
Klein, D.A., & Frintrop, S. (2012). Center-surround divergence of feature statistics for salient object detection. Proc. DAGM/OAGM. Springer.
Schwartz, S.H. (1992). Universals in the content and structure of values. Advances in Experimental Social Psychology, 25, 1-65.
Roccas, S., Sagiv, L., Schwartz, S.H., & Knafo, A. (2002). The big five personality factors and personal values. PSPB, 28(6), 789-801.

Study 1 Saliency Benchmark

Correction notice