LAYA / VISION
INDEPENDENTJEVAL.DEV ↗
API STATUS
●Jevision research / UI grounding

Teaching vision
where to click.

We are adapting a fast typed-decision model to recognize controls, locate them on UI and website screenshots, and return action-ready coordinates. Every comparison uses immutable held-out screens from its benchmark version.

RUN / 0646–52CONTROLS / DENSE SCREEN64HELD-OUT SCREENS117 msFOUR DECISIONS / RTX 3090
CONTROL ROLE32.8 → 73.4%+40.6 percentage points
PRESENCE59.4 → 100%same held-out seeds
CENTER ERROR253.6 → 229.1 pxprobability-weighted coordinate
EXACT CLICK0%not production-ready yet
●Progress / immutable runs

Improvement over time.

Each candidate starts from a pinned checkpoint and is evaluated against disjoint seeds. A run is retained even when one metric regresses, so the record shows what actually worked.

RunRolePresenceCenter errorLatency
01Pinned baselineUntouched Laya checkpoint32.8%59.4%253.6 px123.7 ms
02Head adaptation8.5M trainable parameters48.4%100.0%399.4 px130.9 ms
03Last-two layers15.5M trainable parameters52.3%100.0%276.6 px129.8 ms
04Full text tower114.7M trainable parameters73.4%100.0%229.1 px131.4 ms
05Dense v2 baseline~48 controls / held-out screen17.2%21.9%463.0 px117.3 ms
06Dense v2 candidateNot promoted · binding failed15.6%100.0%467.0 px117.4 ms

Reading this honestly: v1 semantic recognition improved strongly, but its screens were too sparse. Dense v2 made the test about 12× more crowded. Its first candidate learned presence but regressed role and localization, so it was rejected. Exact click success remains zero and no research candidate has replaced the public API model.

●Paired evidence

Same screen. Better decision.

These are individual held-out examples, not training samples. The image, target box, baseline output, and candidate output are keyed by deterministic seed in the row-level evidence.

Synthetic UI benchmark seed 10049SEED 10049
TARGETtext field / “Settings”BOX 122, 220 → 367, 268
BASELINEbutton412.4 px error64.7% present
CANDIDATEtext field280.2 px error98.4% present
Synthetic UI benchmark seed 10054SEED 10054
TARGETcheckbox / “Save”BOX 379, 337 → 534, 371
BASELINEbutton137.4 px error34.5% present
CANDIDATEcheckbox65.1 px error98.8% present
Synthetic UI benchmark seed 10065SEED 10065
TARGETtext field / “Save”BOX 323, 225 → 551, 273
BASELINEbutton190.8 px error82.9% present
CANDIDATEtext field105.9 px error98.2% present
●Next loop

From recognition to reliable action.

The flat 16-way coordinate head learned semantics faster than geometry. The next loop changes the task, not just the training duration.

01Region proposal

Find the coarse screen region containing the described control.

02Crop + refine

Re-encode a focused crop and resolve a precise continuous point.

03Click verification

Compare the next frame and confirm that the intended UI state changed.

04Real websites

Add browser screenshots with DOM-derived labels, then test on unseen sites.

TRAIN

Synthetic interfaces, browser screenshots, multiple themes, resolutions, zoom levels and control states.

MEASURE

Click success, center error, role accuracy, presence calibration, p50/p95 latency and peak VRAM.

PROMOTE

Only candidates that improve held-out action success without hiding regressions or changing the test set.

●Reproducibility

A result is a directory, not a screenshot.

Model revision 8b318c99… · evaluation format jevision-ui-eval-v2 · seeds 10000–10063 · BF16 · RTX 3090 · one warm model · four typed decisions per screen. Training and evaluation scripts live in the private Jevision mirror with create-only outputs and row-level JSONL predictions.