Coverage and accuracy benchmarks

The numbers behind the matcher.

These panels keep the raw coverage and accuracy measurements available for anyone who wants to audit the claim. Every figure is labeled with exactly what it tested. Coverage refreshes hourly; the accuracy benchmarks refresh on their own cadence and each panel shows its last run.

Last updated Jul 13, 2026Shown in your local time. Data refreshes continuously; each panel below also lists its own last run.

How we measure (and what we do not claim)

Some game-matching numbers measure an easier variant of the real task: we take a player who is already in our index, hand the matcher a few of their games, and check whether it puts that player at the top of the list. Walking in cold with games from a player whose record was built without those games is harder, and we do not claim those numbers until they are measured. A leave-N-out holdout mode that tests exactly that already exists, and this page automatically promotes holdout figures to the primary position the moment an artifact publishes them. Until then, every figure says which variant it measured. The standard we hold ourselves to is on the trust page.

Reading the accuracy tables

The held-out accuracy samples are small (currently on the order of dozens of trials per sample size), so the single best-guess rate (top-1) jumps around from run to run and can even fall as sample size rises. Read top-10 as the stable signal: it is what the visitor-facing result set shows. We only promote a number to a public headline after at least 30 trials.

Plain-language glossary for the technical terms below

Searchable account (graph-ready): An account whose opening moves are stored and indexed so the matcher can find it by playing style. A fingerprint without this is invisible to search.
Style record (fingerprint): A compact record of an opponent’s opening moves: which positions they reach, how often, and how recently.
Production search method (opening graph): The newer matcher that stores each account’s actual opening lines instead of only a compact fingerprint.
Older fallback matcher (sparse cascade / sparse fingerprint): A simpler, faster fingerprint matcher kept visible for comparison and regression checks.
Cold-discovery test (leave-N-out holdout): We remove some of a player’s games from their record, then try to identify them using only those removed games. The honest simulation of meeting a new opponent.
Already-indexed test (with_target_indexed): The player’s full record is in the index while we re-rank them. Easier than cold discovery, and labeled wherever it appears.
Right answer in the first N guesses (top-1 / top-3 / top-10): The share of test runs where the correct account appeared first, in the first three, or in the first ten candidates.
Test runs (trials): How many times the test ran. We require 30 or more before publishing a number as reliable.
Shuffled starting points (seeds): Random seeds vary which games each test samples, so a number is never one lucky draw.
Average finishing position (MRR, mean reciprocal rank): Average of 1 divided by the rank. First place every time scores 1.0; always second place scores 0.5. Higher is better.
Measured pool (target frontier / denominator): The pool of accounts coverage is measured against: active accounts above our rating floors on each platform.
Typical and busy players (p50 / p90): The middle player’s game count, and the count that 90 percent of players stay under.
End-to-end test (Scout visitor flow): Held-out games are pasted through the same public pages a visitor uses, parser and API included, before checking the result.
Human-move check (Maia policy): A neural network trained on human games at each rating level. It asks whether a real player at that strength would choose this move.
By-design test result (SYNTHETIC verdict): A pass or fail from constructed scenarios where the correct answer is known by design, not from production player data.

Scout 24-ply fill status

Deeper opening evidence (up to 24 plies, 12 own-side moves) is filling in two layers. Beta is the fast 120-game retrieval layer for broad style and preparation leads. Public is the adaptive 36-month / 5,000-game layer behind the paid reliability claim. Coverage is measured against the frozen launch cohort.

Real ETA: Beta Jul 13, 2026 to Jul 13, 2026; Public Aug 21, 2026 to Sep 29, 2026.

Data refresh · last refresh 7/13/2026

Beta · fast retrieval layer

Filling

100.4%236,985 of 236,110 addressable accounts

Public · adaptive reliability layer

Pre-gate

1.5%3,443 of 236,110 addressable accounts

Beta and Public with acceleration

Acceleration is the operating assumption, not a fallback scenario.

Use this as the real expectation for Beta and Public readiness. The raw current-cadence projection is folded below only as a warning line for what would happen if we stopped accelerating.

Real ETA: Beta Jul 13, 2026 to Jul 13, 2026; Public Aug 21, 2026 to Sep 29, 2026.

Beta with acceleration

Jul 13, 2026 to Jul 13, 2026

Acceleration stays on. Remaining: 0 accounts.

Public with acceleration

Aug 21, 2026 to Sep 29, 2026

Acceleration stays on. Remaining: 232,667 accounts.

Accelerated launch timeline

Open by default because this is the plan we are actually running.

Today

Jul 13, 2026

reached

Beta with acceleration

Jul 7, 2026

reached

Public with acceleration

Aug 21, 2026

~39 days out

Real expectation: Beta Jul 13, 2026 to Jul 13, 2026; Public Aug 21, 2026 to Sep 29, 2026. This is based on the recovery speeds we have actually seen from manual Lichess sweeps, safe Chess.com canaries, and repeated graph catch-up.

Beta operating window

Expected range: Jul 13, 2026 to Jul 13, 2026

Internal target: Jul 7, 2026

Remaining: 0 accounts.

Realistic operating pace: 8,000-15,000/day. Target pace needed: 0/day.

Operator forecast from proven manual Lichess sweep yield, safe Chess.com API canaries, and planned fast-layer acceleration.

Why this is the real estimate

Manual Lichess month sweeps proved that tens of thousands of fast-layer accounts can be recovered in bounded runs when a fresh month has yield.
The Chess.com API lane has run with zero 429s in canaries; current recurring batches are safe but intentionally throttled.
The graph writer is not the main limiter for Beta once source rows exist; bounded catch-up has repeatedly cleared the source-ready tail.

Main risk: If no new high-yield Lichess month or Chess.com ramp is opened, the raw current-cadence diagnostic is the fallback path and Beta slips.

Public operating window

Expected range: Aug 21, 2026 to Sep 29, 2026

Internal target: Aug 21, 2026

Remaining: 232,667 accounts.

Realistic operating pace: 3,000-6,000/day. Target pace needed: 5,974.27/day.

Operator forecast for the stronger adaptive/final reliability claim after Beta, assuming cache reuse and final certification lanes continue improving.

Why this is the real estimate

The fast layer preserves the map and cache needed to prioritize adaptive certification instead of starting the Public layer cold.
The final layer is deliberately stricter: it counts fixed 36-month or 5,000-game evidence plus adaptive-certified rows with matching source and graph checksums.
High-demand and top-ranked uncertified accounts can be promoted from Beta cache into deeper certification lanes after Beta opens.

Main risk: Public readiness depends on sustained adaptive certification after Beta; it is not the same thing as the fast 120-game retrieval fill.

Operating ETAs are the real expectation based on recovery speeds we have already proven and acceleration lanes we are actively operating. Raw current-cadence ETAs remain a diagnostic for what happens if acceleration stops.

Current bottleneck

Source refresh throughput is the current Beta bottleneck.

Graph: Graph catch-up is productive, but larger local all-platform passes can run long; use smaller bounded passes or the Cloud Run graph lane for tails.

Source: Recent movement is mostly chess.com API recovery plus small graph tails; stored-only probes found very little new source inventory and Lichess stayed flat.

Next speed-up: Continue the safer Chess.com API ramp with runtime-budget cooldowns, and prepare a reviewed Lichess month sweep. More graph tuning or stored-only probing alone will not hit the Beta pace.

Caution: Keep Public reliability separate from Beta: final certification still needs adaptive 36-month or 5,000-game evidence.

Show stop-acceleration diagnostic

Raw current-cadence diagnostic only

Raw projection from +-7 Beta / day, +0 Public / day over the last 0.3 days if acceleration stopped. We are not planning to stop acceleration, so this stays collapsed as a diagnostic, not the real ETA.

Today

Jul 13, 2026

reached

Beta only if acceleration stopped

Pending

measuring fill rate

Public only if acceleration stopped

Pending

sustained fill cadence not yet established

Platform	Cohort accounts	Beta ready	Public certified	Still filling	Terminal
chess.com	104,889	100,000(100.9%)	3,443(3.5%)	5	5,797
lichess	142,886	136,985(100.0%)	0(0.0%)	32	5,868
All	247,775	236,985(100.4%)	3,443(1.5%)	0	11,665

Advanced technical notes

Artifact kind scout24_fill. Cohort scout_24_launch_2026_06. Beta is the fast retrieval layer behind style and preparation leads; Public is the stricter reliability layer behind the paid claim, measured against the frozen launch cohort minus evidence-excluded accounts. ETAs project the day-over-day fill rate against the previously published snapshot (prior snapshot); a layer with no recent progress shows a pending ETA rather than a stale date.

Production evidence gates

These verdicts are measured on production data or stay pending until production data exists. Production leak validation (P2) is the B12 gate: real coach labels for precision, recurrence, calibration error, false paid-leak rate, and explicit V3 live-rank approval.

fail

Identification top-1 / top-3 (CQ-1)

Top-1 best cohort: 98.7% at 20 games, Top-3@5: 82.3%

Top-1 (Phase 1 gate, best cohort): 98.7% at 20 games / target ≥ 75.0%
Top-1 (Phase 2 gate): 98.7% / target ≥ 80.0%
Top-3 (≥5-game cohort): 82.3% at 5 games / target ≥ 90.0%
Trials recorded: 1,200 / target > 0

Source: sparse-cascade-benchmark.json, run May 15, 2026

pending

Repertoire matcher tree-match precision (CQ-2)

No repertoire-vector artifact published yet. Pending publication.

pass

AI prompt regression

4 fixture(s) checked across 4 prompt(s)

evidence_v1 • id: evidence_v1 / target = evidence_v1
evidence_v1 • version: 1.0.0 / target = 1.0.0
evidence_v1 • model: deepseek-chat / target = deepseek-chat
evidence_v1 • params.maxTokens: 120 / target = 120
evidence_v1 • params.temperature: 0.2 / target = 0.2
opening_identity_rerank_v1 • id: opening_identity_rerank_v1 / target = opening_identity_rerank_v1
opening_identity_rerank_v1 • version: 1.0.0 / target = 1.0.0
opening_identity_rerank_v1 • model: deepseek-chat / target = deepseek-chat
opening_identity_rerank_v1 • params.responseFormat: json_object / target = json_object
opening_identity_rerank_v1 • systemBlocks.0.label: evidence-discipline / target = evidence-discipline
prep_summary_v1 • id: prep_summary_v1 / target = prep_summary_v1
prep_summary_v1 • version: 1.0.0 / target = 1.0.0
prep_summary_v1 • model: deepseek-chat / target = deepseek-chat
prep_summary_v1 • description: present / target present
prep_summary_v1 • params.maxTokens: 800 / target = 800
prep_summary_v1 • params.temperature: 0.4 / target = 0.4
style_fingerprint_v1 • id: style_fingerprint_v1 / target = style_fingerprint_v1
style_fingerprint_v1 • version: 1.0.0 / target = 1.0.0
style_fingerprint_v1 • model: deepseek-chat / target = deepseek-chat
style_fingerprint_v1 • params.maxTokens: 250 / target = 250

pending

Production leak validation (P2)

No production leak-validation dataset found. Generate unlabeled leak fixtures in-network, collect coach labels, run leak-labels:validate, then promote production-validation.json with leak-labels:promote.

Engineering test harness (synthetic fixtures, not production measurements)

These leak checks use constructed scenarios with ground truth by construction. They validate scorer wiring only; they do not approve paid leak claims or a V3 live-rank flip.

passSynthetic fixture

Leak precision@5 (CQ-2)

SYNTHETIC (constructed scenarios, not production): Precision 93.3% across 12 complete top-5 opponents; lowest: synthetic:synthetic-opponent-03=80.0%, synthetic:synthetic-opponent-08=80.0%, synthetic:synthetic-opponent-10=80.0%

Aggregate precision@5: 93.3% (56/60) / target >= 80.0%
Complete top-5 opponents (synthetic): 12 / target >= 10
Precision artifact schema: 0 malformed / target 0 malformed top-5 rows or opponents
Artifact timestamp: valid / target parseable generated_at

Source: synthetic-leak-scenarios.json, run Jun 10, 2026

passSynthetic fixture

Leak recall sanity (CQ-2)

SYNTHETIC (constructed scenarios, not production): Recall 93.3% across 12 opponents; lowest: synthetic:synthetic-opponent-03=80.0%, synthetic:synthetic-opponent-08=80.0%, synthetic:synthetic-opponent-10=80.0%

Aggregate recall (known leaks in top-10): 93.3% (56/60) / target >= 70.0%
Labeled opponents (synthetic): 12 / target >= 10
Surfaced top-10 width: 0 oversized / target no opponent with >10 surfaced fingerprints
Unique recall fingerprints: 0 known duplicates, 0 surfaced duplicates / target no duplicate known or surfaced fingerprints
Recall artifact schema: 0 malformed / target 0 malformed opponents
Artifact timestamp: valid / target parseable generated_at

Source: synthetic-leak-scenarios.json, run Jun 10, 2026

passSynthetic fixture

Prep report latency P95 (CQ-2)

SYNTHETIC (compute-only timing, not the production API): n=12, P50 9 ms, P95 25 ms

P95 latency (scorer compute step only): 25 ms / target < 90.0 s
Sample count: 12 / target >= 5
Latency sample integrity: 0 malformed / target 0 malformed samples
Artifact timestamp: valid / target parseable generated_at

Source: synthetic-leak-scenarios.json, run Jun 10, 2026

Opening graph vs sparse fingerprint accuracy

Graph-first uses stored opening lines for 215,564 chess.com + Lichess handles, then sparse remains visible as the fallback/reference path.

Data refresh · last refresh 7/12/2026Sparse refresh · last refresh 7/12/2026

Graph quick scan

10 games

smallest sample size with top-10 accuracy >= 50%

Graph recommended

not met

smallest sample size with top-3 accuracy >= 70%

Graph high-confidence

not met

smallest sample size with top-1 accuracy >= 75%

Games	Sparse top 1	Graph top 1	Delta	Sparse top 3	Graph top 3	Sparse top 10	Graph top 10	Delta	Graph MRR
10	6.7%	46.7%	+40.0 pts	6.7%	46.7%	10.0%	50.0%	+40.0 pts	0.470

Benchmark accuracy measures whether the style matcher retrieves the right account. Final identity probability also includes AI Detective validation, which can demote public-identity contradictions.

Advanced technical notes

Artifact kind opening_graph_identity. Current graph read at 10 games: 46.7% top-1, 50.0% top-10. The graph-first matcher ranks candidates on their stored opening lines; the exact retrieval and scoring internals are intentionally not documented here.

Sparse fingerprint fallback diagnostic

Reference path for the same account-finding task. The headline above uses the opening graph when available; this table stays visible so we can compare against the older sparse fingerprint scorer.

Data refresh · last refresh 7/12/2026

Quick scan

not met

smallest sample size with top-10 accuracy >= 50%

Recommended

not met

smallest sample size with top-3 accuracy >= 70%

High-confidence

not met

smallest sample size with top-1 accuracy >= 75%

Games	Trials	Top 1	Top 3	Top 10	Median rank	MRR
10	30	6.7%	6.7%	10.0%	51	0.071

lichess

Games	Trials	Top 1	Top 3	Top 10
10	6	33.3%	33.3%	33.3%

chess.com

Games	Trials	Top 1	Top 3	Top 10
10	24	0.0%	0.0%	4.2%

Last full run took 3 min on 30 trials. Current read at 10 games: 6.7% top-1, 10.0% top-10. These figures measure ranking with the player already indexed, the easier variant of the task. Leave-N-out holdout figures replace them as the primary numbers automatically once an artifact publishes them.

Name-search coverage by FIDE tier

Club players matter as much as GMs. We benchmark coverage at every rating band, not just titled players.

Data refresh · last refresh 7/13/2026

Tier	FIDE pool	Claimed	Coverage	Progress
Titled (GM/IM/FM/CM/WGM/WIM/WFM/WCM)	24,231	12,765(li 1,396 · cc 12,437)	52.68%	66% of 80% v1 target
FIDE ≥ 2200	20,706	12,687(li 1,535 · cc 12,351)	61.27%	target met (40% v1)
FIDE 2000-2199	52,344	3,063(li 816 · cc 2,278)	5.85%	29% of 20% v1 target
FIDE 1800-1999	133,485	3,079(li 2,291 · cc 816)	2.31%	23% of 10% v1 target
FIDE 1400-1799	346,689	206(li 0 · cc 206)	0.06%	2% of 3% v1 target

Advanced technical notes

Coverage counts distinct verified players; a player claimed on more than one platform counts once. Denominators are active accounts above our rating floors on each platform.

Identity coverage by platform rating tier

Coverage is measured against the eligible accounts we have discovered on each platform, bucketed by rating band.

Data refresh · last refresh 7/13/2026

Platform	Rating tier	Target	Indexed	≥10 games	≥50 games	≥100 games	Coverage	Basis
chess.com	2600+	19,370	19,291	19,291	18,437	17,368	99.6%	frontier
chess.com	2400-2599	31,226	31,061	31,061	29,228	27,180	99.5%	frontier
chess.com	2200-2399	25,544	25,296	25,296	24,584	23,941	99.0%	frontier
chess.com	2000-2199	33,201	32,959	32,959	32,312	31,885	99.3%	frontier
chess.com	1900-1999	13,199	13,108	13,108	12,867	12,707	99.3%	frontier
chess.com	1800-1899	11,316	11,207	11,207	10,986	10,847	99.0%	frontier
chess.com	1700-1799	8,646	8,526	8,526	8,338	8,196	98.6%	frontier
chess.com	1600-1699	8,223	8,031	8,031	7,797	7,634	97.7%	frontier
chess.com	1500-1599	8,268	7,941	7,941	7,649	7,399	96.0%	frontier
chess.com	1400-1499	3,522	2,952	2,952	2,523	2,236	83.8%	frontier
chess.com	Exception / unknown rating	31,154	3,728	3,728	2,485	1,866	12.0%	frontier
lichess	2600+	1,107	1,107	1,107	39	33	100.0%	24m dump
lichess	2400-2599	2,761	2,761	2,761	105	88	100.0%	24m dump
lichess	2200-2399	8,997	8,997	8,997	423	336	100.0%	24m dump
lichess	2000-2199	28,835	28,835	28,835	1,269	1,020	100.0%	24m dump
lichess	1900-1999	32,877	32,877	32,877	1,729	1,129	100.0%	24m dump
lichess	1800-1899	55,711	55,711	55,711	2,861	1,666	100.0%	24m dump
lichess	1700-1799	7,692	7,692	7,692	861	366	100.0%	24m dump
lichess	<1700 proxy	5,111	5,111	5,111	1,120	794	100.0%	24m dump

Lichess target counts are measured from the processed 24-month monthly dumps. The rating bucket is still a stored rating proxy, so screenshot-style rank values remain useful as a future calibration source, not the current denominator source.

The ranker diagnostics above use production-strength retrieval settings. The visitor-flow holdout benchmark remains the public end-to-end claim.

Stage 4

Find their leaks by comparing their tree to yours.

A leak isn't a one-off mistake. It's a fork in the road you'll actually meet them at, where they pick the wrong path often enough to matter, and where you already know the right answer.

We rank every leak by five questions.

Will you actually meet on this road? A blunder in a line you never play is noise.
Is the mistake a sprained ankle or a broken leg? A queen-drop ranks far above a half-pawn slip; small inaccuracies never make the list.
Does it happen on move 8 or move 38? Early leaks beat late ones, because more games end before time-trouble does.
Do you already know the punishing move, or would you have to learn a new line? Cheap-to-prepare leaks rise to the top.
How many games back the pattern on each side? Once is an accident. A repeated pattern across their recent games is a leak.

Three flavours of leak: personalized (their weakness plus your prepared response, your highest-ROI drills), surprise (their weakness, you'd need to learn a new line to punish it), and own (you slip up, they can punish you, study these to defend).

How the ranking thinks

A proprietary scoring model blends all five questions, then discounts each leak by how much study it would cost you to exploit it, so the highest return-on-preparation rises to the top. The exact weights and thresholds are our secret sauce; the questions above are the honest summary of what they measure.

The leak report names the SAN line into the position, their bad move, the engine's preferred move, and which of your repertoire branches gets you there. See it on /prepare →