The repository
exaproof
Automated tracing and proofreading of neurons in exaSPIM whole-brain light microscopy.
Everything is documented at https://trace.scidex.ai/docs — the same files as below, rendered on request so a page cannot drift from what the project maintains, arranged by what you want:
| level | pages |
|---|---|
| use it | the tracing desk — every layer, button and key in the viewer (illustrated at /manual) |
| understand it | the models · the design · the data · this file |
| check it | what is settled · every experiment · how claims are checked · what is running |
| run it | every command · running an experiment · layout |
GOAL.md (repository file) is what the project is trying to do. This file is how to run the code.
What this is
Brain 653158 publishes 1,479,239 automated fragment SWCs next to 45 complete gold tracings and a fused 21605 × 28886 × 54049 OME-Zarr volume, all in one micrometre world frame. That combination — automated segmentation, ground truth, and imagery on the same brain — makes an end-to-end split/merge task the proposals believed did not exist, and it makes real decision-level supervision minable rather than synthetic.
exaproof uses it to reconstruct one neuron at a time from a soma seed: it grows through the
fragment graph, cuts suspected merges before it joins anything, resolves competing
reconnections as a global assignment under endpoint exclusivity, buys image evidence only where
that evidence can change the answer, and hands back a ranked, clustered worklist of what it
declined to decide — with a certified bound on how often the things it did decide are wrong.
Install
pip install -e . # the core; pyproject.toml is the authority on versions
pip install -e '.[torch,llm,viz,dev]' # the extras, each needed only by the commands named below
export EXAPROOF_DATA=~/data/exaproof # defaults to this anyway
The core needs numpy, scipy, scikit-learn, lightgbm, zarr, s3fs, fsspec, networkx, typer, rich, pyyaml
and tqdm. The extras are imported only when a command needs them: torch by anything that trains or
runs a network (the roi-* commands, grow, field-cache, encode, encode-pooled, bridge-train,
restore-train, and any command given --roi-model or a rescorer); llm by learn improve without
--offline; viz for matplotlib, by the commands that draw figures (the viewer itself runs on the
standard library); dev for pytest. smoke and simulate drive a headless Chromium through
Playwright (src/exaproof/viz/browser.py), which pyproject.toml does not list. The AWS CLI is used
for the bulk downloads, anonymously. What data fetch takes from s3://aind-open-data is CC-BY-4.0;
DATA.md §1 has the licence of every other source, and one of them is read but never redistributed.
Getting the data
exaproof data fetch --subject 653158 # gold tracings (23 MB) + fragments (2.0 GB)
exaproof data ingest --subject 653158 # -> columnar store + spatial index, ~1 min
exaproof data verify --subject 653158 # confirms one coordinate frame, against voxels
exaproof data status
data verify is not a formality. It samples gold nodes and measures image intensity against
the local background; on 653158 gold nodes sit at a median 16× background with 100 % above
1.5×, which is how the frame was established rather than assumed.
Seven subjects have fragments and gold and imagery — 653158, 653980, 648434, 609281, 789202,
685221 and 685222 (io.datasets.END_TO_END_SUBJECTS, the authoritative list; DATA.md §2f has what
each one holds) — so
brain-held-out evaluation uses real fragments, not injected errors.
Measuring the baseline
exaproof gt align --subject 653158 # project gold onto fragments (cached)
exaproof gt baseline --subject 653158 # the number to beat, with a tolerance sweep
The raw automated segmentation on 653158, 45 arbors, 8.09 m of cable, at 8 µm tolerance:
| value | published 794495 | |
|---|---|---|
| edge accuracy | 76.6 % | 73.78 % |
| normalized ERL | 0.0201 | 0.0095 |
| splits / neuron | 819 | 571 |
| merges / neuron (50-node threshold) | 11.3 | 6.9 |
| omitted edges | 13.6 % | 1.36 % |
The alignment tolerance is a reported parameter, never a hidden constant — gt baseline prints
the full sweep, because at 5 µm the same segmentation reads 21 % omitted and at 12 µm it reads
10 %, and neither number means anything without the other.
Mining supervision, and training
exaproof decisions mine --subject 653158 --tag v3
exaproof train --subject 653158 --tag v3 --name default --alpha 0.01 \
--image-neurons 6
decisions mine walks each gold tracing over the real fragments: where it crosses from
fragment A to fragment B, a human implicitly decided those two are the same neuron. Every other
candidate the proposer generates at that site and is not taken is a hard negative. On 653158
that is 116,677 labelled reconnection decisions — 40,444 positive, 76,233 negative —
over 24,709 true fragment adjacencies at 97.4 % proposal recall.
Proposal recall is printed because it is a hard ceiling: no classifier can join what was never offered. It started at 0.289 — the generator only proposed endpoint-to-endpoint joins, while a third of true adjacencies are endpoint-to-interior branch attachments that such a generator cannot express at all. That single fix moved the ceiling more than any model change could have.
train fits two heads — geometry-only for screening, geometry-plus-voxels for rescoring — and
certifies an operating point rather than tuning a threshold: given a tolerated false-accept
rate α, the accept threshold comes from a Clopper–Pearson bound on the fraction of accepted
decisions that are wrong. With ~570 reconnections per arbor (GOAL.md §1; this line said ~800
until 2026-09-19), a 1 % false-accept rate
manufactures as many merges as the raw segmentation already contains, so precision on accepts
is the constraint that matters and it is the one that gets a guarantee.
When the requested α cannot be certified, the run says so and reports the tightest α that can be. That is not a hypothetical: on 653158 geometry and priors alone top out at α = 0.061, against a problem that needs ~1 %.
Adding image evidence was the obvious way to close that, and a matched control — same
decisions, same split, differing only in whether the image columns are present — says it mostly
does not: the certifiable floor moves 0.0675 → 0.0638, about 5 % of the ~6× needed. Twelve
hand-designed patch statistics are not a substitute for a learned representation over the patch.
RESULTS.md §5–§7.
Proofreading a neuron
exaproof proofread N001 --subject 653158 --minutes 55
Writes reconstruction.swc, worklist.json (what it would not decide), journal.json (every
edit with its evidence, replayable and revocable) and result.json (per-round telemetry,
wall-clock, image reads).
Two strategies, selectable with --mode. sequential grows the neuron a frontier at a time:
detect and cut merges in the regions that changed → regenerate proposals there → score with
geometry → buy image evidence only inside the uncertainty band → accept under the risk bound →
defer the rest. global collects candidates over the whole neighbourhood without committing,
then solves one maximum-weight forest under endpoint exclusivity, acyclicity and a cable budget.
RESULTS.md §8 compares them: global buys precision at the cost of recall rather than beating
the trade-off, and both percolate below an accept threshold of ~0.3. Merges are cut before
splits are joined because ERL gives a merged label run length 0 at full weight: joining clean
fragments into a contaminated segment scores worse than doing nothing, which
exaproof bench exploits reproduces on demand.
Looking at it — the proofreading interface
This section is how to launch it, share it, and what it records. The per-control reference —
every layer, button and key, what to trust and the known gotchas — is
docs/VIEWER.md. Keep new UI detail there and launch/deployment detail here;
the two have drifted into parallel descriptions of one interface before.
exaproof view N001 --subject 653158 --port 9000 # reconstructs, then serves at :9000
With the calibrated chooser and the segmenter's field, which is the configuration the measurements support:
exaproof view N001 --subject 653158 --port 9000 \
--choice-model scratchpad/e184_model_from_648434_context.json --roi-model ms_100
Serves the gold tracing, the local fragments and the reconstruction as Neuroglancer
precomputed skeleton layers over the S3 imagery, draws every deferred decision as a line
between the two endpoints it is asking about, and puts a clustered, ranked worklist beside the
viewer with keyboard accept / reject / cut / skip.
The walker's fork questions. Record them by passing a recorder as grow_cable's ask (it answers nothing, so
the walk is unchanged), then add them to the list as fork rows; exaproof.viz.simulate.answer_forks answers them
from the gold over the same API:
from exaproof.roi.forks import ForkRecorder
rec = ForkRecorder(subject="653980", neuron="N001", walk="w0") # set rec.walk before each walk
grow_cable(image, est, seed, direction, ask=rec, ask_margin=0.05)
rec.save("scratchpad/forks_N001.json")
exaproof view N001 --subject 653980 --forks scratchpad/forks_N001.json # 1-9/n look, Enter answers, q none
Sharing it with other people. Add --auth and every route needs a signed-in account, and every logged decision
carries whose it is (src/exaproof/viz/auth.py):
exaproof users add ann@alleninstitute.org --name "Ann" # prompts for a password
exaproof users list
EXAPROOF_SIGNUP_CODE=$(openssl rand -hex 6) \
exaproof view N006 --subject 653158 --host 0.0.0.0 --port 9000 --auth ...
With EXAPROOF_SIGNUP_CODE set, people create their own password account with that code on the sign-in page;
without it, only exaproof users add and Google sign-in create accounts. Google sign-in appears when
GOOGLE_OAUTH_CLIENT_ID, GOOGLE_OAUTH_CLIENT_SECRET and GOOGLE_OAUTH_REDIRECT_URI
(https://<host>/login/google/callback) are set, and EXAPROOF_ALLOWED_EMAILS (a comma-separated list of addresses
or @domain suffixes) limits who may use it. Sessions are cookies over 30 days; /account shows a person their own
decisions and changes their password.
A public URL needs HTTPS, because the embedded Neuroglancer is served over HTTPS and browsers block its fetches
to an http:// source (localhost is the exception, which is why the tunnel below works). The deployment here is
nginx on sandbox-02 terminating TLS and proxying to this machine:
scratchpad/ holds the config as trace.scidex.ai.conf, live on sandbox-02 as
/etc/nginx/conf.d/trace.scidex.ai.conf since 2026-09-17. The viewer needs --host 0.0.0.0 so the proxy can reach
it, and it sets the session cookie Secure whenever X-Forwarded-Proto says the browser used HTTPS.
The viewer behind it runs as the systemd unit deploy/exaproof-viewer.service,
which restarts it after a crash, an earlyoom kill or a reboot (its header has the install commands). A 502 from
trace.scidex.ai means that unit is down: systemctl status exaproof-viewer, journalctl -u exaproof-viewer.
Change the neuron or flags in the unit, not by launching a second viewer on port 9000.
Before deploying a viewer change, run both gates against a viewer started from the branch on a spare port
(view N006 --subject 653158 --port 9017 --no-image --log-dir <scratch> is enough for layout): exaproof smoke
(every page, tab and tool works) and exaproof shots --url http://127.0.0.1:9017 --out <dir> (nothing overlaps,
clips or scrolls sideways at four screen sizes), and pytest tests -m browser (seven tests, ~30 s; the fast suite leaves
them out, and on 2026-10-15 one of them was the only check that saw /api/desk return 500 on a server with no brain
loaded, shipped a day earlier). Smoke cannot see layout: on 2026-10-15 a Help tab and a Demo link
had pushed the tab row 70 px past the side panel, and the whole panel scrolled sideways on the live site for two
deploys while smoke passed. A --no-image viewer cannot show what only imagery triggers: the blank-imagery warning
scrolled the live page 150 px sideways at 1024 px (now a browser test). After a deploy, shots the live site too —
and know that smoke there walks the subject switches and leaves the shared desk on its last brain (609281, whose
imagery is gone): reload 653158/N006 afterwards (sudo systemctl restart exaproof-viewer does).
Reaching it from your laptop. The server binds the host machine; open an SSH tunnel and use
localhost (the embedded viewer is HTTPS and the local skeleton sources are HTTP — browsers
allow that for localhost but block it for a bare public IP):
ssh -L 9000:localhost:9000 <host> # then open http://localhost:9000
The default bind, 127.0.0.1, is what the tunnel reaches; --host 0.0.0.0 is for the proxy above or
for another machine on the network. The imagery streams from public S3, so no credentials are needed
in the browser.
What you are looking at. Layers (toggle in Neuroglancer's top bar):
| layer | colour | meaning |
|---|---|---|
| exaSPIM | greyscale | the raw light-microscopy imagery (window adjustable in the shader) |
| reconstruction | orange | what automation has built for this neuron so far |
| gold | green | the reference tracing, when one exists (absent for a fresh seed) |
| fragments | grey | the local FFN fragments available to join |
| deferred decisions | yellow line | a join/gap the model would not commit — endpoint A↔B |
| suspected merges | red point | a place the model thinks two neurites were wrongly fused |
| tube tree default (rows 265/272), tube tree (row N) | blue / red / yellow | a gold-free tube-tree run of record (viz/tree_runs.py TREE_RUNS): its tree and attached pieces by node ownership at 8 µm — own, foreign, off-gold (pieces lighter). Off until switched on in the Gold tab, whose "show in 3D" button opens the whole neuron beside the gold with the run's recall@8 and own share |
Selecting a worklist item recentres the camera on that decision and rotates it so the gap lies across the view (both stubs and the dark stretch between them visible at once, not edge-on).
The order is the point. The worklist is ranked and re-ranks after every decision: a
suspected merge outranks a join (a false merge corrupts ~20× the cable); a path review outranks a
single join; larger un-worked clusters come first; decisions on longer fragments (more cable at
stake) and ones the model is least sure about float up; once you answer in a neighbourhood, its
remaining questions get a bonus so you finish it while its imagery is loaded; and with
--choice-model, each item is multiplied by the model's expected yield — its own probability
that there is anything there to join at all.
That last term is the one that is calibrated rather than tuned, and it is there because the others
answer the wrong question: they ask where an answer would be most informative, while an hour of
annotator time is spent on what is most productive. Measured over a session's scanned ends, and
after the barren-question rule has already fired, ordering by it roughly doubles the productive
share of the first hundred-odd questions (RESULTS.md §122). A caution that belongs next to it:
the same ordering inside the automated session does not reliably help — it reverses on one of
three brains (§125), because there an ordering also changes how much work exists. A worklist is
built once and only re-scored, so that failure mode does not apply here, but the two numbers are
about different things and should not be quoted for each other.
Deciding. Keys (mouse buttons mirror them):
| key | action |
|---|---|
j / k |
next / previous item |
a |
accept — the two endpoints are the same neurite; commit the join |
r |
reject — they are different neurites; leave them apart |
c |
cut — this is a merge; separate the fused neurites |
s |
skip — undecided, come back later |
1–9 |
on an item with a proposed chain, accept the first n hops; on an end with competing continuations and no chain, look at alternative n |
n |
with --choice-model, cycle the competing continuations at this end — each is framed along its own gap with the others still drawn |
Enter |
join the framed competitor |
q |
none of these continues it |
Competing continuations (--choice-model <fit.json>). A path review asks yes-or-no about the
model's single best chain. The decision you are actually making at a stalled end is which of
these, if any — so with a fitted choice model the item carries every nomination the scan made,
each with a calibrated probability, and P(none of these) on the same scale, because they come out
of one softmax over the set. All of them are drawn at once. That matters most on the common case:
roughly three quarters of the questions a session asks are barren, and the model's own answer to
"is there anything here" is the fastest way to dispatch one.
Two honest details you will see. If some competitors have since been joined into the reconstruction they are dropped and the header says so — the remaining probabilities then no longer sum to one and are deliberately not rescaled, because a rescaled softmax is a number that looks like the model's and is not. And opening an item re-scores it against the reconstruction as it stands, since an end with nothing to join early can have something later.
With --roi-model as well, the alternatives are ordered by the chooser's probability combined with
the segmenter's join level — the level at which its predicted field connects the end to each
candidate — which picks the right continuation more often on three brains (FINDINGS.md §4). Each
alternative shows "joins at 0.85" and its rank under the chooser alone; when the order changes, a line
under the header says why. When the chooser is already confident (top probability ≥ 0.65) the
prediction is skipped, because it never changed such a pick, and the panel says so.
A pick is logged as human_choice rather than human_path: confirming the model's own proposal and
overruling its ordering are different acts, and the second is the more informative. The log also
records the chooser's own rank of the pick.
The agent's checklist. With --playbook <file> (fitted by exaproof playbook-fit on another brain's
stall ends; it needs --choice-model), each end also shows what an annotation agent would do — nothing here, join, or needs a person — as a
numbered checklist: what it read, how sure it became, and the named reasons the case is hard (for example "two
candidates at 7 and 9 µm" or "three fibres run alongside within 12 µm"). Its verdict is logged beside yours. On
real stall ends it closes about a quarter as empty at 0.97 precision and does not yet join reliably
(FINDINGS.md §6), so treat its join suggestions as suggestions.
Check this branch. Most stall ends that lie off the neuron sit on long fragments the automatic segmentation
merged across two cells (FINDINGS.md §5). The check this branch button (or branch in the Ask tab) says how far
along its fragment an end is from where the reconstruction entered it, and marks in magenta the most merge-like places
on that stretch, which point 2–3× closer to where a foreign branch leaves than chance. It does not decide.
Tabs beside the worklist.
- Decide: the worklist and the tools above.
- Ask: type
help,explain,view,next,progress,conflicts,gold on/gold offoragent start/agent stop. The system answers these itself. Other questions are written toruns/<subject>_<neuron>/assistant_inbox.jsonl, where a Claude session watching the file can answer them (POST /api/answer); the page says a question is waiting until then. Comments (with tags such as crossing, faint or gold looks wrong) are saved with the decision and added to the run's hard-case registry. - History: every logged event, newest first, with the model's and the agent's numbers.
- Gold: switch the gold tracing and each annotator's own tracing on and off. It lists the places where two
annotators disagree, and where two gold neurons trace the same fibre (
gt.annotators), each with a button to go there. - Agent (needs
--playbook): the agent works the open stall ends in the background. It closes the ones it is sure are empty, and joins only if you allow it. The list can be ordered with the agent's least-sure decisions first, and your answers refit its calibration.
Why the system thinks this. Under the viewer, each item shows the evidence the decision loop
gathered (exaproof.adaptive): what it measured, what each measurement said, and when it declined to
look further. The look buttons render the same views a looking agent uses (plane, field of view,
contrast), and each look is logged beside the decision. The look and merge-check panels also say what the image
shows at that point, in one line with the numbers behind it (exaproof.adaptive.scene): part of the view outside
the imaged volume, a cell body, dense labelled neuropil, low contrast, or faint cable on the fragment itself.
Every human action is appended to runs/<subject>_<neuron>/decisions.jsonl with the model's
probability, epistemic/OOD uncertainty, cable-at-stake, and the fragments involved. That is
deliberate: light microscopy has no CAVE-style edit history, so capturing decisions from the tool
is the only mechanism by which the corpus grows past what can be mined from existing gold — and it
is what turns the simulated-human session numbers (VALIDATION.md §5) into real ones.
A timed row — one per answered question, written by /api/decision (yes or no on a deferred
join or merge), /api/choose (j: which competing continuation at a stalled end, or none) and
/api/path (k: how many hops of a proposed chain were accepted) — carries, beside index,
action, item and user:
| field | what it is |
|---|---|
dwell_ms |
wall clock from the item being opened to the answer. It keeps this meaning whatever else is logged; the fields below are subtracted by the analyst, never here |
away_ms, idle_ms |
of that: time with the tab hidden, and time with no pointer, key or wheel event for longer than 45 s |
score_wait_ms |
of that: time spent waiting on the server — from the request for the item's view to the later of the view being answered and the item's scoring (competitors, join levels, the agent's checklist) first becoming available. 0-ish when the background agent prepared the item before it was opened; null when the server never saw the item opened (a decision posted without a view). METHOD rule 33: a dwell read whole measures the machine's load that afternoon, so scratchpad/e176_measured_costs.py reports medians net of this beside the raw ones |
view_toggles |
how many times v (the 90° roll) was pressed on this item — the one difficulty signal the interface collects for free |
session_id |
one short id per sitting at one neuron, minted when the server starts and again when the picker switches neuron; every row, timed or not, carries it. Two sittings at one neuron share a file, and the first decision of each is orientation, not a decision |
overruled_decline and decline_p mark an answered question the skip rule had hidden; p_model,
p_none, model_rank, chooser_rank and agent sit beside a choice so the disagreement with the
model is on the row.
Switching neurons. The brain/neuron picker at the top of the worklist lets you start from
any neuron without relaunching: choose a subject and neuron and press Load — the server
reconstructs it in place (~½–2 min; the brain's fragment store and model are cached, so switches
after the first are quicker) and repopulates the worklist. exaproof precompute --subject <id> builds
those reconstructions ahead of time with the viewer's own settings, so a switch is instant. The catalog
lists only neurons that actually reconstruct (653158: 45, 653980: 21, 648434: 12, 609281: 25). The
neuron on the launch command is just the starting point.
Notes: --no-image skips only the server-side imagery reads; the browser still streams the imagery.
Evaluating
exaproof session --subject 653158 --model a10 --threshold 0.60
This is the only command denominated in what the one-hour target is actually about: human decisions per neuron. The system grows, stalls at a join it will not commit, asks one question, and continues. Watch it when changing anything — batch precision and recall can both improve while this does not.
--threshold -1 runs at the operating point the risk bound certified instead of 0.60; session --help
says why 0.60 is nonetheless the default. --inward and --propose-from-generated change which
questions exist, so a run with either is not comparable with stored arms without it — each says so in
its help. --skip-barren fits the barren-question skip rule on past session logs; certify its
threshold per brain with exaproof skip-certify first.
Add --long-radius 250 to let the human be asked about continuations across segmentation
omissions (automation never commits beyond the proposal radius; a directional cone gates the
wide scans). Over the dev split at threshold 0.60: median final coverage 0.495 (0.766
ungated), median coverage within a 60-minute human budget, benchmark cov60 (current value on
/leaderboard), which rose over the run before the frontier fix and long-range questions. The remaining gap is question targeting and automation
share — RESULTS.md §23 has the decomposition. The one-hour budget (415 joins in 120
questions) needs a leverage of 3.46.
exaproof eval --subject 653158 --split dev
exaproof bench exploits # the three metric exploits, reproduced as tests
exaproof bench splits # neuron-level splits and the held-out brains
eval runs the agent on held-out neurons and reports the full panel plus join precision and
recall against real gold adjacencies, deferred count, wall-clock, and image cost. The headline
is human decisions per neuron, not accuracy: the goal is one neuron per hour, and at ~30 s
of human attention per decision an hour buys ~120 of them.
eval also reports the number the goal is actually about: the ratio of unaided human minutes
to human minutes with the system, both counted the same way (sites collapsed per endpoint,
clustered into vignettes).
These are the first end-to-end measurements (
RESULTS.md§9); later work changed the picture substantially — seeFINDINGS.md§1.Median saving 0.97×, on every neuron tested, at every operating point tested. The system costs human time rather than saving it. Median join precision 0.556, median recall within reach 0.110; it removes the 13–52 joins it gets right and adds every site it defers at that was not a real join.
RESULTS.md§9.
The per-neuron binding constraint turns out to be reach, not precision: only 24 % of an
arbor's true joins are ever collected by breadth-first exploration, because connected growth from
a soma makes end-to-end recall the probability of an unbroken chain (q^k). A best-first
frontier is ~50× more efficient per fragment and holds precision at 0.73 instead of 0.016
(RESULTS.md §9.3) — and still does not survive the chain. The same model scores recall 0.545 at
precision 0.846 on the mined corpus and delivers r = 0.11 end to end.
The accounting closes in one inequality: with recall r, the best achievable saving is
1/(1−r). At the measured r = 0.11 the ceiling is 1.12×; reaching 60 minutes from a
102-minute baseline needs r ≥ 0.41, and the proposals' 120-decision budget needs r ≥ 0.71.
Recall cannot be bought by lowering the threshold, because the component percolates from
r ≈ 0.13. Both coordinates have to move, precision by two orders of magnitude.
RESULTS.md §9.3.
Proofreader-Time Equivalent is reported as a band across plausible coefficients rather than a point score, because no public exaSPIM timing calibration exists. Note the coefficients are not the 20× cable-damage ratio of a merge to a split — a merge may take one cut to fix, while chasing a dim terminal branch can take minutes. Metric damage and human cost are different quantities and the panel keeps both visible.
Learning from its own mistakes
exaproof learn mistakes --subject 653158
exaproof learn improve --subject 653158 --rounds 1
mistakes clusters the system's false accepts and false rejects in feature space and describes
what distinguishes each cluster. improve hands those clusters, the feature registry and the
performance table to Claude, which writes new feature functions; each one is statically
screened, sandboxed, validated on a real batch, rejected if it duplicates an existing column,
then measured — promoted only if it improves the dev split and survives the validation
split. Every attempt lands in a ledger with its measured effect, accepted or not.
--offline swaps the LLM for a built-in catalogue of physically-motivated candidates, so the
acceptance machinery — which is where the discipline lives — runs without credentials.
The model is not the classifier here, and that is on evidence: ConnectomeBench measured zero-shot frontier models at 61–63 % on merge identification, below a fine-tuned ResNet-50. Its job is reading a failure description and writing a candidate explanation as code. Whether that explanation is right is decided by measurement.
The agentic tracer: tracing whole neurons from the soma
exaproof view N006 --subject 653158 --from-scratch # the soma's fragment alone; every join is made on the list
exaproof agentic bench --out DIR --arms person,rules+desk+person --reach all --truth-reach-um 250 \
--neurons "653158:N006,N007" # trace from the soma under each tier stack, scored against gold
exaproof agentic fit-desk --corpus DIR # the desk model, one per brain with that brain held out
exaproof agentic report DIR # the arm x brain table
exaproof agentic replay DIR/person/653158_N006.json --page demo.html # a 3D replay (also served at /demo)
exaproof shots --url http://127.0.0.1:9000 --out /tmp/shots # the viewer at four screen sizes, layout faults measured
exaproof.agentic works the viewer's own list through the same answer methods a person's keys call, deciding each
question with a stack of tiers, cheapest first, each deciding only what it is sure of: named rules; the
desk model (three calibrated gradient-boosted heads — P(nothing to join here), P(this candidate), P(this hop) —
fitted on gold-answered questions from other brains); the ROI tier (refresh_alternatives: the multiscale
ms_100 field rerank and the playbook's "nothing here"); Claude (claude -p or the Anthropic SDK, batched, the
dossier in words plus the desk model's readings, escalating when unsure); and finally the person (live) or the
gold oracle at the harness's prices (benchmark). When the list runs dry, reach sources propose what the list
cannot: attachments from outside fragments (the session's inward pass, every place offered), long questions at dead
ends (250 µm, a 90° cone about the end's heading, since §720) and long attachments; --walk adds the ROI walker on the dying frontier. Every step records
the dossier, the verdict with its tier, rule, confidence and rationale, the gold's answer and the standing against
the gold, which is also the replay.
Live, the Agent tab runs the same stack beside the person (/api/agentic): it never takes the decision the person
has open, leaves what it is not sure of on the list marked agent left this for you with its reason, logs every
decision as an agentic_decision row, and waits for the person's answers before carrying on. Use view --log-dir for
any test or demo viewer, so its rows stay out of the people's decision corpus.
Measured (registered, from the soma, the gold as the person, 133 neurons of four brains no desk model saw): closing at
the desk model's 0.90 bar takes 97 % of the person's minutes and a median 0.146 of each arbor (§712); holding those
questions for last instead keeps the arbor (Δfinal 0.000), leaves cov@60 where it was (+0.0005), and where it reaches
the person-alone coverage (109 of 133) gets there in a median 0.77 of the minutes (§715) — and a person who loads a
neuron from the soma gets that order on the list itself (held questions at the back, badged; the list refills
from the reach sources when it runs dry; agentic bench --person-desk traces on that desk); the walker on stalled traces
reaches nothing new (§713). So every stack the Agent tab offers holds — the desk model (deskhold), the segmenter's
checklist (roihold) and Claude (llmhold) — and none runs the rules tier, which made all 7 agent wrong joins in §715
from 40 decisions in 190 neurons. rules+desk+roi+gold stays on the tab, labelled, as the closing comparison. The live
desk model is <data>/agentic/models/desk_e685.pkl (desk_model.DEPLOYED), the set §712 and §715 measured.
The scoreboard: auto@soma, the metric of record. exaproof agentic score traces every gold neuron from its soma
with nobody answering (default arm desk+auto: the desk tier joins at P ≥ 0.95, auto leaves everything else;
desk models held out of each brain; the bench's flags), then scores each trace (agentic.scoreboard). It reports
the share of the gold arbor in the final soma-connected reconstruction, with its precision (wrong joins, cable off the
gold). It splits the whole arbor into joined / reachable but not joined / beyond 250 µm gaps / unsegmented, by
compartment (the gold's node types), by distance from the soma, and by CCF region where the brain has a registration
(only 653158's is public). The headline is the worst brain's median. Each run writes scoreboard/<commit>.json and
appends a line to scoreboard/history.jsonl with the change since the last scored commit, which is the Measure step
of the loop in AGENTS.md §1. Run it from a worktree at origin/main, so the commit it names is on main, and commit
what it writes. --no-run --out DIR re-scores traces already on disk.
The knowledge base: priors, heuristics, context, the wiki, the literature
What should inform a tracing decision is kept in one PostgreSQL database with pgvector (exaproof, on this machine's
postgresql@16-main; spec docs/KNOWLEDGE.md). It holds:
- entries: priors, heuristics, contextual heuristics, context, prompts and cases;
- wiki pages;
- papers;
- the cell-type databases and atlases;
- the tool registry: every model, scanner, tracer and matcher a tracer can call at a decision, with where it is called, what it costs and what it was measured to be worth (banked where it was found not to help), scoped to the assembly situations it serves.
Each is an item with a stable id, a scope (brains, CCF regions, compartments, cell types, decision kinds), a source, evidence and a status (proposed → measured → adopted, or banked with the reason). Every save is a revision.
On the site, /kb lists and searches everything, and /wiki, /lit, /types and /tools narrow it. /kb/<id> shows an item
with its links both ways, its revisions with diffs, and its comments. Anyone signed in, except a share-link guest,
edits there; an admin, or an attached measurement, adopts.
exaproof kb init # tables and the vector extension, where missing (idempotent)
exaproof kb seed # 150 verified papers, 8 sources, the LLM prompt, CLOSED.md banked, census priors,
# the named rules, a page per brain and region, the tool registry; numbers computed
# from the censuses, never typed
exaproof kb seed --only tool # one group (here the tool registry), without the minutes-long region census
exaproof kb export # the git snapshot, knowledge/*.yaml (current revisions); `kb import` rebuilds from it
exaproof kb ls --kind prior -q gap ; exaproof kb show prior-arbor-split-from-soma
exaproof kb records-bench <data>/agentic/e688_H # every agent decision of a bench run, joined to the gold
exaproof kb records-desk # every answer a person gave on the desk, with their reasons
exaproof kb records # what the reasoning records hold (also /kb/records on the site)
The database is dumped hourly by Reprise and nightly by scratchpad/backup_s3.sh (full history; what, where and
how to restore: docs/BACKUPS.md (repository file)). Tests use a throwaway
schema (tests/test_knowledge.py, marker postgres, skipped where the database does not answer).
Tracing from raw imagery
exaproof trace N001 --subject 653158 --seconds 120
The "from scratch, no fragments" experiment, scoped honestly: whole-arbor from-scratch tracing on CPU will not beat the published fragments, and the design says so up front. What the same machinery is good for is local — a minimal-path bottleneck across a 25 µm gap is among the strongest evidence available to the split classifier, and 10-54 % of gold cable has no fragment (§193, per brain; this line said ~13 % from a single early brain) within tolerance at all, which no amount of reconnection can reach. Results report coverage and precision, because a tracer that wanders scores well on one and badly on the other.
The benchmark that decides the goal
exaproof train --group # endpoint-level head, beside the pairwise top-1 it must beat
The one number that governs whether unattended tracing is possible: recall at the precision a
false merge tolerates. A merge fuses two arbors, so a ~400-join arbor needs precision ~0.998;
today's recall there is under 1 %. Any new feature, model, or heuristic should be judged on this
column first — batch precision and recall can both improve while it does not. RESULTS.md §15
records the endpoint-head experiment that produced it, and why group context did not help.
exaproof encode # learned encoder over raw gap voxels (matched control)
exaproof ceiling # how often do two experts agree? the benchmark's own limit
exaproof repair # can a merge be found again after it is made?
exaproof triage # can a human find committed merges by reviewing the doubtful end?
exaproof reach # the hard ceiling on connected growth: soma-reachable cable vs jump range
exaproof audit # are the mined negatives actually negative?
Before trusting any ceiling this system reports, check the labels it was measured against.
exaproof audit finds every negative pairing two fragments from the same arbor and asks whether
they are adjacent on the tracing with nothing between them. On 653158 the answer is zero out of
5,938 — the negatives are distant branches of one cell, median separation ~230 µm. RESULTS.md
§16 records it, including the shortcut bug that first made the labels look broken.
See DATA.md for every data source with provenance and regeneration commands. Where each
line of work stands is exaproof goals, not a document; ROADMAP.md holds only the
current session's handoff — what is running and what to do with each verdict.
Extending it
Adding a feature is one decorated function:
from exaproof.features import CandidateBatch, feature
@feature("my_signal", group="geometry", cost="cheap",
outputs=("my_a", "my_b"), description="what it is for")
def my_signal(b: CandidateBatch):
return np.stack([..., ...], axis=1)
It is picked up automatically by the model, the feature cache, exaproof features list, and
the learning loop. The cost class is load-bearing, not documentation: cheap and prior
features are computed for every candidate, image and expensive ones only where the agent
decides the evidence is worth buying, and an uncomputed feature is NaN — missing, not zero —
which the gradient-boosted default handles natively.
Other extension points, all behind the same interfaces: models/base.py (Estimator) for a
different classifier, agent/merge.py for merge detectors, candidates/conflict.py for the
assignment objective, learn/propose.py (Proposer) for a different proposal source.
Every command
Generated from exaproof --help and each command's own --help on 2026-09-21: the command, the first
sentence of its help, and where it is explained. Seventy commands — 47 at the top level and 23 under
eight sub-apps. Flags are not repeated here: exaproof <command> --help is their reference, and the
flag text there carries the measurement each default rests on, so it cannot drift from the code.
Top level, in the order --help lists them:
| command | what it does | explained in |
|---|---|---|
train |
Fit the decision model and certify its operating point. | training; MODELS.md §2 |
proofread |
Reconstruct one neuron and emit a ranked worklist of what it could not decide. | proofreading |
eval |
Run the agent on held-out gold neurons and report the full panel. | evaluating |
smoke |
The release gate: drive every page, tab, tool and subject switch in a browser and list the blockers. | the viewer; viz/smoke.py |
simulate |
Work through decisions in a real browser the way a person does, and report what the workflow costs. | GOAL.md (repository file) §5 G; viz/simulate.py |
field-cache |
Precompute the model's probability field along one neuron's cable, for the viewer to serve instantly. | VIEWER.md §4, paint: field |
precompute |
Build the viewer's reconstructions ahead of time, so opening any of these neurons is instant. | the viewer |
view |
Launch the Neuroglancer proofreading viewer for one neuron. | the viewer; VIEWER.md |
shots |
Screenshots of the viewer at four screen sizes, with the layout faults measured in each. | the agentic tracer; viz/shots.py |
agentic bench / report / fit-desk / replay |
Trace neurons from the soma under tier stacks and score them; the table; the desk model; 3D replays. | the agentic tracer |
audit |
Check the mined negatives against the tracing they came from. | the benchmark |
review-report |
Turn a proofreader decisions.jsonl into annotator metrics — the real-human measurement. |
VALIDATION.md §5 |
asker |
Train the does-this-end-continue head that ranks the session's questions. | models/continuation.py; session --asker |
reach |
The hard ceiling on connected growth: soma-reachable cable vs allowed jump distance. | the benchmark; FINDINGS.md §1 |
bridge |
Can the imagery bridge an omission gap the fragments cannot? | GOAL.md (repository file) §5 C; bench/bridge.py |
bridge-corpus |
Slab corpus for the gap verifier: per mined gap, the nominated decoys plus the true re-entry pair, labelled. | MODELS.md §6; bench/bridge.py |
bridge-train |
Fine-tune the certified encoder into a gap verifier on the mined-pair corpus. | MODELS.md §6 |
roi-corpus |
ROI training cubes for the local segmentation model. | MODELS.md §1; FINDINGS.md §2 |
synth-corpus |
Render CORAL geometry through this instrument's measured optics into a training corpus. | GOAL.md (repository file) §5 E; synth/ |
roi-train |
Train the ROI U-Net and measure what matters: gap connectivity vs decoy connectivity. | MODELS.md §1; FINDINGS.md §2 |
restore-train |
Train the restoration U-Net (E052 Stage 1) on bright corpus cubes. | E052_restoration.md (repository file) |
adaptive-eval |
Run the adaptive decision loop over a holdout and score it. | FINDINGS.md §6; adaptive/ |
merge-weights |
Fit the merge-window score's weights on contact-centred pairs, for the viewer's scanner. | MODELS.md §4; FINDINGS.md §5 |
look-session |
Build a blind session for a looking agent: shuffled sites in a manifest, answers sealed. | FINDINGS.md §6; adaptive/looks.py |
look |
Render views of sites in a blind session, and record that they were looked at. | FINDINGS.md §6 |
look-verdict |
Record a looker's verdict and its reasoning. | FINDINGS.md §6 |
look-score |
Unseal a blind session: the looker against the model on the same sites, and the traces. | FINDINGS.md §6 |
adaptive-extend |
Ask stored traces a new question: rebuild their sites and append probes, verifying the match. | adaptive/harness.py |
scene-calibrate |
Measure a brain's own thresholds for the viewer's image line. | the viewer; adaptive/scene.py |
playbook-fit |
Fit the stall-end playbook's per-step calibrators and action thresholds, and save them. | MODELS.md §7; the viewer |
adaptive-heuristics |
Learn readable rules from recorded traces, and say what readability costs. | FINDINGS.md §6; view --heuristics |
roi-panel |
Re-panel a trained ROI model on a chosen threshold grid, without retraining it. | MODELS.md §1; roi/eval.py |
roi-compartments |
Add compartment labels (unknown, axon, dendrite) to an existing ROI corpus. | roi/corpus.py |
roi-seeds |
Add the seed channel to an existing ROI corpus: the entry object rasterised. | roi/corpus.py |
roi-objlabels |
Object-conditioned labels over an existing corpus's saved centres. | roi/corpus.py |
roi-mine |
Cubes at the training gaps a model fails to connect, as a new mining round. | roi/corpus.py |
triage |
If merges cannot be prevented, can a human find them cheaply? | the benchmark |
encode |
Train the learned encoder over raw gap voxels, against geometry as a matched control. | the benchmark; MODELS.md §6 |
encode-pooled |
Train one encoder on every brain's slabs and save a deployable rescorer per brain. | MODELS.md §6 |
endpoint-bench |
Endpoint-level recall at tight precision, cheap vs fused. | the benchmark |
disputes |
One evidence dossier per join the annotators disagreed about. | GOAL.md (repository file) §5 H; VIEWER.md §6 |
ceiling |
How often do two experts agree on the same join? The benchmark's own ceiling. | the benchmark |
repair |
Fuse two gold arbors with a real merge and try to find the bad edge again. | the benchmark |
session |
Simulate a proofreading session and report the goal's own metric. | evaluating; METHOD.md |
trace |
Trace from the raw imagery alone, with no fragments, and score it against gold. | tracing |
grow |
Generate cable where the segmentation produced none, from stalled fragment ends. | FINDINGS.md §9; GOAL.md (repository file) §5 B |
skip-certify |
Price a barren-question skip threshold for one brain, on both metrics. | evaluating; GOAL.md (repository file) §5 A |
goals |
Where every goal stands, checked against the records it cites. | AGENTS.md (repository file) §2, §5 |
record |
Read one result out of the record without opening the record. | AGENTS.md (repository file) §3, §5; docs/INDEX.md |
| exaproof status | Read the leaderboard (--path alt file). Per-brain recall, TBS F1, benchmarks. | AGENTS.md (repository file) §2, §5; exaproof goals |
data — fetch and ingest exaSPIM data:
| command | what it does | explained in |
|---|---|---|
data status |
What is present locally, per subject. | the data; DATA.md §2 |
data fetch |
Download the gold tracings and the automated fragment archive for a subject. | the data; DATA.md §1 |
data ingest |
Flatten the fragment archive into the columnar store and build spatial indices. | the data |
data inventory |
What exists for every named brain — imagery, tracings, segmentation, somas — from the written indexes. | DATA.md §2f |
data index-s3 |
Write a depth-limited listing of a bucket to a file, so later searches are local. | DATA.md §1; io/s3_index.py |
data verify |
Check that gold tracings, fragments and imagery share one coordinate frame. | the data |
gt — gold-to-fragment alignment and the raw baseline:
| command | what it does | explained in |
|---|---|---|
gt align |
Project every gold arbor onto the fragment set and cache the result. | the baseline |
gt baseline |
The raw automated segmentation scored against gold — the number to beat. | the baseline |
docs — compaction docs, read-only audit / lint:
| command | what it does | explained in |
|---|---|---|
docs audit |
Audit docs (memory, KB, links); write TSV + audit_summary.md. --repo . --out |
docs/METHOD.md; AGENTS.md (repository file) §5 |
docs lint |
Report bare numbers in prose (SPEC_COMPACTION §6), never edits. Pass files... |
docs/METHOD.md §6 |
decisions — mine labelled reconnection decisions:
| command | what it does | explained in |
|---|---|---|
decisions mine |
Mine labelled reconnection decisions from gold tracings over the real fragments. | training |
decisions summary |
Describe a mined corpus. | training |
decisions augment |
Fold session ask logs into a decision corpus — the retrain-on-live-mistakes loop. | gt/session_supervision.py |
features — inspect the feature registry:
| command | what it does | explained in |
|---|---|---|
features add-image |
Add image evidence to an existing feature cache, for more neurons. | training |
features list |
Every registered feature, its cost class, and what it is for. | extending |
learn — learn from the system's own mistakes:
| command | what it does | explained in |
|---|---|---|
learn mistakes |
Cluster the system's own errors and describe what distinguishes each cluster. | learning; FINDINGS.md §9d |
learn improve |
Propose new features from the mistake report, measure them, keep what helps. | learning; FINDINGS.md §9d |
users — accounts for the proofreading viewer (view --auth):
| command | what it does | explained in |
|---|---|---|
users add |
Create a viewer account. | the viewer; VIEWER.md §9 |
users list |
Every viewer account. | VIEWER.md §9 |
users password |
Set or reset an account's password. | VIEWER.md §9 |
users reset |
Issue a one-time link with which someone chooses a new password themselves. | VIEWER.md §9 |
users remove |
Delete an account (its logged decisions keep the email they were recorded with). | VIEWER.md §9 |
viz — serve-side pieces of the proofreading viewer:
| command | what it does | explained in |
|---|---|---|
viz fetch-ng |
Mirror the Neuroglancer client so view can serve it from its own origin. |
VIEWER.md; view --ng-base |
bench — benchmark utilities and adversarial checks:
| command | what it does | explained in |
|---|---|---|
bench exploits |
Reproduce the three known metric exploits, so a harness change cannot reopen one. | evaluating; bench/exploits.py |
bench splits |
Show the neuron-level train/calib/dev/test split, and the brains available held out. | evaluating; METHOD.md §2 |
Tools (PYTHONPATH=src; python tools/X.py --help): bench_freeze freezes a benchmark (--id, --items, --md5, --dry-run); brief writes a brief (new); ledger_split splits ledgers (results|rows); memory_lint lint (dir); viewer_gates deploy gates (--dry-run, --port).
Layout
One line per package, from each package's own docstring; ls src/exaproof is the check.
src/exaproof/
cli.py every command above; `exaproof --help` is its index
config.py paths and run configuration; `EXAPROOF_DATA` is the data root
goals.py `exaproof goals`: goals.yaml validated against the records it cites
record.py `exaproof record`: retrieval over RESULTS.md and VALIDATION.md without opening them
io/ SWC, fragments, gold, OME-Zarr imagery (`EXAPROOF_VOLUME`), S3 indexes, the brain inventory, somas, MouseLight, CCF
manifests/ the data owners' location lists, checked in as received (its README says what each maps)
imagery_meta/ each volume's zarr metadata, kept here because the sources have lost theirs (its README)
gt/ gold->fragment alignment; the decision miner; annotator agreement, disputes, gaps, session supervision
metrics/ the panel (ERL, edge accuracy, merges at 0/10/50 nodes)
features/ the registry: geometry, context, image, priors, slabs, rays, texture, tracts, calibre
models/ estimator, calibration, conformal risk control, OOD; the choice, continuation, rescoring and encoder heads (docs/MODELS.md)
candidates/ proposal generation, conflict graph, exact assignment
agent/ journalled state, merge detection, the sequential and global loops, the result cache
bench/ end-to-end evaluation, the simulated annotator and session, reach, repair, triage, bridge, exploits
roi/ the local segmenter: corpus, training, tiled prediction, panels, flood tracer, growth, restoration, field cache
synth/ rendered training cubes: known geometry through the instrument's measured optics and noise
adaptive/ per-decision evidence loops, traces, rendered looks, the playbook, merge scan, scene line, branch check
viz/ precomputed skeletons, Neuroglancer state, the proofreading server, accounts, the docs site, browser gates
learn/ mistake mining, LLM proposals, sandbox, acceptance loop
agentic/ the agentic tracer: dossiers, tiers (rules, desk model, ROI, Claude, person), reach and walker sources, live worker, bench, replays
tracing/ minimal paths and the from-scratch tracer
Tests
PYTHONPATH=src pytest tests -m "not data and not browser and not heavy and not server and not node" # ESSENTIAL suite (default, tests/essential_suite.toml; docs/TESTING.md); add --suite extended|all for the deferred rest
PYTHONPATH=src pytest tests # everything: ~2 min, ~6 GB peak
Markers are registered in pyproject.toml with --strict-markers: data (reads real data under
/home/ubuntu/data), torch, browser (playwright/Chromium), node, server (a live viewer on an
ephemeral port), heavy (over 2 GB or 30 s alone), slow. The essential suite is what to run beside an
experiment; the full suite is itself a heavy job on this machine (AGENTS.md section 6).
In a worktree, PYTHONPATH=src is not optional. The editable install points at the main
checkout's src/, so a bare pytest in a worktree runs that worktree's tests against the main
checkout's code: a change you just made is neither exercised nor able to fail the suite, and a
green run means nothing about it. Found 2026-09-19, when a new server method passed review and
then raised AttributeError under the test that was supposed to cover it.
Concentrated on the traps that would otherwise be invisible in the final numbers: the SWC
OFFSET convention, point-to-segment distance (fragment nodes are ~10 µm apart, so nearest
node is not nearest fragment), merge-threshold semantics, exactness of the assignment solver
against brute force, the risk bound holding, and the sandbox refusing dangerous code.
Full measured results, including every bug that looked like a model failure and was not, are
in RESULTS.md — reached with exaproof record <n>, not opened.
Running an experiment
Each rule below has cost a run; AGENTS.md (repository file) §6 has the incidents and
docs/METHOD.md the rules for the measurement itself.
- Work in a worktree under
~/worktrees/agentic_proofreading/, never onmain. A worktree has no.venv, so every run and every test there isPYTHONPATH=src /home/ubuntu/git/agentic_proofreading/.venv/bin/python -m exaproof.cli …(or-m pytest); a bareexaproofruns the main checkout's source. A job launched from a worktree writes its outputs there. - Set
EXAPROOF_VOLUME=denoised. The rawfused.zarris gone upstream for several brains and reads as zeros rather than failing (DATA.md§2b).io/imagery.pyreads the variable; the driver below exports it. - Gate on memory, not on the GPU. RAM is the binding resource and an OOM kill leaves no traceback.
scratchpad/mem_gate.sh(sourced) providesmem_ok [GB], which admits when at least GB is available and memory pressure was low over the last minute — or over the last ten seconds with swap-in quiet — andheavy_running, which counts running jobs matching itsHEAVY_SCRIPTSlist: add a heavy script there when you write it.scratchpad/slots.shis the older slot gate for parallelroi-train. - Launch through
tools/launch_run.sh(every run is capped, logged and tracked in OpenResearch,orx runs <project>); a queue is one such run whose command is the driver, detached:tools/launch_run.sh run --prereg ... --tag ... --minutes N --mem-gb N -- 'bash scratchpad/run_queued.sh <exp> <script.py> [--after-pid PID] [--marker FILE] [--gb N] [--max-heavy N]'. It waits for the PID or the marker, then formem_okand fewer than N heavy jobs, runs the script with its output inscratchpad/<exp>_read.out, and records every phase inscratchpad/<exp>_queue.state.json. Its header is the full contract; its own PID is what the next queue passes to--after-pid. - See what is running with
pgrep -af 'bash scratchpad/'and.venv/bin/python scratchpad/queue_state.py, which reads those state files. Never wait onpgrep -f PATTERNin a loop: it matches itself. - A scratchpad script that needs a sibling loads it with
import _load; m = _load.load("e618_walk_close")(scratchpad/_load.py): once per process, with this checkout'ssrcon the path. - When it lands, follow
AGENTS.md§5: aRESULTS.mdsection,goals.yamlin the same commit, and the outputs the section rests on committed by name.
Limits, stated rather than buried
- One GPU. The development host has a single NVIDIA L4 (23 GB) and 30 GB of RAM; two segmenter trainings at once saturate it.
- 10–54 % of gold cable has no fragment within tolerance, depending on the brain (10 % on
653158, 54 % on 648434). That is a ceiling on any fragment-based method, and it is why the local
segmenter and
tracing/exist. - Coordinates are 3× expanded. exaSPIM images expanded tissue, so every micrometre in this repository is an expanded micrometre. External data in tissue units must be scaled first.
- Ground truth is uncertain to ~9 % on axons. Two experts tracing the same exaSPIM axon differ by 8.9 % of cable on average, 16.6 % at worst. No improvement smaller than that against a single annotator's tracing is meaningful, and the harness reports cluster-bootstrap intervals with the neuron as the resampling unit for exactly this reason.
- PTE is uncalibrated. Reported as a band. Calibrating it needs instrumented proofreading sessions, which the viewer's decision log is designed to produce.
- Three gold
consensusfiles in 653158 are in the wrong coordinate frame (N016, N036, N049 — offsets tens of millimetres outside the volume). The loader detects and reports this and falls back to the per-annotator file; it is a data issue worth reporting upstream.