← Exaproof  ·  all docs  ·  understand it  ·  DATA.md

The data

Data sources

Everything this project reads or produces, with provenance and how to regenerate it. The machine-readable registry is src/exaproof/io/datasets.py (KNOWN); this file is the prose account. Written for the move to a new machine — the verification commands at the bottom are what "the data survived the copy" means.

Read §2f Table A first, and treat it as the only inventory. This file grew by appending a dated block each time the buckets changed under us, so its sections run 1, 2, 2b, 2e, 2f, 2c, 2d, 3 — and one of them is marked "SUPERSEDED the same day" by a block below it. §2f, "The full inventory", is the reconciled state: every brain we have a name for, what exists for it, and which names do not resolve. The blocks before it are the discoveries that produced it, kept for their provenance, and any of them may be a snapshot of a state that lasted hours.

This is not a style complaint. A stale row in an earlier block hid a whole brain from an analysis for the life of that analysis — 609281 was listed as "gold only" while its store held 737,085 fragments — and four RESULTS sections carried a caveat about a missing labelling cache that was never missing. If a row here disagrees with §2f, §2f wins; if you are about to write "brain X has no Y", check the bucket rather than this file.

The live check is exaproof data status, and the search commands are exaproof data index-s3.

1. Remote sources

s3://aind-open-data — the public bucket, and the only one anything is redistributed from

AWS Open Data Program, anonymous read (no credentials needed; the code reads via fsspec/s3fs with anon=True). Three kinds of object per exaSPIM subject:

kind key pattern format
imagery <fusion_prefix>/fused.zarr OME-Zarr (zarr v2), 5D TCZYX, 7-level pyramid, blosc-zstd, 256³ chunks; level 0 at 0.748 × 0.748 × 1.0 µm/voxel
gold tracings <reconstructions_prefix>/… SWC, µm, two layouts (below)
automated fragments <fusion_prefix>/seg/<seg_key> zip of SWC skeleton fragments from FFN (653158: 1.48 M fragments)

Subject registry (from KNOWN; prefixes abbreviated — full strings live in the code).

This table and Table A in §2f disagree in four rows, and Table A is the later one. The imagery column here predates the upstream deletions and still reads ✓ for 653158, 653980 and 648434, where only the denoised twin reads (§2b); 609281 reads "stitched only" here and "nothing readable" in §2b; 721332 is "—" here and "raw" in Table A, and the code agrees with this table (live_for_subject raises has no fused volume in s3); and 685221's archive is 7.4 GB here against 22 MB in Table A, a factor of 336 that nobody has resolved — do not quote either without checking the bucket. Left as a disagreement rather than silently picked, which is the convention for everything else in this repository.

subject imagery gold arbors fragments archive end-to-end?
653158 ✓ 45 (Complete_raw) seg.zip (2.0 GB) ✓
653980 ✓ 21 (Complete_raw) swcs-processed.zip ✓
648434 ✓ 12 (specimen_space_reconstructions) 648434_swcs.zip (283 MB) ✓
609281 stitched only 25 (Complete_raw) seg.zip → 737,085 fragments ingested ✓
685222 ✓ 38 (specimen_space) corrected-swcs.zip 4.1 GB (other bucket) ✓ 2026-09-18, coverage 0.816
789202 ✓ 14 (gold/, other bucket) original_swcs.zip 305 MB (other bucket) ✓ 2026-09-18, coverage 0.921
685221 ✓ 60 (specimen_space) corrected-processed-swcs-s3.zip 7.4 GB (other bucket) ✓ 2026-09-18, coverage 0.889
651324 ✓ 13 (Complete_raw) — imagery+gold only
721332 — 22 (specimen_space) — gold only

Seven subjects support the split/merge task (END_TO_END_SUBJECTS); 685221, the seventh, finished aligning at 19:59 PDT on 2026-09-18. Three of them were found by widening a search rather than by any measurement, and all three had been recorded here as unusable:

609281's row said "gold only" and was wrong for the life of the project. Its fragment store was ingested from raw/609281/seg.zip — 737,085 fragments, 41.9 M nodes — and meta.json had said so since. The table was never updated, so a fourth end-to-end brain sat unused until §226 went looking. It is also the hardest of the four: 41.7 % of its gold cable has no fragment, the highest pendant share (44.1 %) and the fewest simple spans (17.9 %).

685221 and 685222 were wrong the same way, for a different reason. Both were verified on 2026-09-14 as having no fragment store and no seg archive — in aind-open-data. Both have one in aind-msma-morphology-data (7.4 GB and 4.1 GB). With 60 and 38 gold arbors they are the largest tracing sets in the corpus after 653158's 45, and the check that excluded them was scoped to one bucket.

789202 needed both buckets at once: imagery from aind-open-data, fragments and all 14 tracings from the other. See §2f for the full inventory of what exists where.

Only 651324 and 721332 are genuinely gold-only. Every brain-held-out result computed before 2026-09-18 was computed over four brains.

Gold layouts: Complete_raw is one SWC per (neuron, compartment, annotator) with occasional consensus files (three of 653158's carry out-of-frame OFFSET headers and are rejected by a bounds guard — RESULTS §1); specimen_space_reconstructions is one file per neuron/part. Node types arrive as 0 and are stamped from the filename compartment at load.

The owners' own lists (received 2026-09-18) are checked in under src/exaproof/io/manifests/: the fused imagery URL of every exaSPIM subject in the bucket (81, against KNOWN's five with imagery) and the segmentation dataset produced for eight further subjects. The README there says how they line up with KNOWN. The CSV's segmentation names identify a volume, not a location; the outputs computed over those volumes are in aind-msma-morphology-data and are now ingested (§2e).

Known data quirks, all handled in code and documented in RESULTS: per-subject seg_key names are not uniform; 653158 N016/N036/N049 consensus files sit tens of mm outside the volume; 648434/653980 segmentations under-cover gold cable (44.1 % / 54.0 % at 8 µm vs 653158's 86.8 % — RESULTS §24), which bounds everything downstream. The two brains added on 2026-09-18 sit above both: 789202 reads 0.921 (n=14, min 0.797) — the best in the corpus — and 685222 reads 0.816 (n=38, min 0.564, worst N004), and 685221 reads 0.889 (n=60, min 0.794 N027, max 0.943 N010) — the largest tracing set in the corpus, second only to 789202 in coverage.

653158 has two pairs of gold arbors that trace the same fibres (E406, scratchpad/e406_label_audit.py). N015 and N016 (somata ~260 µm apart, different annotators) share 21.9 mm of path within 3 µm, 21 % of N016's cable in 381 separate stretches; N003 and N067 (same annotator) share 27.8 mm, 6 % of N067's. Either the axons run together below the resolution limit or a tracing followed the other neuron; the image cannot say which. Consequences: 46 % of 653158's merge instances come from these two pairs, and 39 % of all its instances sit on a shared path, where no image check can find a merge. 653980's largest shared path is 0.8 mm and 648434's 0.07 mm. The split keeps N015 and N016 together in dev; N067 is a test arbor and N003 a training arbor, but only 7 of the 150 standard test gaps lie within 3 µm of a training, calibration or dev tracing.

s3://aind-msma-morphology-data — readable, internal, in use since 2026-09-18

Anonymously readable but not part of the AWS Open Data Program and carrying no license. This project treated it as off-limits until 2026-09-18, when the maintainer lifted that: "you may use the unlicensed data." Reading, ingesting and training on it are allowed; redistributing or publishing it outward is not, and any number drawn from it must say so — the bucket a figure came from is part of the figure.

What is read from it, as of 2026-09-18: the seven newer brains' fragment archives under agrim-whole-brain-predictions/ (§2e — 122.8 M nodes, the largest corpus this project has), registered as extras["seg_url"] in KNOWN and fetched by the same fetch_seg_zip that handles aind-open-data. A written index of the bucket's structure lives at registry/s3_aind-msma-morphology-data.txt (exaproof data index-s3), so searching it costs a grep rather than a few dozen round trips.

2. Local tree (EXAPROOF_DATA, default /home/ubuntu/data/exaproof)

Every directory, what it holds, and how to regenerate it. "Derived" means losing it costs only compute; sizes are as of the sandbox-04 copy.

dir size contents regenerate with
raw/<subject>/ 3.6 GB downloaded gold SWCs + fragment zip per subject (653158 2.0 GB, 653980 1.4 GB, 648434 311 MB). Imagery is never local — fused.zarr streams from S3 exaproof data fetch --subject S
fragments/<subject>/ 2.4 GB columnar fragment store (xyz/radius/parent/node_frag as mmapped .npy) + spatial grid indices + end_nodes exaproof data ingest --subject S
fragments/<subject><suffix>/ ~ base + grown augmented store: the delivered fragments plus cable generated for them (exaproof grow --suffix g). Same layout as fragments/, with meta.json carrying base_subject and n_base_fragments so generated cable is always separable from delivered. Exists so alignment, mining and the session measure the difference without knowing anything new (RESULTS §58) exaproof grow --subject S <neuron> --suffix g
labellings/ 150 MB gold arbors aligned to fragments (cache keyed by tol/spacing) exaproof gt align / first get_labellings() call
decisions/<subject>/<tag>/ 3 MB mined labelled join corpora (653158 v3: 116k; 653980 v1: 20k; 648434 v1: 32k) exaproof decisions mine
features/<subject>/<tag>_<hash>/ 84 MB feature tables, keyed by extractor version hash (97c5e30fac current) exaproof train rebuilds on miss
slabs/<subject>/<tag>/ 579 MB oriented gap-voxel slabs (float16), incremental caches (v3 + wide) — expensive: ~56 MB/s S3 streaming to rebuild exaproof encode
models/<subject>/<name>/ 40 MB decision heads + calibration + OOD + split metadata. Key names: 653158 a10 (main), askdyn (question ranker), slab_enc (CPU encoder); 653980/648434 native exaproof train, exaproof asker, exaproof encode
runs/ 18 MB benchmark outputs (session/reach/triage/… JSON) — the numbers RESULTS.md cites the corresponding exaproof command
cache/ 73 GB fsspec chunk cache of streamed imagery. Disposable; refills on demand (copying it saves ~a day of warm-up)
learned/653158/ <1 MB LLM feature-proposal loop ledger exaproof learn
somas/<subject>.csv (+ .source) ~3 MB detected somas per brain; the .source sidecar names the bucket and run a file came from (the imagery asset has none) io.somas.for_subject fetches on first use
viewer/kept/<subject>/<neuron>.jsonl small cable people kept in the viewer ("keep this cable"), one record per keep with who and when. Not derived and not regenerable: back it up like viewer/users.db written by the viewer
runs/e463_walks/, runs/e465_paths/, runs/e465b_paths/, runs/e468_paths/ small per-stretch records of the write-back experiments (E463: each walk's skeleton with probability and distances to tracing and seed; E465/b, E468: each kept path). E464's analyses read them the experiment runners in scratchpad/
registry/<subject>.json <1 MB per-subject ingest metadata (bounds, counts) written by ingest

ROI training corpora live under runs/roi_corpus_<subject>[_ms]/: one N<arbor>.npz of cubes per arbor plus _seed and _objlabel companions (roi-corpus, then roi-seeds, roi-objlabels). Built so far: 653158 (native and _ms, plus _ms3 with a level-4 channel, §288), 653980c _ms, 648434 _ms, and 609281 _ms2 (2026-09-15: 18 train+calib arbors, context 2:4 + 3:8, no touch cubes — a third brain for pooled training). 685221 _ms (2026-09-18, E466): the 40 training arbors of scratchpad/e466_split.json, ms_100's recipe, plus compartment, seed and object-label companions. roi-seeds takes --split now, or falls back to the corpus's own shards when the subject has no decision model.

609281's imagery reaches the uint16 ceiling and cubes are stored float16, where 65535 casts to inf: one such voxel survives median/IQR normalisation into the network. extract_cube now clips at 65504 and normalise_cube sanitises any older corpus. The first 609281 attempt (roi_corpus_609281_ms, one shard) predates the fix and is not used. The other brains' cubes peak at 1,900–17,200, far below the ceiling.

Provenance chain: S3 → raw → fragments → labellings → decisions → features/slabs → models → runs. Only raw touches the network for anything but imagery; everything after is deterministic given the code version (feature tables carry the extractor hash for exactly this reason).

2b. The imagery metadata is kept locally (2026-09-17)

s3://aind-open-data/exaSPIM_653980_.../fused.zarr lost its .zattrs, .zgroup and every <level>/.zarray some time on 2026-09-17 — the chunk keys are all still there and read fine, but zarr cannot open the group, so the brain became unreadable mid-experiment (E421's first run died on it; E419 had finished an hour earlier only because its process already held the volume open). The bucket is public and not ours to fix.

ImageSource.open now falls back to a saved copy of the metadata under imagery_meta/<hash of url>/, and imagery.snapshot_metadata(url) takes that copy from any volume that still has it. 653158, 648434, 609281 and 651324 were snapshotted from their sources; 653980's was recovered from this machine's zarr chunk cache and verified against the shape of the fused-denoised.zarr twin in the same prefix — (1, 1, 23146, 29005, 62654), 8 levels, 128³ chunks, <u2 — and then by reading imagery at a known coordinate.

A second copy is checked into the repository at src/exaproof/imagery_meta/ (50 KB, one directory per volume, each with a SOURCE file naming it) and local_metadata falls back to it, so a fresh checkout on another machine can open these volumes even if the bucket loses more of them.

The raw volume of 653980 exists only through that copy (2026-09-18) — SUPERSEDED the same day, see the table below

The release owner explained the deletion: 653980's fusion was denoised and the raw volume removed, with the denoised image published beside it as fused-denoised.zarr in the same prefix. The chunks of the raw volume were never deleted — only its metadata — so fused.zarr still reads through the saved copy, and that copy is now the only thing that makes the baseline for every prior 653980 measurement readable. Do not delete src/exaproof/imagery_meta/.

The two volumes sit on the same grid: same shape (1, 1, 23146, 29005, 62654), same 0.748 × 0.748 × 1.0 µm voxels, same eight levels, same dtype. Coordinates, skeletons and the atlas registration are unaffected; only the pixels differ. Measured on 60 gold nodes of N002: cable peak 393 → 389 (×0.99), background 74 → 75 (×1.01), background noise sd 6.0 → 4.2 (×0.69), contrast 5.31 → 5.19. Whether the models care is E427.

Select one with ImageSource.for_subject(subject, variant="denoised"), or EXAPROOF_VOLUME=denoised for a whole run. Both volumes' metadata is snapshotted.

Snapshot a new subject's metadata the first time it is opened. A few hundred bytes of JSON is the only part of a remote volume that cannot be recovered by retrying.

653158's and 648434's raw volumes are GONE — not just their metadata (2026-09-18)

This is worse than the 653980 case above and it failed silently. list-objects-v2 under exaSPIM_653158_.../fused.zarr/ and exaSPIM_648434_.../fused.zarr/ returns zero objects: the chunks themselves have been removed, not only the metadata. Eight gold-cable points per brain read 0/8 non-zero at two resolution levels while the fused-denoised.zarr twins in the same prefixes read 5/5.

brain raw fused.zarr fused-denoised.zarr what to use
653158 (primary, 45 gold arbors) gone present EXAPROOF_VOLUME=denoised — there is nothing else
648434 gone present EXAPROOF_VOLUME=denoised — there is nothing else
653980 gone (was metadata-only earlier the same day) present EXAPROOF_VOLUME=denoised — there is nothing else
609281 gone (checked 21:30 PDT 2026-09-18) none published nothing readable — only a ktx octree and one low-resolution TIFF remain

And 609281 has no readable imagery at all (checked 21:30 PDT 2026-09-18). list-objects-v2 under its stitched_…/fused.zarr/ returns zero objects. What remains in that prefix is a MouseLight-style octree/ (ktx/ tiles plus one 134 MB default.0.tif), which this project does not read, and there is no denoised twin. The viewer reports reads: false. Found by the viewer's metadata route (/ng/volume/): the saved metadata served fine, and every redirected chunk came back 404. Any 609281 figure that reads imagery must be dated before the deletion or re-read from the octree. Figures from skeletons and fragments alone are unaffected.

653980 went the same way later on 2026-09-18 (checked 17:00 PDT): list-objects-v2 under exaSPIM_653980_.../fused.zarr/ returns zero objects, and 24 fragment-node points read blank while the denoised twin reads. The section above — "the chunks of the raw volume were never deleted" — was true when written and is not now. exaproof view no longer needs the variable for a tool a person looks at: ImageSource.live_for_subject moves to the denoised twin when the default reads blank at cable and the twin reads, and the browser's Neuroglancer URL follows it (zarr_url_for checks the default's .zattrs once). Measurements still get exactly the volume they ask for (for_subject is unchanged), so a blank read there stays a loud failure rather than a quiet substitution.

Why it was silent, and the guard that now stops it. The saved-metadata fallback above resolves the shape from our copy; with the objects gone, every chunk 404s and zarr fills zeros. A volume of zeros never raises — it answers every question with "nothing here", which from a segmenter looks like an answer. ImageSource.reads_data and .check_alive turn it back into a loud failure, exaproof view shouts at startup, and any corpus build must check before and after (see scratchpad/e440_denoised_corpus.sh). Hand them points where there IS cable — gold nodes — because dark background is common and only a lit point proves the volume is alive.

Two consequences for anyone reading old numbers. Every stored 653158 and 648434 figure was measured on imagery that can no longer be reproduced, so §337's rule binds hard: no new figure may be compared with an old one unless both were read on the same volume. And runs/roi_corpus_653158_ms (37 GB, 39 arbors) is now the only copy of raw-imagery training cubes for 653158 that will ever exist — do not delete it. runs/roi_corpus_653158_msd is the denoised replacement (E440).

There is also a deliberate variant="blank" — the real volume with every read zeroed, same grid, same seeds. It exists because the accident above turned out to be a useful control: a cue that claims to read the image is scored against it (§346, §347).

2e. The eight newer brains, surveyed (2026-09-18)

The owners' manifests (src/exaproof/io/manifests/) name 81 subjects with fused imagery and 8 with a segmentation dataset. Those 8 are the interesting ones — imagery plus a segmentation is what this tool needs — so every prefix under s3://aind-open-data carrying their subject number was listed and opened and read. Every "readable" below is a verified read of real voxels, not an inference from a listing, because listing was misleading in both directions here.

subject volume that reads levels · shape (z,y,x) somas (CCF) CCF align
709221 fused-denoised.zarr — the raw one is gone 8 · 21186×40386×66437 6,321 ✓
718162 fused.zarr and fused-denoised.zarr 7 · 17411×38903×62516 3,718 ✓
789202 fused.zarr 7 · 21558×41022×66869 92 ✓
794491 fused.zarr in a different asset (_processed_2025-11-08_09-50-12) 7 · 19587×40078×64486 649 ✓
794492 fused.zarr 7 · 20685×40538×65967 724 ✓
794493 fused.zarr 7 · 20991×39179×64507 1,077 ✓
794495 fused.zarr 7 · 22973×40789×67530 119 ✓
802449 fused.zarr 7 · 16901×38042×62544 57 ✓

789202 has no fused-denoised.zarr. EXAPROOF_VOLUME=denoised and variant="denoised" fall back, with a warning, to its raw fused.zarr, so every 789202 row run "denoised" read raw (§809). Before row 299, variant="raw" was not a raw switch: it asked for fused-raw.zarr and fell back to the registry's default, which for 685221 is the denoised volume. It now opens the raw fused.zarr and raises when that is gone; strict=True makes any variant raise. 685221's raw fused.zarr beside its denoised volume is deleted (fused.zarr.deleted, 2026-09-10). Its other fusions with a fused.zarr (flatfield-correction_2025-02-20…_fusion_2025-02-20_19-56-18: 3,626×3,679×7,859 at a stated 0.748 µm; fusion_2024-05-02_20-16-08: 5.984 µm) are not on the denoised grid (row 294).

721332 is the one gold brain with both volumes on one grid (row 297, §810; registered in datasets.KNOWN by row 299, raw fused.zarr as its default, variant="denoised" its twin, the 22 gold fetched; raw and denoised give identical separation verdicts there, §811): exaSPIM_721332_2025-04-05_14-48-36_flatfield-correction_2025-04-24_20-06-12_fusion_2025-05-06_17-30-15/ holds fused.zarr and fused-denoised.zarr (25,187 × 52,320 × 62,708 at 0.748 µm, 8 levels), and its 22 gold neurons are specimen-space SWCs in exaSPIM_721332_2025-04-05_14-48-36_reconstructions_2026-08-28_23-04-54/specimen_space_reconstructions/swc/ (41 MB) that land on its fibres. datasets.KNOWN["721332"] still has fusion=None. Every other gold brain has one volume. EXASPIM25's block crops are raw on the denoised grid for 653158, 650010 and 651324.

No segmentation exists for any of them in this bucket. segmentation_datasets.csv names pipeline runs (raw.unet_r3_ckpt_425250_seedfix_overlapfix, mean40.stddev105.mask…ffn.mt_0.1); the immediate children of all 42 top-level assets for these subjects were listed and none contains a seg/ or anything matching. Where they are stored has to come from the owners. Until then these brains can be looked at and not proofread: the worklist is built from fragments, and fragments come from a segmentation.

What is genuinely new and usable: somas. soma_detection/aligned/soma_locations_ccf.swc gives 57 to 6,321 detected somas per brain in CCF space. Every reconstruction starts at a soma and until now the seed had to be given by hand. ccf_alignment/ also carries the full forward and inverse SyN warps (~1.8 GB each), not only the per-node displacement SWCs of §2c.

What is NOT usable, and was nearly recorded as if it were. Four subjects have denoised/probability/ and denoised/mask/ directories, and a first pass through this section listed them as a per-voxel probability volume this project could use. They are empty shells — 794492's probability/SPIM.ome.zarr is a single 24-byte .zgroup and nothing else; 789202's is empty. Listing a directory is not reading it, which is the same mistake in the same day as §343's blank volume. Where real denoised output does exist it is per-tile (denoised/SPIM.ome.zarr on 789202's and 802449's second assets), not a fused volume on the imagery grid.

Two traps, both of which silently produce the wrong data.

  1. The newest asset is not the one with the data. Each subject has two to seven _processed_ assets; four of the eight keep their somas in an earlier asset than their latest, and 794491's live imagery is in a different asset from the one the manifest names. Picking "the last prefix that matches" — what discover did before it read the manifest — gets the wrong one.
  2. The manifest can name a volume that has been deleted. 709221's and 794491's manifest entries both point at fused.zarr prefixes with zero objects. The bucket marks these explicitly — fused.zarr.deleted, fused.n5.deleted, denoised_full_res.deleted — so a deletion is detectable rather than something to be discovered by reading zeros (§2b). Any code that resolves a volume should look for a .deleted sibling and for a fused-denoised.zarr twin before giving up.

The segmentations, found (2026-09-18)

segmentation_datasets.csv names a run per subject and no object under aind-open-data matches, because the outputs live in s3://aind-msma-morphology-data — which this project had been told to treat as internal until the maintainer lifted that on 2026-09-18 ("you may use the unlicensed data"). Read and ingest yes; do not redistribute it, and say which bucket a number came from.

They are under agrim-whole-brain-predictions/<subject>/<run>/, and each run's metadata.json carries the segmentation_id that the CSV names, which is what ties the two together:

{"brain_id": "709221", "segmentation_id": "202508_...mean50.mask.136168199.no_omitted_20k.ffn.mt_0.1",
 "min_fragment_size": "40.0um", "node_spacing": 4, "use_somas": true, "search_radius": "30.0um",
 "model_name": "GraphNeuralNet-20250331-v2-0.8577.pth", "accept_threshold": 0.825}

So these are not raw segmentations but a graph-neural-net proposal pass over one: fragments above 40 µm, up to three proposals per leaf within 30 µm, accepted at 0.825. That is the same decision this project's chooser makes, already made by another model — which makes them a comparison, not only a corpus.

Seven of the eight are ingestable, and all seven now are. An earlier revision of this section said only four were, because three of them nest the archive one directory deeper — …/<run>/corrected-swcs/corrected-swcs.zip rather than …/<run>/corrected-swcs.zip — and a listing of the run prefix shows that as a bare corrected-swcs/ next to connections.txt, which reads like a directory of loose files. It is one zip. Descend before concluding a thing is absent.

subject run taken as primary archive fragments nodes
709221 …855_mean50.mask…ffn.mt_0.1_20250829_020215 3.84 GB (nested) 2,947,651 61,488,872
789202 …raw.jin_masked_mean40_stddev105_less_aggressive_seedfix_20260304_004343 68 MB 19,075 1,137,366
794491 …mean40.stddev105.mask…ffn.mt_0.1_20251117_054446 1.18 GB (nested) 595,890 19,487,832
794492 …jin_masked_mean40_stddev105_less_aggressive_20260209_190101 500 MB 203,823 8,285,727
794493 …mean40.stddev105.mask…ffn.mt_0.1_20251117_053518 500 MB (nested) 241,445 8,265,318
794495 …raw.unet_r3_ckpt_87750_aggressive_20260307_210855 478 MB 258,723 7,989,421
802449 …jin_masked_mean40_stddev105_20260308_032531 985 MB 308,370 16,106,818
718162 — nothing published under this prefix — —

That is 122.8 million nodes across seven brains, against the 10 brains this project had — and 709221 alone, at 2.9 million fragments, is larger than the other six together. Its ingest took 366 s and 9 GB of resident memory; the other six took 9-25 s each.

Which run, when there are several. 802449 publishes 24 and 794495 eight. The rule taken here is the newest corrected_<subject>_… run, which is the pipeline's own output at its current settings; original_* are the same segmentation before the GNN's proposals were applied, and corrected_nongnn_* / corrected_heads4_* are ablations of the accepting model. The first pick for 802449 was corrected_nongnn_… — chosen by date alone, and by name the one run with the GNN switched off. It has been replaced, and the ablations are kept in the registry as extras["seg_variants"], because a segmentation and the same segmentation without the joiner is a matched pair, which is the cleanest measurement of what an accepting model contributes that this project has access to.

The CSV names a segmentation, not a run, and mostly not a published one. segmentation_datasets.csv gives e.g. 802449,raw.unet_r3_ckpt_425250_seedfix_overlapfix; the runs published for 802449 are all over jin_masked_mean40_stddev105. Only 794491 and 794493 have a run whose metadata.json segmentation_id equals the CSV's string. The newer runs (789202, 794492, 794495, 802449) carry metadata_graph.json + metadata_ml.json instead of metadata.json, and no segmentation_id at all — the segmentation is named only in the run's directory name. So the CSV identifies the volume each group intends, and the volume itself is not in either bucket. segmentation/exaSPIM_<subject>_zarr/ exists for 28 other subjects and for none of these eight (checked against the written index, exaproof data index-s3). Despite the _zarr in the name, those prefixes hold no voxels. A recursive listing on 2026-09-24 found only SWC archives and sidecars under segmentation/: 919 objects, split into 401 .zip, 299 .swc and 209 .json, with no array chunk or .zarray. The FFN run directories inside them, e.g. exaSPIM_685221_zarr/…ffn.mt_0.1/, hold swcs_0.zip–swcs_15.zip (§2d).

The accepting model is named too, and differs by run: VisionHGAT-20260205-40-0.8723.pth at threshold 0.8 for 789202 and 794495, VisionHGAT-20260205-31-0.8617.pth at 0.7 for 794492, VisionHGAT-20260301-58-0.8692.pth at 0.8 for 802449 — so the four newer brains are not a single homogeneous corpus, and a cross-brain reading has to say which acceptor produced each. Every newer run also ships proposal_predictions.json: another model's probability for each proposal it considered, on the same decision this project's chooser makes.

Also in that bucket and worth knowing: segmentation/794495_it2_split_corrected_distance_search/ (299 SWCs with JSON sidecars), exaspim_soma_detection/, training-data/, training_blocks/, and merge-site collections for 685221 and 719654.

789202 has manual tracings, and is the fifth end-to-end brain (2026-09-18)

The same bucket holds an annotator's working tree, anna.grim/, and in it groundtruth_tracings/789202/world/ — 14 manually traced neurons, one file per (neuron, annotator), named N005-789202-SP.swc in the same convention Complete_raw uses without the per-neuron directory. That is 8.06 metres of gold cable (259–978 mm per arbor, 6.9 M nodes), and with the fragments above it makes 789202 the first subject since the original four to support the end-to-end split/merge task — and the first assembled from two buckets: imagery from aind-open-data, fragments and gold from aind-msma-morphology-data.

The frames agree, which was checked and not assumed — the rule that BIL's voxel-index SWCs taught this project. Thirteen of the fourteen arbors sit a median 2.5 to 4.8 µm from the nearest fragment node, so the tracings are already in the fragments' world micrometres and need no scaling. N023 reads 72.8 µm, which looked like a frame problem and is not one: aligned, it covers 0.411 of its cable against a median of 0.715, so it is simply the worst-segmented arbor and its distance distribution is bimodal — 41 % of nodes within 8 µm, the rest far.

Aligned coverage puts it FIRST in the corpus — but only for the right segmentation. Three segmentations of this brain exist and they do not agree:

registered as archive fragments / nodes median coverage @ 8 µm
789202 the CSV's raw.unet_r3_ckpt_425250…, pre-proofreading 60,622 / 4.52 M 0.921 (min 0.797)
789202final the same after the proofreading pipeline 75,914 / 4.37 M measured in E449
789202wbp agrim-whole-brain-predictions' 68 MB output 19,075 / 1.14 M 0.715 (min 0.411)

0.921 beats 653158's 0.868 and every other brain. The first ingest here made 789202wbp canonical — it was the archive the whole-brain-predictions survey found — and recorded 0.715 as the brain's coverage. That archive holds a quarter of the cable. The owners' CSV named the better segmentation all along, and the lesson is the one §353 records: the file you find first is not evidence about the brain.

It is the only one of the eight with tracings: groundtruth_tracings/ contains 789202 and nothing else.

Three more collections in anna.grim/ are worth a later look, recorded here so the search is not repeated:

prefix what it is
merge_detection/<subject>/result_<segmentation>_<date>/ detected_sites.zip + fragments.zip (~250 MB) for 789202, 794491, 794492, 794493 — the pre-correction fragments beside the sites a detector flagged, which is the matched pair for studying merges
image-carveouts/802449/whole-brain-N001…N039/ input.zarr + mask.zarr + swcs_extracted/, one whole-brain-sized mask.zarr per gold neuron. It is a binary mask: every chunk sampled on 2026-09-24 across N001 and N036 holds only 0 and 1. It is likely drawn from the tracing and is not segmenter output. It could supervise a segmenter rather than a skeleton model. image-carveouts/754612/blocks/block_000/ has the same layout at 1024³, with a malformed store: chunk files are written in both /- and .-separated form, and some lie outside the declared shape
automated_proofreading/<subject>/<segmentation>/ 789202 and 794495, under the segmentation names segmentation_datasets.csv gives — this is where the CSV's names finally resolve

A written index of that prefix (depth 6) is how these were found; regenerate with exaproof data index-s3 --bucket aind-msma-morphology-data --root anna.grim/ --depth 6.

2f. The full inventory (2026-09-18)

Both buckets were walked and the listings written to registry/s3_*.txt and registry/wbp_survey.json, so the two tables below are regenerated offline in a second (exaproof data inventory) rather than by another few dozen aws s3 ls calls. Three fixed-name assumptions had each hidden data before this: an archive nested one directory deeper, a gold directory under a layout name the loader did not know, and a run whose archive is called corrected-processed-swcs-s3.zip rather than corrected-swcs.zip. Every search here matches by shape — any .zip in the run, any *.swc in the directory — not by an expected name.

Table A — every brain we have a name for, and what exists for it

Built offline from the written indexes (exaproof data index-s3, registry/s3_*.txt) plus registry/wbp_survey.json. imagery is what the owners' exaspim_image_prefixes.json entry resolves to; denoised only means the raw fused volume it names is gone and fused-denoised.zarr sits beside it. somas: yes = the imagery asset's soma_detection/; bucket = the production detector's runs in the morphology bucket (exaspim_soma_detection/<subject>/results_*/, 122 brains, 51 of them only there), which exaproof.io.somas.for_subject falls back to, newest run first. That source was added 2026-09-18 evening. Until then this column said "—" for every end-to-end brain.

685221's archive was 22 MB in this table and 7.4 GB in §1's registry, a factor of 336. Settled 2026-09-19 against the local artefact: raw/685221/seg.zip is 7,444,560,935 bytes — 7.4 GB decimal, 6.9 GiB — which is §1's figure, so this table was wrong and now reads 7.4 GB. For reference the tracings beside it are 596 MB (specimen_space_reconstructions), so 22 MB was not a mislabelled tracings size either; it appears to predate the archive being identified.

brain imagery tracings segmentation archive somas CCF status here
609281 — manual (open-data) own seg/ bucket yes end-to-end
613814 denoised only — — bucket yes —
648434 denoised only manual (open-data) 291 MB + own seg/ bucket yes end-to-end
650010 denoised only — — bucket yes —
650011 denoised only — — bucket — —
651324 denoised only manual (open-data) — — yes registered
651895 denoised only — — bucket yes —
652440 denoised only — 5,321 MB yes + bucket yes —
652779 denoised only — — yes + bucket yes —
653158 denoised only manual (open-data) 3,420 MB + own seg/ bucket yes end-to-end
653159 denoised only — 999 MB bucket yes —
653980 denoised only manual (open-data) own seg/ — yes end-to-end
657676 denoised only — 3,923 MB bucket yes —
658735 denoised only — 14,204 MB bucket yes —
665081 denoised only — 2,672 MB bucket yes —
667996 denoised only — 1,112 MB bucket yes —
667997 denoised only — — yes + bucket yes —
671477 raw — — — — —
674185 denoised only — 0 MB bucket yes —
674190 raw — — — yes —
674191 denoised only — — bucket yes —
675375 denoised only — — bucket yes —
676009 denoised only — — — yes —
681465 denoised only — — bucket yes —
685221 denoised only manual (open-data) 7.4 GB bucket yes registered
685222 denoised only manual (open-data) 4,143 MB bucket yes registered
686951 denoised only — — yes + bucket yes —
686955 raw — — bucket yes —
697837 denoised only — — yes + bucket yes —
703071 denoised only — — yes + bucket yes —
704522 denoised only — 11,008 MB yes + bucket yes —
704523 — — 6,416 MB bucket yes —
706301 none — 1,069 MB bucket yes —
708365 denoised only — — bucket yes —
708369 denoised only — 0 MB bucket yes —
708373 none — — bucket yes —
709203 denoised only — 5,166 MB bucket — —
709221 denoised only — 3,842 MB yes + bucket yes registered
709222 denoised only — 51 MB — yes —
709393 denoised only — — bucket yes —
713600 denoised only — — bucket yes —
713601 denoised only — — bucket yes —
715345 raw — 883 MB bucket yes —
715346 raw — — bucket yes —
715347 raw — 2,529 MB bucket — —
717445 raw — 1,089 MB bucket — —
717614 raw — — — — —
718162 raw — — yes + bucket yes registered
718168 raw — — bucket yes —
719654 raw — 2,038 MB bucket yes —
720164 none — — yes + bucket yes —
720165 raw — — bucket — —
721332 raw manual (open-data) — bucket yes registered
721828 raw — 0 MB yes + bucket yes —
721830 raw — — yes + bucket — —
730223 raw — 34 MB yes + bucket yes —
730902 raw — 313 MB bucket yes —
733420 raw — — yes + bucket yes —
737563 raw — 281 MB bucket yes —
737826 raw — — yes + bucket yes —
744783 raw — 59 MB yes + bucket yes —
747807 raw — 2,893 MB yes + bucket yes —
747820 raw — 678 MB yes + bucket yes —
750318 raw — 3,159 MB yes + bucket yes —
751473 raw — 20 MB — yes —
754610 raw — 5 MB — — —
754611 raw — 465 MB yes + bucket yes —
754612 raw — 0 MB bucket yes —
754613 raw — 555 MB bucket yes —
763259 raw — — bucket yes —
776235 raw — — yes + bucket yes —
776241 raw — — yes + bucket yes —
784666 raw — — yes + bucket yes —
784802 raw — — yes + bucket yes —
784896 raw — — yes + bucket yes —
785688 raw — 269 MB yes + bucket yes —
789202 raw manual (morph) 68 MB yes + bucket yes end-to-end
794491 other sibling — 1,182 MB yes + bucket yes registered
794492 raw — 500 MB yes + bucket yes registered
794493 raw — 500 MB yes + bucket yes registered
794495 raw — 478 MB yes + bucket yes registered
802449 raw — 553 MB yes + bucket yes registered
802450 raw — — yes + bucket yes —
820994 — — 499 MB yes + bucket yes —

84 brains named.

can we use it? count brains
tracings + segmentation (end-to-end reachable) 7 609281, 648434, 653158, 653980, 685221, 685222, 789202
segmentation, no tracings 38 652440, 653159, 657676, 658735, 665081, 667996, 674185, 704522, 704523, 706301, 708369, 709203, 709221, 709222, 715345, 715347, 717445, 719654, 721828, 730223, 730902, 737563, 744783, 747807, 747820, 750318, 751473, 754610, 754611, 754612, 754613, 785688, 794491, 794492, 794493, 794495, 802449, 820994
tracings, no segmentation 2 651324, 721332
neither 37 37 brains, imagery only

All 7 are in use: 653158, 653980, 648434, 609281, 789202, 685221 and 685222. (This line said "5 of the 7" until 2026-09-19 and was already wrong when written -- 685221's ROI corpus is described in this file's own section 2, and FINDINGS carries results on it.)

Table B — the names we were given, and which ones resolve

Both manifests arrived from the data owners on 2026-09-18. Every entry was probed.

list entries resolve as given resolve to a sibling nothing there
exaspim_image_prefixes.json 81 42 36 3
segmentation_datasets.csv 8 see below — —

The 36 that resolve only to a sibling are one thing, not thirty-six: the raw fused.zarr the manifest names has been deleted and fused-denoised.zarr published in its place. §343 recorded this for 653158 and 648434 as an incident; it is bucket policy. Any code that opens the manifest's URL reads nothing, silently.

brain the manifest's URL ends in what actually reads
613814 fused.zarr fused-denoised.zarr
648434 fused.zarr fused-denoised.zarr
650010 fused.zarr fused-denoised.zarr
650011 fused.zarr fused-denoised.zarr
651324 fused.zarr fused-denoised.zarr
651895 fused.zarr fused-denoised.zarr
…and 30 more fused.zarr fused-denoised.zarr

Three have nothing readable in the prefix at all — not a rename, an empty prefix:

brain manifest URL
706301 s3://aind-open-data/exaSPIM_706301_2024-04-23_11-24-24_fusion_2024-05-21_00-00-03/fused.zarr/
708373 s3://aind-open-data/exaSPIM_708373_2024-04-02_19-49-38_fusion_2024-09-03_19-00-37/fused.zarr/
720164 s3://aind-open-data/exaSPIM_720164_2025-07-07_17-55-45/SPIM.ome.zarr/tile_000004_ch_488.zarr/

The segmentation CSV. All eight subjects it names have outputs published, but the CSV names a segmentation (a volume) and the published objects are runs over one. Where the two can be matched:

brain CSV names is that segmentation published?
709221 20251001_mean40.stddev105.mask.136168199.no_omitted_20k.ffn.mt_0.1 no — the published runs are over 202508_…mean50…, a different segmentation
718162 raw.jin_masked_mean40_stddev105_less_aggressive_seedfix no run published at all under this brain
789202 raw.unet_r3_ckpt_425250_seedfix_overlapfix_rempm yes — anna.grim/automated_proofreading/789202/raw.unet_r3_ckpt_425250_seedfix_overlapfix_rempm/
794491 mean40.stddev105.mask.136168199.no_omitted_20k.ffn.mt_0.1 yes — the run's metadata.json segmentation_id matches exactly
794492 denoised.unet_r3_ckpt_155250 no — published run is over jin_masked_mean40_stddev105_less_aggressive
794493 mean40.stddev105.mask.136168199.no_omitted_20k.ffn.mt_0.1 yes — segmentation_id matches exactly
794495 raw.unet_449_792_202_splits_and_merges_831600 no — seven runs, none over raw.unet_449_792_202_splits_and_merges_831600
802449 raw.unet_r3_ckpt_425250_seedfix_overlapfix no — 22 runs, all over jin_masked_mean40_stddev105

So three of eight CSV names resolve to a published segmentation (789202, 794491, 794493); the other five name a volume nobody has published, while the brain does have outputs over a different one. The segmentation/exaSPIM_<subject>_zarr/ prefixes exist for 28 other subjects and for none of these eight. They hold skeletons, not segmentation volumes (§2d), so connections.txt cannot be resolved into cable.

What this changes

  • agrim-whole-brain-predictions holds 43 brains, not the eight the CSV lists, and every one of them has an ingestable archive (68 MB to 14.2 GB). The CSV is a list of eight segmentations someone cared about, not an inventory of what exists.
  • INVENTORIED 2026-09-19 (scratchpad/e516_wbp_inventory.out): all 43 have imagery, and 38 of them have no tracings. Every prediction brain resolves to an exaSPIM_<subject> asset on aind-open-data (220 exaSPIM brains have imagery there in total). Five of the 43 are brains this project already traces. The 38 untraced ones carry 245 GB of skeletons, from 708369 and 730223 at under 0.1 GB to 706301 at 29.8 GB. What they can and cannot supply. The ROI corpus samples omission gaps and terminals, and both need gold: a gap is cable a person traced that the segmenter missed, and a terminal is where an arbor ends rather than where the segmenter stopped. So these 38 brains cannot supply the supervision the ROI model exists for. What they can supply is IDENTITY — this neurite against the one beside it — which is the measured cross-brain failure (§364: recall holds on unseen brains, selectivity falls 0.084–0.113) and what §441 showed setting what every write-back lever buys. The trap to design around: the segmenter breaks rather than merges, so labelling its unclaimed voxels as background would train our model to reproduce its misses. The corpus format already carries the fix (0 bg, 1 neurite, 3 hard background, 255 ignore): seed the fragment as 1, other fragments as 3, every unclaimed BRIGHT voxel as 255, and only dim voxels as 0 — thresholded by E171's measurement that missed cable sits at the 89th–97th percentile of its own background. Centre cubes on fragment interiors, never ends, because a fragment end is usually the segmenter's error rather than a terminal. Goal D4-multibrain-roi.
  • Seven brains are end-to-end reachable and five are in use. The two unused are 685221 (60 gold arbors) and 685222 (38) — the largest gold sets in the corpus after 653158's 45, both previously recorded here as "gold only, no fragments", which was true of aind-open-data and false of the other bucket.
  • 653158 and 648434 have a second, independent segmentation (3.4 GB and 291 MB) of brains this project has measured for a year. 648434's 44.1 % coverage bound is the tightest constraint in the corpus and has never been tested against an alternative segmentation of the same brain.
  • Only two brains have tracings and no segmentation anywhere: 651324 and 721332.

2c. The atlas (CCF), and the registration that reaches it

What exists, and where (listed and read 2026-10-16). Every gold brain's reconstruction release in aind-open-data publishes its tracings a second time in CCF micrometres, axes (AP, DV, LR) — checked on 648434 N012 against the release's own per-node allenId: the soma reads SUT in both, and 46.6 % of its nodes match as written against 0.7 % with DV flipped (most of the rest lie behind the atlas volume's 13.2 mm, where the release writes root and the annotation has nothing). exaproof.io.ccf.registration(subject) pairs each gold node with its own atlas position and caches the pairs (and the neuron each came from) under ccf/registration_<subject>.npz:

form brains (files) what it is paired by
<reconstructions>/Complete_annotated/<N>-<subject>-<annotator>.json 609281 (26), 651324 (13), 653158 (42), 653980 (21) the annotator's own Complete_raw files: axon = the soma then the axon file, dendrite = the dendrite file, in file order, with a per-node allenId node for node, only where every parent pointer agrees (pair_annotated); a part edited since the release, branch by branch
<reconstructions_2026-05-06…>/ccf_space_reconstructions/{swc,json}/<N>-<subject>.swc 648434 (12), 685221 (60), 685222 (38), 721332 (22) the same neuron after the owners' refinement (snapped to the image, A*-rerouted, re-sampled — 648434 N012 has 2,720 nodes here and 3,091 there) and transform through the exaSPIM template; topology kept, node order and count not branch by branch (pair_tracings); validated below

721332's local gold has not been fetched, so it builds on first use after exaproof data fetch --subject 721332. The 2026 brains have four reconstruction releases each (2026-03-27, -04-15, -05-06, -08-28); the 2026-05-06 one is the one KNOWN names and the one the local gold was fetched from (648434 N012 byte-identical). The raw transforms also exist — CCF_Atlas_Registration/{affine_transforms.mat, warp_transforms.nii.gz} in 648434's and 653158's fusion prefix, CCF_Atlas_Registration_10/ (a 5.4 GB 10 µm warp) in 653158's and 653980's, none in the fusion prefixes KNOWN names for 685221 and 685222 — and are not needed: the CCF-space tracings are the transform already applied. (The "CCF" column of §2f's table means such a folder exists, not that a registration was built.) In aind-msma-morphology-data the only CCF-space tracings are cameron.arshadi/{snapshot-test,nmcp-snapshot-test}/ccf_space_reconstructions/ (55 neurons of 685221, NMCP export tests of November 2025); nothing there is CCF-space for 653980 or 789202 (the 2026-09 indexes under registry/, and live listings of anna.grim/groundtruth_tracings/).

Per brain, as built (scratchpad/ccf_internal_verify.py brains, …_brains.out). The checks need no truth: neighbours — pairs of nodes of different neurons within 10 µm of each other in the brain, and how far apart the registration puts them in the atlas (a correspondence must keep them close; consistency, which registration applies, refuses a median over 50 µm); cable ratio — each gold arbor's length over its length mapped through its own neuron's pairs, which should sit at the brain's linear scale; soma — the gold soma's region against the allenId the release itself gives that soma.

brain source gold neurons pairs neighbours: pairs · atlas p50 / p90 µm · over 50 µm cable ratio p50 (range) worst arbor covered somas in annotation · region = release's
653158 Complete_annotated, node for node 45 (42 annotated; N016, N036, N049 have no JSON) 305,184 1,123 · 1.8 / 4.0 · 0 2.31 (2.23–2.40) 1.00 of the 42 42/42 · 42/42
653980 Complete_annotated (N022's axon, edited since, by branch) 21 206,589 237 · 3.2 / 4.3 · 0 2.35 (2.19–2.42) 0.95 21/21 · 20 (21 nested)
609281 Complete_annotated, node for node 25 137,381 420 · 3.4 / 5.4 · 0 1.84 (1.73–1.97) 1.00 25/25 · 23
651324 Complete_annotated, node for node 13 250,191 29,975 · 2.3 / 4.4 · 0 2.12 (2.02–2.24) 1.00 13/13 · 13
648434 ccf_space_reconstructions, by branch 12 142,814 14 · 15.6 / 61.4 · 2 2.41 (2.31–2.47) 0.90 12/12 · 11
685221 ccf_space_reconstructions, by branch 60 2,415,714 28,075 · 1.6 / 3.5 · 4 2.76 (2.67–2.87) 0.86 60/60 · 54 (58 nested)
685222 ccf_space_reconstructions, by branch 38 1,635,413 2,810 · 2.3 / 4.4 · 0 2.67 (2.51–2.79) 0.85 38/38 · 32 (33 nested)

"Worst arbor covered" is the share of the least-covered arbor's nodes within 15 µm of its own neuron's pairs (a branch the matcher could not settle is left out, not guessed). Every mapped soma sits exactly on the release's own soma (0.0 µm on 648434, 685221 and 685222); where the region still differs it is the annotation, not the pairing — the release reads its allenId on the 10 µm volume, io.ccf on the 25 µm one, and the misses are neighbours (P against MV or PB, scp against MRN).

The linear scale is not 3: expanded micrometres per atlas micrometre, measured from the pairs, is 2.1–2.4 on 653158, 653980, 651324 and 648434, 1.84 on 609281 and 2.7–2.8 on 685221 and 685222. (653158's own expansion() read 1.56 under the old file-order pairing; it reads 2.27 now.)

How good the branch-by-branch pairing is, against truth (…_synthetic.out). The node-for-node brains supply truth: each gold arbor of 653158 and 653980 is given a "release-style" copy made from its own true CCF tracing (every branch re-sampled at 8 µm, 1 µm jitter, children shuffled, and in a second run one terminal twig removed), and pair_tracings pairs the gold against it. Without a lost twig, 99.9 % (653158) and 99.7 % (653980) of nodes are paired, the per-arbor median node lands 1.4 and 1.6 µm from its own atlas position, 90 of 304,851 (0.03 %) and 3,731 of 199,049 (1.9 %) are off by more than 25 µm and 1 and 1,249 (0.6 %) by more than 50 µm, and none of the 41,904 matched branch ends is off by more than 25 µm — no branch is paired with the wrong one. With one twig removed from every arbor, 99.5 % and 96.2 % are paired (653980 N019 loses 488 of its 838 branches to a twig lost near the soma) and 8 of 41,284 branch ends are off by more than 25 µm, twig merges taken the wrong way. The error that remains sits inside long unbranched stretches whose registration changes scale along them (653980 N031: CCF µm per local µm goes 0.42 → 0.90 → 0.28 along one 4.5 mm axon). On the real 2026 brains it is the few neighbour pairs above 50 µm in the table: two nodes on 648434 N020's 23 mm axon, 73-75 µm from where N023 puts the same place, and one spot where 685221 N012 and N027 run together, ~60 µm.

789202 has none. Its 14 tracings are anna.grim/groundtruth_tracings/789202/world/ in the morphology bucket, with no CCF copy beside them (world/ is the only folder), and aind-open-data has no reconstructions release for it. What its processed asset (exaSPIM_789202_2026-02-03_15-09-59_processed_2026-02-15_23-47-18/ccf_alignment/) does hold: the brain → exaSPIM-template SyN (789202_to_exaSPIM_SyN_{0GenericAffine.mat, 1Warp.nii.gz, 1InverseWarp.nii.gz}, 1.8 GB each), registration_metadata/ (the resampled and moved images), and ccf_anno_to_sample/ccf_anno_in_sample_space.zarr — the CCF annotation warped into the brain's own grid (level 0 at 8 × 5.984 × 5.984 µm, z y x, origin 0, i.e. multiscale level 3 of the imagery). The template → CCF half of the chain (reg_exaspim_template_to_ccf_25um_v1.4, named in soma_detection/register_soma_process.log) is in neither bucket's index. Two ways to a registration: (1) the owners run their exaspim-swc-transform + aligned-swc-processing on these 14 tracings (what produced every ccf_space_reconstructions above), after which registration("789202") works unchanged; (2) the template → CCF transforms are found and the chain is applied to the gold nodes (needs ANTs or a NIfTI reader; neither is installed), checked on the asset's 92 somas, which carry both xyz_raw and xyz_ccf_auto. Region lookups alone need neither: read ccf_anno_in_sample_space.zarr at a point's own micrometres — it gives the 92 somas the same structure as the CCF annotation at their xyz_ccf_auto for 80 (82 equal or nested), and is not yet wired into io.ccf.

What this replaced. Until 2026-10-16 registration read 653158's CCF_Atlas_Registration/displacement_swcs/ (one zip per neuron, one two-node SWC per node of the release's own *_joined.swc) and paired file _i with gold node i. The paragraphs below are the record of that and of its first correction.

s3://aind-open-data/exaSPIM_653158_.../CCF_Atlas_Registration/displacement_swcs holds one zip per traced neuron, and inside it one two-node SWC per gold node giving that node's position in CCF micrometres. The file index is the gold node index — checked node-for-node, 7,879 files against 7,879 nodes for N006 — so pairing them with the local gold gives a dense correspondence (257,670 nodes for 653158) without touching the 854 MB displacement field. exaproof.io.ccf.registration(subject) builds and caches it under ccf/registration_<subject>.npz.

CORRECTED 2026-10-16 (K5, scratchpad/k5_denominator.out): the file index is NOT the local node index. The counts agree (31 of 45 neurons pair), the order does not: paired by index, two neurons' nodes within 10 µm of each other locally land more than 200 µm apart in CCF in 1,067 of 1,068 cases, and a gold arbor comes out ten times longer in the atlas than in the expanded brain. Each neuron's CCF points are its own arbor as an unordered cloud, so exaproof.io.ccf.rematched_registration(subject) re-pairs them by shape (cached as ccf/registration_<subject>_rematched.npz): the same pairs then agree to a median 9.4 µm, and the gold somas move 0.6-6.4 mm, from isocortex (21 of 31 as released) to thalamus (24 of 31). registration() is unchanged, and every region split read through it on 653158 (agentic.scoreboard.Regions, the viewer's place lookup) is wrong until its caller switches.

653980 has no displacement_swcs — only affine_transforms.mat and warp_transforms.nii.gz — so a confirmation on that brain needs the warp applied, which has not been done.

SUPERSEDED 2026-10-16 by Complete_annotated (above): the same tracings node for node, so nothing is inferred. Against it the file-order cache was off by a median 1,179 µm per gold node (somas 3,775 µm; no node within 25 µm; …_old.out), and the shape re-pairing of PR #78/#79 agrees with it to a median 36.7 µm (p90 66) — of which the two published registrations' own difference is a part: N006's displacement_swcs cloud and its Complete_annotated cloud lie a median 15.6 µm apart, nearest point to nearest point. Caches now carry a version, so the file-order one (no version) is rebuilt on first use, never read; rematched_registration returns registration.

file where from licence
ccf/annotation_25.nrrd download.alleninstitute.org/.../annotation/ccf_2017/annotation_25.nrrd Allen Institute, research use with attribution
ccf/structures.json api.brain-map.org structure graph 1 Allen Institute, same
ccf/registration_<subject>.npz built from the public releases above (Complete_annotated, ccf_space_reconstructions) —
ccf/depth_{pia,tract}_25_v1.npy built from the annotation (exaproof ops depth-maps --build, 3.7 min, ~2 GB peak; E1224, RESULTS §1049): uint16 CCF µm to the nearest voxel outside the brain and to the nearest fiber tracts voxel, read by op-read-depth-layer derived from the annotation; same

Downloaded to the data root on first use and never redistributed from here.

Whole-brain affines for the agent's CCF read (agentic/ccf_affine.json, 2026-11-10, E1215). Three brains that had no map now have one. 721332: registration("721332") builds from its reconstructions_2026-05-06 / 2026-08-28 ccf_space_reconstructions (494,011 pairs over 22 neurons, 587 s; the cache is ccf/registration_721332.npz); held out one neuron at a time the affine errs a median 189.8 um (p90 377.5, leaf region agreement 0.586), the worst registration brain. 794495, 820994: the CCF-registered soma list (soma_detection/soma_locations.csv with xyz_ccf_auto) sits in an EARLIER processed asset than the imagery (_processed_2026-01-29_22-02-08, _processed_2026-05-20_16-45-09, beside ccf_alignment/), so io.somas had cached the morphology bucket's list, which has the same rows and no CCF column; io.somas._published now tries the subject's other _processed_ assets before the bucket, and the two caches were replaced (the old files are kept as somas/<id>.csv.morph.bak and somas/morph_lists_2026-11-10/). Held out one soma at a time: 794495 93.4 um (p90 209.2, agreement 0.714, 119 somas), 820994 89.1 um (p90 156.2, 0.742, 124 somas). Built by context_ops.build_ccf_affine; outputs scratchpad/e1215/ccf_affine_new.json.

Tile seams (io/tile_seams.py, 2026-11-10, E1215). The fused volume's tiles and their aligned blocks, from the XML the fusion ran on (fusion/processing.json's s3_xml_path, else tile_alignment/bigstitcher_affine.split.xml, tile_alignment/ip_split_affine_alignment/bigstitcher_affine.split.xml, tile_alignment/rhapso/..., else the session's _alignment_ assets), accepted only when the views' bounding box matches the fused level-0 shape within 16 voxels (all built maps: within 1.8). Built for 709221, 718162, 721332 (from _alignment_2025-04-29_13-08-50), 789202, 794491, 794492, 794495, 802449, 820994 and 841260; the fusion mode is recorded only on 794495 (LOWEST_VIEWID_WINS) and 820994 (AVG_BLEND). No map: 685221 and 685222 (none of the 55 / 9 XMLs in their _alignment_ assets spans the fusion_2024-07-22 / 2024-07-29 shape; closest 583 / 134 voxels off: the registration those fusions ran on is not published), 794493 (its XMLs are 1,718+ voxels off), and the 2023 fusions 609281, 648434, 651324, 653158, 653980 (no alignment XML in any of their assets). RESULTS 1031's site B5 (794495 N008, "the fibre stops at a vertical tile seam") reads 0.8 um from the tile 1 / tile 5 seam; a random point of the volume is within 10 um of a tile seam 0.6 % of the time.

2d. What the upstream segmentation is, and what is not published (2026-09-18)

The fragments this project joins come from a flood-filling network pipeline: the Allen Institute runs FFN-based segmentation with GNN post-processing on exaSPIM in collaboration with Google Applied Sciences (platform page). An FFN's own output is a voxel segmentation — a recurrent 3D convolutional network that "directly produces individual segments from a raw image", one object at a time, by moving a field of view to the frontier of what it has accepted (Januszewski et al. 2016, Nature Methods 2018).

Those voxels are not published. Searched on 2026-09-18 across the whole aind-open-data bucket — 69,395 top-level prefixes, of which 922 are exaSPIM assets — no prefix anywhere matches segment, ffn, flood or mask. The exaSPIM collection publishes only these kinds:

assets kind
337 raw acquisition
192 alignment
168 processed
56 fusion
48 flatfield-correction
20 reconstructions
2 each fused, stitched, raw-fusion

The only segmentation output is <fusion>/seg/*.zip, which is SWC skeletons. The precomputed/ and precomputed_neuroglancer/ folders under the reconstructions prefix look promising and are not it: they are the gold tracings as Neuroglancer skeletons.

Nor are they in aind-msma-morphology-data (re-searched 2026-09-24). Every store there that looks like a voxel volume was opened and turned out to be something else:

prefix what it actually is
segmentation/exaSPIM_<subject>_zarr/ (28 subjects) SWC archives only, despite the name (§2e)
685221-precomputed-test/ its info declares a uint64 segmentation at 1 µm, but no scale data was ever written; only skeleton/ and segment_properties/ exist
anna.grim/image-carveouts/{802449,754612}/…/mask.zarr binary 0/1 masks, likely drawn from gold tracings (§2e)
EXASPIM25/{train,test}/<brain>/…/label_mask.n5 ground-truth voxel labels, not FFN output. metadata.json records an SNT Dijkstra fill grown from the human tracing (fill_method: dijkstra, fill_threshold: 0.135)
exaspim_whole_brain_masks/exaspim_<subject>_medsam2/ whole-brain tissue masks from MedSAM2 on a downsampled image

The EXASPIM23/ prefix in the 2026-09-18 index has since been deleted, and EXASPIM25/ replaces it. By its README it is a segmentation benchmark: 1024³ blocks (some smaller), each with an input.n5 image, a uint8 label_mask.n5 and gold swcs/, at 0.748 × 0.748 × 1.0 µm. Counted: 60 labelled blocks, split into train 653158 (23), 609281 (11), J141R (10), 650010 (6), 613814 (4) and 651324 (2), and test 650010 (2) and 651324 (2). The README says 61. This is the only voxel ground truth for brains this project measures (653158, 650010). It would let our ROI/flood field be scored voxel for voxel. It does not provide an FFN mask to compare against.

Read with exaproof.io.exaspim25 (2026-09-24). fetch(brain) copies labels, SWCs and metadata, never images, to external/exaspim25/. blocks(brain) reads them. The labels are instances: label k + 1 is SNT_Data-000k.swc. A block voxel is chunk_origin + local on our level-0 grid.

This was verified on both 653158 and 650010: the block's input.n5 phase-correlates with our fused-denoised at zero shift, and every SWC node lands on its label. 653158's blocks overlap: its 23 blocks form 16 groups (overlap_groups). Scored first in §704 (the flood tracer).

Only three of the six brains can be scored on our volumes (checked 2026-09-25, e680_voxel_panel.py --verify, which phase-correlates each block's input.n5 with our volume at two SWC sites per block):

brain blocks on our grid? why not
653158 23 yes —
650010 8 yes (r 0.71–1.00, zero shift) —
651324 4 yes (r 0.86–1.00, zero shift) — but its train blocks are not labelled k + 1: block_000 is k + 2, block_001 2k + 1. Read the label from the fill: exaspim25.label_of
609281 11 no cut from single raw tiles (image_url = tile_x_…), voxel_spacing in µm, block z at a 0.748 µm pitch. A sub-study placed all 11 on the fused octree (per-block offsets, r 0.8–0.95; not in the repository), but 609281's fused.zarr is deleted and reads zeros, so no model can run there
613814 4 no labelled on the 2023-03-18 acquisition, whose fused.n5 is gone (octree only); the 2023-04-15 volume ImageSource opens is another acquisition (r ≈ 0)
J141R 10 no no volume in aind-open-data; raw-tile coordinates

Block.on_fused_grid is False for the raw-tile blocks.

Consequences, which are load-bearing for several results here.

  • Calibre cannot be read from the corpus: every fragment node carries radius 1.0 (§244). Thickness has to be measured from the imagery (features.caliber) or from our own model's object (roi.flood.calibre).
  • There is no upstream voxel mask to compare our field against, so "our segmenter does better locally than the FFN" cannot be tested on equal terms — only against the skeletons, which is a different comparison.
  • The FFN variant, its training data and its thresholds are, as far as can be found, unpublished for this application: Viren Jain's publication list has no light-sheet or expansion-microscopy work, and the exaSPIM pipeline is described only in Allen Institute material, not in a paper.

Worth asking the data owners, since the voxels demonstrably existed (the skeletons were derived from them): whether the FFN segmentation volumes were retained internally.

reconstructions/precomputed/segment_properties/info names 42 neurons with the annotator in the label — N001-653158-AW, N006-653158-consensus, N011-653158-PL, N070-653158-DT, N015-653158-SK. That independently confirms §334 (the gold is one person's tracing, and which person varies) and shows there are more annotators than AW and PG: at least PL, DT and SK.

3. Verifying a copy (what was run on sandbox-04)

export EXAPROOF_DATA=/home/ubuntu/data/exaproof
python3 -m pytest -q                      # 105 tests
exaproof data status                      # per-subject presence table
# the end-to-end reproduction — expect median cov@60min 0.162:
exaproof session --subject 653158 --model a10 --threshold 0.60 \
    --long-radius 250 --long-cone-deg 45 --asker askdyn

sandbox-04 (172.30.3.53, NVIDIA L4) verified 2026-08-27: full tree copied with sizes matching the source, including the same-day 653980/648434 artifacts. The cache/ copy came across too, so no S3 warm-up is needed there.

Imaging: exaSPIM versus MouseLight, and the coordinate space every external source must be converted into

Our coordinates are EXPANDED micrometres. exaSPIM images tissue after ~3× physical expansion, samples it at 0.748 × 0.748 × 1.0 µm in the expanded specimen, and the gold arbors were traced on that image (Janelia Workstation; # ORIGINAL_SOURCE Janelia Workstation Large Volume Viewer). Verified 2026-09-16: gold arbors span up to 43 × 17 × 25 mm, against a native mouse brain of ~13 × 8 × 11 mm. Every external corpus below is in TISSUE (native) micrometres, so any distance, curvature or rendered morphology taken from one must be scaled by the expansion factor (~3, the published value; 648434's 43 mm span suggests ≥ 3.2 for that brain) before it is compared with ours. Angles are scale-free; lengths are not.

exaSPIM (this project) MouseLight (Janelia)
modality axially swept light sheet, 3× expanded tissue (Glaser et al., eLife) serial two-photon tomography, point-scanning, vibratome block-face (Economo et al., eLife)
sampling, specimen 0.748 × 0.748 × 1.0 µm (expanded) 0.30 × 0.30 × 0.99 µm (tissue)
sampling, tissue units ~0.25 × 0.25 × 0.33 µm 0.30 × 0.30 × 0.99 µm
optical resolution, tissue units ~0.5 µm lateral, ~1.0 µm axial 0.45 µm lateral, 1.33 µm axial
label sparse Cre-dependent AAV, GFP; one channel in our data sparse Cre-dependent AAV, eGFP / tdTomato; two channels
clearing hydrogel expansion + delipidation (dilutes the fluorophore) DMSO / sorbitol refractive-index matching (n = 1.468), gelatin embedding
typical artefacts tile seams, refractive-index inhomogeneity at depth, uint16 saturation (609281) a section boundary every ~250 µm in z, tile seams, uint16 saturation (65,520)
automated fragments FFN skeletons machine segmentation SWCs (segmentation/)
coordinates expanded µm, image space image µm in the imagery bucket; CCF µm in the NeuronBrowser export

Effective resolution in tissue is similar; the sampling and the physics are not. MouseLight is sampled 3× coarser along z, its PSF is a two-photon point-scan's rather than a light sheet's, and its noise and background come from a different contrast mechanism. To put a MouseLight patch on our grid, resample by ~1.2× in x and y and ~3× in z (tissue → expanded).

MouseLight imagery on AWS Open Data — CC BY 4.0

s3://janelia-mouselight-imagery (anonymous; registry). Licence Creative Commons Attribution 4.0 International; required citation: "Mouse Brain Anatomy: MouseLight Imagery was accessed on [DATE] from https://registry.opendata.aws/janelia-mouselight." (first accessed 2026-09-16). Contents: Horta KTX octrees and low-resolution TIFFs for 11 brains; machine-segmentation SWCs; consensus tracings; and, for brain 2018-08-01, a full-resolution carveout, carveouts/2018-08-01/fluorescence-near-consensus.n5/ volume-rechunked — uint16, gzip N5, 33,792 × 25,600 × 13,312 × 2 voxels (x, y, z, channel), blocks 128 × 128 × 64, stored only near the 133 traced neurons (all green channel), with the tracings in the same image space (transform.txt: origin and level-0 voxel size in nm, divided by 2^(nl−1) for full resolution). Verified: a traced node reads 65,520 in channel 0 against a block median of 12,339.

Gold QC — stretches traced into two neurons (rows 244, 245, 2026-10-24)

Seven of the eight gold brains have axon traced into two gold neurons along one path (685221 113.9 mm, 651324 99.5 mm, 685222 10.0 mm, 653158 8.2 mm, 789202 1.3 mm; 609281 and 653980 under 1 mm); the held-out 648434 has none. The per-brain table, the renders, what it means for scoring and training, and the questions for the annotators are in docs/GOLD_QC.md (repository file); the opt-in filter is exaproof.gt.shared.SharedGold.

Leakage register — released datasets that contain OUR brains (row 229, 2026-10-23)

Allen Institute papers release data from the mice this project scores. Before any released morphology set or image set is used as a reference, a prior, training data or a benchmark, check its subject ids here (docs/METHOD.md rule 148). The KB carries the same ids as each source's or paper's data_subjects; the #234 guard drops those items for those brains.

release what it is subjects verdict evidence
su2026: Su et al. 2026, Nature, doi 10.1038/s41586-026-11026-0. Code: AllenNeuralDynamics/aind-LC-morphologies. Data: Code Ocean asset LC_Snapshots 12f47d79-…; public copies at s3://aind-open-data/exaSPIM_<subject>_…_reconstructions_2026-05-06_*/ccf_space_reconstructions 132 LC-NE whole-axon reconstructions, exaSPIM, CCF 648434 (12), 685221 (60), 685222 (38), 721332 (22) OUR GOLD on all four. Never a prior or reference for these brains; usable held out by brain for the others (120 neurons for 648434) The notebooks print "LC Neurons: 132 Mice: 4" with those samples. The neuron ids are ours (N004/N011/N020/N023/N024-648434). The snapshot …LC-manuscript_2026-05-06_10-44-00 matches our reconstructions folders of 2026-05-06 13:28–13:48. Somas equal the paper's table to 0.1 µm. CCF axon length is 1–3 % under the paper's. <data>/external/su2026/PROVENANCE.txt
grim2025: Grim, Chandrashekar, Sümbül, arXiv 2501.01022 (supervoxel loss) its "new public 3-D light-microscopy mouse-brain dataset" is s3://aind-msma-morphology-data/EXASPIM25 (paper footnote 3), which this repo reads as external/exaspim25 (§2d). Voxel labels are SNT fills grown from the human tracings 653158 (23 blocks), 609281 (11), J141R (10), 650010 (8), 613814 (4), 651324 (4); none from 648434 OVERLAPS 653158, 609281, 651324 (gold brains) and 650010, 613814 (registry subjects). Independent of 648434. Its labels are our tracings' fills: never a held-out test for those brains Paper §Experiments and footnote 3 (<data>/external/leakcheck/grim2025.pdf); block counts from the bucket index (§2d)
glaser2025: Glaser et al., eLife 2025, the exaSPIM paper (papers/exaspim_2025_elife.pdf) exaSPIM imaging datasets exaSPIM_708373, exaSPIM_671477 (mouse), Z13288-QN22-26-036 (macaque), H17.24.006-CX55-31B (human) INDEPENDENT of every gold brain, 648434 included. Our registry has no gold for 708373 or 671477 (Table A) Its "Data availability" statement. None of our nine ids appears anywhere in the text

Our own models are not held out everywhere (row 287, E906, 2026-10-25). The deployed instance model inst_s9_ho648434_bndffn is held out of 648434 ONLY. Its e726.json lists its training brains as 653158, 653980, 789202, 685222 and 685221, with 178 soma blocks; 685221 alone contributes 60 soma blocks (e716). Any read of it on 685221, 685222, 789202, 653158 or 653980 is in-sample, and has to be labelled so. A held-out read on the LC brains needs a model retrained with both 685221 and 685222 left out. Done by row 300 (E919, 2026-10-25): inst_s9_ho648434-685221-685222_bndffn is held out of all three LC-NE brains (267 blocks of 653158, 653980, 789202, 609281, J141R, 613814, 651324; e726.json carries leak_check), 434 s for the same 1,500 steps; its first reads are RESULTS §813. It is a weaker model (80 of 178 soma blocks), and the deployed model stays in-sample on 685221 and 685222. The fragment stores are not ours (row 287 provenance): 685221 is a GNN-corrected FFN whole-brain run, 685222 a GNN-corrected run, 653158 / 653980 / 648434 FFN skeletons from the fusion asset, and 789202 a U-Net run.

Independent LC-NE sources (subjects not ours), checked for row 229:

  • The Allen Mouse Brain Connectivity Atlas has 5 Dbh-Cre_KH212 injections including LC: 519735413, 480689656, 159510205, 159942862 and 287029894. They are downloadable as 25 µm projection density (external/allen_connectivity/), under Allen Institute terms of use. Row 229's E849b read them as a tract field: NULL-HARM.
  • schwarz2015 (TRIO) and kebschull2016 (MAPseq on LC) release region-level projection data, not single-neuron paths.
  • mckinney2023 (Dryad) has local slice morphologies only.
  • MouseLight has 0 somas in LC (1 within 200 µm, AA1505).

External morphology corpora — licences and attribution

Neither of these is imagery. Both are skeletons only, which is what makes them usable: a branching angle is cheap to store, and §216 shows one half of it is a property of the neuron rather than of the instrument that recorded it. Each carries a licence that permits reuse with attribution, and that attribution is recorded here because the licence requires it and because a corpus whose provenance is not written down is a corpus nobody can check.

corpus path licence what it is
CORAL external/coral_swc (32 neurons, 87 MB) CC BY 4.0 Whole-brain light-microscopy tracings, hand-annotated. §135's branch-angle prior comes from these.
MouseLight external/mouselight_swc (1,651 neurons, 340 MB) CC BY-NC 4.0 — non-commercial; use in this repository was authorised by the repo owner on 2026-09-15 Janelia whole-brain serial two-photon tomography single-neuron reconstructions with complete axonal arbors (an earlier version of this row said light-sheet; that was wrong). The only external corpus with whole-brain axons and public full-resolution imagery (section above). Fetched 2026-09-16 from ml-neuronbrowser.janelia.org (GraphQL listing, POST /export format 0; scratchpad/fetch_mouselight.py). Coordinates are TISSUE micrometres in CCF space (verified: node step median 8.6 µm, arbor extent ~3.4 mm) — multiply by the expansion factor (~3) before comparing any length with exaSPIM's expanded coordinates, node types 1/2/3 = soma/axon/dendrite; each file's header carries its own DOI. Nothing derived from it is redistributed, and no commercial use.
MICrONS external/microns_swc (300 neurons, 129 MB) CC BY 4.0 minnie65 v661 skeletons from the IARPA MICrONS cortical mm³ electron microscopy volume, machine-derived from a volumetric segmentation at 1.5 µm. Retrieved 2026-09-13 from s3://bossdb-open-data/iarpa_microns/minnie/minnie65/skeletons/v661/skeletons/, selected by file size (the 300 largest). The bulk v661 set is documented as automatically reconstructed with little to no human proofreading, so the selection was checked against meta/ax_proof_v661_sids.csv: 270 of the 300 are axon-proofread, and the other 30 carry no axon bifurcations at all (§216). Cite per the MICrONS Explorer citation policy.

What they are for, and what they are not for. MICrONS is not training data for this segmenter: electron microscopy has no fluorescent label and an unrelated contrast model, and no downsampling makes it look like light-sheet. It is an independent morphology corpus, and §135's entire prior rested on 32 CORAL neurons — MICrONS supplies 33,555 axonal bifurcations against CORAL's 4,899, from a reconstruction process with none of the same failure modes. §216 uses exactly that independence: sibling divergence agrees to 7.2° across both of these and all three exaSPIM brains, while the continuation turn does not agree at all and tracks how the skeleton was made.

Sources checked and their status, so nobody re-opens a closed question

source licence status
Brain Image Library (BIL) — BICCN whole-brain fMOST single-neuron SWCs (Zeng lab, Allen Institute / Luo) CC BY-SA 4.0, re-confirmed verbatim 2026-09-14 on brainimagelibrary.org/download.html: "BIL Data is distributed under the Creative Commons Attribution-ShareAlike 4.0 International License." Attribution required; derived datasets carry CC BY-SA. RECOVERED AND IN USE (2026-09-14, ~12:25 PDT). download.brainimagelibrary.org answers again (the morning outage was the data tier; doi. still returns 500). /biccn/zeng/luo/fMOST/cells/mouseID_*/1/*.swc holds 5,094 complete reconstructions across 32 mice, standard SWC with compartment types (radii all 1.0, like ours) — but coordinates are anisotropic voxel indices of the fMOST volume, not µm (lateral ≈ 0.30–0.35 µm, axial ≈ 1 µm; one mouse's terafly grid is 35,000 × 22,793 × 10,553): scale before any metric use (RESULTS §269 note, E347). A sample of up to 10 per mouse — 311 SWCs, 477 MB — is at /home/ubuntu/data/exaproof/external/bil_fmost_swc/ with PROVENANCE.txt. First use: §236-style branch-angle prior comparison (E339) — read in §269: continuation turn p75 26.4° (pipeline-dependent, as §216), sibling p5 16.8° (the one corpus whose tail departs). Attribution: Brain Image Library, BICCN fMOST reconstructions, CC BY-SA 4.0; nothing derived from it is redistributed.
SELMA3D challenge data, MICCAI 2024 Light-sheet, but blood vessels, c-Fos cells, Aβ plaques and nuclei — not neurites. Wrong structure for a morphology prior.
Allen Institute Cell Types Database public API, no download Used (§236). api.brain-map.org/api/v2/data/query.json?criteria=model::NeuronReconstruction publishes average_bifurcation_angle_local for 701 reconstructions / 19,426 bifurcations — a fourth modality (patch-clamp slice) for two HTTP calls and no bulk transfer. Median 74.5°, inside the 72.2–83.3 range the other five corpora span.
Allen Institute full-morphology (fMOST, CCFv3-registered) not checked A different release from the row above — whole-brain fMOST rather than slice. Probed 2026-09-14: api.brain-map.org is up and answering (HTTP 200), and ApiCellTypesSpecimenDetail filtered on nr__reconstruction_type = 'full' returns 367 rows — but those are the patch-seq/slice reconstructions of the row above, not the whole-brain fMOST release. The fMOST release is not served by this API and its access path is still unexplored.
Janelia MouseLight see the licensed-corpora table above NOW IN USE (2026-09-15, owner-authorised): 1,651 SWCs under external/mouselight_swc. The NC term stands — internal research only, nothing redistributed, no commercial use.

The trap in that row, recorded because it would have caught me. Janelia's general open-science page states "CC BY 4.0 terms for Janelia data" with a link to the CC BY 4.0 deed — and hedges it with "Most (but not all) of such resources". MouseLight is one of the "not all": its own terms page carries the NC variant. Read the project's page, not the institution's.

| NeuroMorpho.org (v2 API) | per-archive; site terms require citation | Reachable, not used. neuromorpho.org/api/neuron?page=0&size=1 answers HTTP 200 with JSON. It aggregates hundreds of archives with per-neuron provenance and per-archive terms, so it is a place to find corpora rather than a corpus, and any use has to resolve the licence of the specific archive drawn from. Noted as a path, not opened. |

s3://aind-msma-morphology-data is a different matter and is covered above: anonymously readable, not part of the AWS Open Data Program, and carrying no licence. It is internal, and nothing is built on it or redistributed from it.