← mep

Cheatsheet

the whole project on one page · every number, every verdict, every caveat · surface-coverage recall@K=1 · 5 seeds · n_cand=20 · hybrid_top_k=12 · mean ± SE across seeds

The problem in one paragraph

Scanning a building means deciding where to point the scanner next to find its hidden MEP (mechanical, electrical, plumbing), the pipes, ducts, and equipment buried inside a BIM that you cannot see from outside. This is the next-best-view (NBV) problem. The classical answer is OctoMap-IG information-gain view selection (Bircher et al., ICRA 2016). We ship a learned-ranker plus classical-lookahead hybrid: the model scores M candidate poses, the top-K=12 go into a 2-step lookahead. Walk through what one scan captures →

The result, in three lines

Hybrid surface recall, test_locked = 0.541. Beats classical OctoMap-IG by +5.4σ / +5.0σ / +5.6σ on all three splits (test_locked, held_out, OOD), every cut a real win, at ~35% the cost of exhaustive greedy_lookahead_1.

OOD specialist v7_realsynth = 0.071 on unseen buildings, +0.0088 (2.6σ) over shipped v4, 10W / 0L / 40T. Honest: n=3 buildings, small magnitude, read it as an ordering result.

The point. The headline is not a SOTA number. It is that corpus diversity beats target engineering (3 target tricks died, 1 data fix won), and that we found and fixed a metric pathology that inflated our own oracle 5 to 50×.

The headline table: 3 splits × key policies

surface-coverage recall@K=1, hybrid policy · mean ± SE, 5 seeds · bold green = best non-oracle in column · red = the floor / a failed version · oracle = myopic Δ ceiling (greyed) · scoreboard →

policy	test_locked 3 residential	held_out 7 GNI / heatpump	ifc-bench OOD 10 scenes, 3 buildings
random no policy	0.441 ±.021	0.068 ±.008	0.031 ±.004
grid fixed ring	0.416 ±.005	0.057 ±.007	0.023 ±.004
greedy_coverage classical	0.563 ±.004	0.062 ±.002	0.037 ±.007
octomap_ig Bircher 2016 · the floor	0.500 ±.006	0.054 ±.002	0.026 ±.004
greedy_lookahead_1 2-step, true extractor	0.494 ±.017	0.129 ±.010	0.070 ±.006
v4 mep_recall · SHIPPED	0.541	0.147	0.063 ±.002
v6 pure pairwise rank	0.507 ±.016	0.129 ±.009	0.059 ±.008
v7_attn cross-attention · DROP	0.496 ±.012	0.088 ±.008	0.067 ±.011
v7_realsynth realism-fixed synth	0.486 ±.019	0.094 ±.009	0.071 ±.003
v7_k12 K=2 combined · DROP	0.474 ±.016	0.086 ±.011	0.054 ±.005
oracle myopic Δ ceiling	0.572 ±.003	0.185 ±.017	0.093 ±.006

Best-in-column among directly-comparable non-oracle policies: greedy_coverage on test_locked, v4 on held_out, v7_realsynth on OOD. v4 is listed on ifc-bench OOD from the same fixed-eval run; its test_locked 0.541 / held_out 0.147 come from an earlier run and are cited that way across the site. Oracle is the myopic Δ ceiling, greyed.

The sigma margins

hybrid over OctoMap-IG

+5.4 / +5.0 / +5.6σ

vs the classical info-gain planner (Bircher 2016), on test_locked / held_out / OOD. Every split a real win.

v7_realsynth over v4 · OOD

+2.6σ

+0.009 ± 0.0035 surface on ifc-bench OOD, 10W / 0L / 40T. Real, but a specialist.

learned ranker alone vs greedy_coverage

−7.5σ

test_locked, −0.075 ± 0.010. The model alone is well below the classical baseline. The lookahead wrapper is what ships.

σ uses combined SE = √(SEₐ² + SE_b²) on paired-per-seed diffs (4 dof). OOD has only 3 unique buildings (effective n ≈ 15), so read OOD as an ordering result, not a calibrated recall.

The negative results, one line each

full ledger →

v5 (surface-coverage target): regressed below v4 on every cell. DROP
v7_attn (2.6M-param cross-attention head, ~10× params): ρ tied v6 at 0.13, no real gain, regressed hybrid off-domain. DROP
v7_k12 (K=2 pair-coverage target): mathematically degenerate, the (i,j)=(j,i) pair symmetry gives exactly 0 top-2 gap and zero gradient. The combined K2+0.1·K1 fix broke ties 10/10 but eval still regressed (ρ 0.10, OOD 0.054 < v4 0.063). DROP
The learned ranker alone never beats greedy_coverage (e.g. v6 learned 0.430 vs greedy 0.563 on test_locked). The classical lookahead wrapper is what ships. KEY
Metric pathology: recall once divided by instances visible in the partial cloud, so seeing 1 instance scored 1.0 and the oracle inflated 5 to 50×. Re-anchored to scene-full GT: on gni_model_173 oracle dropped 1.0 → 0.0093. FIXED

Scans to coverage

hybrid vs OctoMap-IG · scans to 80% of achievable

~2.3 vs ~5.1

To reach 80% of one-shot-achievable surface coverage of the hidden MEP, the hybrid needs ~2.3 scans versus ~5.1 for the classical OctoMap-IG baseline (about 2.85 fewer, paired over 3 seeds, robust). OctoMap-IG fails to reach 80% within 6 scans in 6 of 9 scene-seed cells.

honest hedge · vs strong classical greedy

tied

Against the strong non-learned greedy baselines the hybrid is essentially tied: greedy_lookahead_1 within noise (~2.3 scans), exhaustive greedy_coverage marginally ahead (~2.0). On this small dense in-distribution set the honest headline win is over OctoMap-IG, not over every classical method.

K=6 surface-coverage rollouts, 3 seeds, test_locked (ahu_heating, b03_heating, boiler_gasradiator). scans-to-X% is relative to each scene's oracle@K=6 ceiling.

policy	cov @ K=1	cov @ K=6	scans → 50%	scans → 80%
hybrid_learned_lookahead shipped	0.519	0.907	0.96	2.27
greedy_lookahead_1 2-step	0.530	0.920	0.96	2.34
greedy_coverage classical	0.589	0.915	0.85	2.00
octomap_ig Bircher 2016 · the floor	0.500	0.713	1.52	5.12
random no policy	0.443	0.782	1.32	4.28
oracle exhaustive Δ ceiling	0.592	0.923	0.84	1.98

The color key

green = a real win / what ships red = dead / regressed / negative result amber = caveat / tied / honest hedge periwinkle = neutral method / info

What ships, what does not

SHIPS deployable

v4 + hybrid default policy: 0.541 test_locked, 0.147 held_out.
v7_realsynth as the OOD specialist: 0.071 (+2.6σ) on unseen buildings.
The OctoMap-IG baseline integration (the thing we beat).
The scene-full GT metric fix (de-inflates the oracle).
The scanner-z anchoring + raycast floor filter pose fix.

DOES NOT SHIP cut

The learned ranker as a standalone planner (loses to greedy_coverage).
v5, v7_attn, v7_k12 (all regressed or no-gain).
Any claim of SOTA on a public benchmark (none exists for NBV over BIM).
Any OOD claim stronger than "ordering result at n=3 buildings".

Glossary

term	what it is
MEP	Mechanical, electrical, plumbing: the pipes, ducts, and equipment hidden inside a building.
BIM	Building information model, the digital 3D model of the building and its systems.
IFC	Industry Foundation Classes, the open file format BIM models are exchanged in.
NBV (next-best-view)	Deciding where to point the scanner next to reveal the most new structure.
recall@K=1	The headline metric: fraction of hidden MEP covered after a single chosen scan.
surface coverage	Class-agnostic recall over the MEP surface voxels, vs the geometric-extractor `mep_recall`.
hybrid	`LearnedPlusLookaheadBaseline`: model scores M candidates, top-K=12 go into a 2-step lookahead.
OctoMap-IG	The classical info-gain NBV planner (Bircher et al., ICRA 2016), the baseline we beat.
oracle	The myopic Δ ceiling: the best single next pose by hindsight, the gap a perfect ranker could close.
test_locked / held_out / OOD	The 3 splits: 3 residential / 7 GNI-heatpump / 10 ifc-bench scenes across 3 unseen buildings.
Spearman ρ	Rank correlation between the learned score and the true Δ-recall target (weak everywhere: the target is hard).
σ (combined-SE margin)	Paired-per-seed diff divided by combined SE = √(SEₐ² + SE_b²); 2σ is the bar for a real effect.
SHIP / MEH / DROP	The honest per-experiment verdicts: shipped, no-gain, or regressed and cut.
hybrid_top_k=12	How many top-ranked candidates the lookahead refines; lower the model pre-filters more, raising toward M degenerates to greedy_lookahead_1.

What is still open

directions →

More, more-varied buildings: the bottleneck is corpus diversity, not loss formulation.
A shared public NBV-over-BIM benchmark, none exists today.
Topology / frontier-following NBV (open experiment).
Calibrated OOD intervals beyond the current n=3-building ordering result.

Eval protocol held fixed across the study: surface-coverage recall@K=1 · 5 seeds · n_cand=20 · hybrid_top_k=12 · mean ± SE across seeds. Splits: test_locked (3 residential) · held_out (7 GNI / heatpump) · ifc-bench OOD (10 scenes across duplex_mep, wbdg_office_mep, west_riverside_hospital_fire). Hybrid = LearnedPlusLookaheadBaseline (model scores M candidates, takes top-K=12, runs 2-step lookahead). Classical baseline: OctoMap-IG (Bircher et al., ICRA 2016). Corpus: 297 manifest scenes plus IFC-Bench v2 (sylvainHellin/ifc-bench, CC-BY, 93 sub-scenes). The scans-to-coverage table is a separate K=6 rollout, 3 seeds, test_locked. Every number reproduces from a committed script.