next-best-view planning for hidden MEP inside a BIM · a learned ranker wrapped in classical lookahead · with honest comparisons against what it replaces
We pick where to point a scanner next to uncover a building's MEP, the mechanical, electrical, and plumbing buried inside its structure. The shipped policy is a learned ranker plus classical lookahead hybrid (the ranker scores 20 candidate views, the top 12 go to a 1-step lookahead). It reaches 0.541 surface-coverage recall on the locked test set and beats the classical info-gain planner OctoMap-IG (Bircher et al., ICRA 2016) by +5.0 to +5.6 sigma on every split. The honest headline is not a SOTA number: the learned ranker alone never beats greedy_coverage, the classical lookahead wrapper is what actually ships, and corpus diversity beat target engineering.
| policy | test_locked (3) | held_out (7) | ifc-bench OOD (10) |
|---|---|---|---|
| greedy_coverage | 0.563 | 0.062 | 0.037 |
| octomap_ig classical info-gain | 0.500 | 0.054 | 0.026 |
| learned_joint (v4) ranker alone | 0.488 | 0.090 | 0.021 |
| hybrid (v4) ships | 0.541 | 0.147 | 0.052 |
| hybrid (v7_realsynth) OOD specialist | 0.540 | 0.146 | 0.061 |
| oracle (per-step) | 0.572 | 0.185 | 0.093 |
surface-coverage recall@K=1, mean across 5 seeds (SE ~0.005-0.018) · bold = best non-oracle in column · oracle is the myopic per-step ceiling, not a competitor
The whole project on one page: the headline table across 3 splits, the sigma margins, the negative-results one-liners, the color key, and the what-ships / what-does-not verdict.
Every model version and ablation with dates, motivation, and verdicts: the v5 / v7_attn / v7_k12 regressions documented in full.
The NBV field (PC-NBV, OctoMap-IG / nbvplanner, NBV-Net, MA-SCVP, NeU-NBV, GenNBV), honestly tagged by what is comparable.
Interactive: scenes, candidate poses, captured MEP instances, and what recall@K=1 actually measures on a partial point cloud.
Pipeline as a node graph: pose sampling, raycast capture, the joint head, the lookahead wrapper. Click any node for what it is and the command behind it.
Compact dashboard: every policy by split, the OctoMap-IG margins in sigma, the OOD win, and the negative-results graveyard.
The specific human-in-the-loop calls and references that steered each experiment, and which ones changed the result.
Full writeup: NBV for MEP, the hybrid policy, the metric pathology, the corpus-vs-loss finding, and the OOD result with citations.
It beats the classical info-gain planner. To scan a building you have to decide where to point the scanner next. The classical answer is OctoMap-IG: pick the view with maximum expected information gain. The hybrid beats it by +5.4 sigma on test_locked, +5.0 sigma held-out, and +5.6 sigma on out-of-distribution buildings, every split, not one lucky cut. Concretely, to reach 80% of the surface coverage achievable in one scan, the hybrid needs ~2.3 scans versus ~5.1 for OctoMap-IG (paired over 3 seeds, robust; OctoMap-IG fails to reach 80% within 6 scans in 6 of 9 scene-seed cells). We scope that claim to OctoMap-IG on purpose: against the strong non-learned greedy baselines the hybrid is essentially tied (greedy_lookahead_1 within noise, exhaustive greedy_coverage marginally ahead). The honest win is over the classical info-gain planner, not over every classical method.
It generalizes to unseen building types. The hard test is a building type the model never trained on. On ifc-bench OOD (10 scenes across a duplex, an office, and a fire-damaged hospital), the OOD specialist v7_realsynth scores 0.0714 ± 0.003, the best of any version, beating the shipped v4's 0.0626 by +0.0088, which is +2.6 sigma over 10 wins, 0 losses, 40 ties across 50 paired cells. The realism-fixed synthetic corpus transfers where target tweaks do not.
The metric pathology. The recall metric originally divided captured instances by the instances visible in the partial cloud, using the partial cloud as its own ground truth. So observing one instance scored 1.0, and oracle inflated 5-50x. We re-anchored it to the scene-full GT (scene.instance_class): on gni_model_173 oracle dropped from 1.0 to 0.0093. The field standard is already to normalize against the complete scene ground truth, so this was a bug in our own pipeline and we make no claim that prior work is affected.
The corpus-vs-loss finding. Three target-engineering tricks all failed. The K=2 pair-coverage target g_star_k2 has a mathematically unavoidable degeneracy: pair (i,j) = (j,i) symmetry makes the two members of every winning pair tie, exactly 0 top-2 gap and zero gradient. The combined K2 + 0.1·K1 fix broke the ties but eval regressed (ifc-bench hybrid 0.054, below v4's 0.063). Meanwhile the one fix that won was a more diverse, realism-corrected synthetic corpus, which says the bottleneck is the data, not the loss.
A discipline that refuted its own findings. The exciting v7_realsynth OOD win only counted once we caught that v4 had never been evaluated on ifc-bench; the comparison was apples-to-oranges. We re-ran v4 on the same OOD scenes (0.063) before claiming the +2.6 sigma. The rule "match the eval before you believe the result" is what makes this number trustworthy, and three exciting target-engineering wins died under it.
The user set direction in plain language (try a+b, secondary audit, think about scope, is this interesting, get more target signal, match the eval), and an orchestrating LLM decomposed each call into self-contained briefs for specialized sub-agents that dispatched Modal training, ran paired-bootstrap eval, audited the metric, and committed results with honest SHIP / KEEP / DROP verdicts. The directions page traces the specific calls that shaped the outcome.
Eval protocol held fixed across the whole study: surface-coverage recall@K=1, 5 seeds, n_candidates=20, hybrid top_k=12, mean ± SE across seeds. Splits: test_locked (3 residential), held_out (7 GNI / heatpump), ifc-bench OOD (10 scenes, 3 buildings). Hybrid = the v_joint_v4 ranker scoring 20 candidates, taking the top 12, then running a 1-step lookahead. Classical baseline: OctoMap-IG (Bircher et al., ICRA 2016). Corpus: 297 manifest scenes plus IFC-Bench v2 (sylvainHellin/ifc-bench, CC-BY). Every number reproduces from a committed script.