Explaining Human Preferences via Metrics for Structured 3D Reconstruction

2025/03/11 by Jack Langerman, Denys Rozumnyi, Langerman, Jack +5 · 2 voices · 2 citations
#cs.CV

paper · pdf · doi:10.48550/arxiv.2503.08208

Abstract

"What cannot be measured cannot be improved" while likely never uttered by Lord Kelvin, summarizes effectively the driving force behind this work. This paper presents a detailed discussion of automated metrics for evaluating structured 3D reconstructions. Pitfalls of each metric are discussed, and an analysis through the lens of expert 3D modelers' preferences is presented. A set of systematic "unit tests" are proposed to empirically verify desirable properties, and context aware recommendations regarding which metric to use depending on application are provided. Finally, a learned metric distilled from human expert judgments is proposed and analyzed. The source code is available at https://github.com/s23dr/wireframe-metrics-iccv2025

Citations

• A distance measure between attributed relational graphs for pattern recognition
• RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS

Cited by

• S23DR 2026 Winning Solution
• Edge Prediction for Roof Wireframe Reconstruction with Transformers

Discussions

• Explaining Human Preferences via Metrics for Structured 3D Reconstruction @jacklangerman.bsky.social Denys Rozumnyi, Yuzhong Huang, @ducha-aiki.bsky.social tl;dr: we asked 3D modelers to rank wire [bsky, 28 points, 1 comments]
• 📊 NEW Evaluation Metric Say goodbye to WED. Hello to a human-aligned combo of: • Vertex F1 • Edge IoU Backed by this paper: 👉 arxiv.org/abs/2503.08208 [bsky, 0 points, 1 comments]

Related

• DumbleDR: Predicting User Preferences of Dimensionality Reduction Projection Quality
• Textured Geometry Evaluation: Perceptual 3D Textured Shape Metric via 3D Latent-Geometry Network
• 3D Reconstruction using Structured Light from off-the-shelf components
• Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluation
• 3D Arena: An Open Platform for Generative 3D Evaluation