Too Tiny to See: Hazardous Obstacle Detection Dataset and Evaluation

The figure above summarizes the 2T2S evaluation pipeline for assessing small, long-range obstacles across different sensing modalities. Starting from a measured point cloud P_d together with a high-fidelity ground-truth mesh for each lost-cargo object, we first place the mesh into the closest short-range capture (15m or 25m) to obtain a reliable initial pose. This reference alignment is then transferred to each evaluation distance using ICP, ensuring consistent object geometry across the full sequence.

Because long-range measurements are sparse and sensitive to small pose errors, we refine this alignment through a dedicated fine pose optimization that minimizes a weighted point-to-mesh objective. After alignment, we crop the measurement to isolate the object region and remove ground points, yielding the refined evaluation point cloud P_e. This ensures that evaluation focuses on the obstacle itself rather than being dominated by planar ground surfaces.

For computing the 2T2S metric, we uniformly sample the aligned ground-truth mesh to obtain P₀, and feed both P_e and P₀ through a shared point-cloud feature encoder. Intermediate feature activations from multiple layers are extracted, normalized, and compared in feature space; the aggregated discrepancy across layers forms the final 2T2S score. This produces a perceptual, modality-agnostic similarity measure that captures whether the reconstructed 3D structure preserves the semantic and geometric characteristics of the object, even under sparsity and incomplete observations.

The examples above demonstrate reconstructions from summer and winter scenes using different sensors. At long range and under varying environmental conditions, small lost-cargo objects are consistently difficult to recover in 3D. LiDAR sensors such as Luminar and Velodyne VLS-128 may register only a handful of returns on these tiny obstacles; despite this, conventional depth metrics can rate their predictions favorably because the few points they do observe are metrically accurate, even though the resulting geometry provides almost no usable shape information.

Image-based stereo and monocular methods show complementary failure modes: while they provide dense predictions, depth accuracy degrades with distance, causing small objects to collapse into the ground surface or disappear entirely. Across modalities and conditions, the reconstructions diverge significantly in how well they preserve object geometry, yet these differences are largely invisible to standard metrics dominated by road and background regions.

These observations further motivate the need for 2T2S: an evaluation method that captures object-level fidelity by comparing predicted point clouds to accurate ground-truth meshes in a perceptual feature space, rather than relying solely on global distance errors. This allows the evaluation to reflect the actual reconstruction quality of small obstacles that matter for safe autonomous navigation.

As shown above, the averaged, distance-binned evaluation results for all perception methods across both summer (S) and winter (W) recordings. As object distance increases, reconstruction quality diverges substantially across modalities; however, most standard depth metrics fail to reflect these differences in a consistent or meaningful way, frequently favoring LiDAR outputs because of precise but sparse range measurements that do not preserve object shape.

In contrast, the 2T2S metric aligns closely with the qualitative behavior observed in the data. By evaluating similarity in a learned feature space, 2T2S measures whether a reconstructed point cloud actually retains the geometric and semantic structure of a small, distant obstacle, rather than only matching its depth values on a few pixels or returns. This leads to stable and interpretable rankings across distances and conditions, with camera-based methods—particularly Gated RCCB Stereo—achieving the highest similarity to ground truth due to their ability to preserve object shape at long range.

These results demonstrate that 2T2S captures the cross-modality trade-offs between resolution and accuracy and provides a robust, object-level evaluation signal that existing depth metrics overlook. This makes 2T2S especially suitable for assessing reconstruction quality in safety-critical long-range perception scenarios.