Context
The problem
Track the full 6-DoF pose of four rigid objects across a monocular sequence despite occlusion, symmetry, and reconstruction outliers.
System
Architecture
- 1,404-frame monocular sequence
- Point clouds and segmentation masks
- Camera intrinsics and template meshes
- Geometric and temporal pose optimization
- Mesh-rendered overlays and per-frame evaluation
Engineering judgment
Technical decisions
Compare geometric and temporal tracking approaches
Implemented and evaluated six approaches, including 24-start ICP, robust sequential tracking, quaternion smoothing, dynamic programming, and joint optimization.
Design around difficult object geometry
The pipeline explicitly addresses occlusion, symmetry, and reconstruction outliers that can destabilize object-pose estimates.
Evaluate stability as well as geometric fit
Measured silhouette IoU, Chamfer distance, and rotation-jump behavior alongside mesh-rendered overlay inspection.
Constraints
Challenges
- Occlusion can remove discriminative geometry from an individual frame.
- Symmetric objects can produce plausible but incorrect rotations.
- Reconstruction outliers can destabilize direct geometric alignment.
Result
Outcome
Reached 0.693 mean silhouette IoU and 3.12 cm mean Chamfer distance without rotation jumps above 60°.