teaser_video.mp4
This repository provides the official implementation of the paper PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verify.
This paper proposes a novel “auto-agglomerative” 3D fracture assembly method, PuzzleFusion++, resembling how humans solve challenging spatial puzzles.
Starting from individual fragments, the approach 1) aligns and merges fragments into larger groups akin to agglomerative clustering and 2) repeats the process iteratively in completing the assembly akin to auto-regressive methods. Concretely, a diffusion model denoises the 6-DoF alignment parameters of the fragments simultaneously (the Denoiser in the figure above), and a transformer model verifies and merges pairwise alignments into larger ones (the Verifier in the figure above), whose process repeats iteratively.
Extensive experiments on the Breaking Bad dataset show that PuzzleFusion++ outperforms all other state-of-the-art techniques by significant margins across all metrics. In particular by over 10% in part accuracy and 50% in Chamfer distance.
Please refer to the installation guide to set up the environment.
Please refer to the data preparation guide to download and prepare for the BreakingBad dataset, as well as downloading our pre-trained model checkpoints.
Please follow the test guide for model inference, evaluation, and visualization.
Please follow the training guide for details about the training pipeline.
If you find PuzzleFusion++ useful in your research or applications, please consider citing:
@article{wang2024puzzlefusionpp,
author = {Wang, Zhengqing and Chen, Jiacheng and Furukawa, Yasutaka},
title = {PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verify},
journal = {arXiv preprint arXiv:2406.00259},
year = {2024},
}
Our method is deeply inspired by PuzzleFusion and Jigsaw, and benefited from their open-source code. Please consider reading these papers if interested in relevant topics.
This project is licensed under GPL, see the license file for details.
