Volumetric Shape Correspondence: Advancements in Mapping 3D Objects with Tetrahedral Meshes

Shape correspondences play a crucial role in various applications within graphics and geometry processing. These applications include texture and segmentation transfer, animation, and statistical shape analysis. The primary objective of these applications is to establish a dense map between two input shapes, allowing for semantically meaningful information transfer with minimal distortion. Traditionally, shape correspondence algorithms have focused on mapping two-dimensional surfaces. These algorithms leverage unique geometric properties specific to surfaces. For instance, key shape properties such as curvature are defined over the entire surface domain, enabling reasonable correspondences by matching geometric features directly without considering distortion. Other methods use techniques like Tutte’s embedding or discrete conformality to achieve properties like invertibility. Mapping volumes to volumes instead of surfaces to surfaces has not been explored before. Volumetric correspondence offers several benefits for various tasks. In graphics and CAD, boundary representations of shapes are commonly used, even when evaluating surface-to-surface mapping techniques. Therefore, finding volumetric correspondences can enhance the correspondences of these boundary representations, ensuring the preservation of thin features and preventing volumetric collapse. For example, it helps prevent artifacts like the candy-wrapper effect, where regions twist about a point and change orientation while surface area is roughly maintained but the volume degenerates.

Volumes do not share many geometric properties that enable mapping techniques for surfaces, necessitating new approaches. Existing methods that come close to volumetric mapping focus on volumetric deformation and parameterization. However, these approaches differ in several aspects from volumetric mapping. Volumetric deformation and parameterization assume a reasonable initial guess and flexibility in the target domain or specialize in a single target. On the other hand, mapping problems involve geometrically distinct source and target domains, often requiring a coarse map initialization and symmetric distortion energy.

To overcome these limitations, Massachusetts Institute of Technology researchers: Dr. S. Mazdak Abulnaga, Dr. Oded Stein, Professor Polina Golland, and led by Professor Justin Solomon  introduced an approach that aligns 3D shapes by mapping volumes to volumes instead of surfaces to surfaces. Their method draws insights from both 2D surface mapping and 3D deformation techniques. It builds upon the discretization of maps used in a state-of-the-art surface mapping algorithm but introduces new objective functions and optimization methods to achieve effectiveness in the volumetric mapping context. Specifically, they propose a set of symmetrized distortion energies that are invariant to the map’s applied domain. The objective is to produce inversion-free, low-distortion matchings that conform closely to the boundary. This technique represents shapes as tetrahedral meshes, including the mass inside the 3D object. By incorporating volumetric information, this approach better models fine details of objects, avoiding twisting and inversion issues typically encountered in surface-based mapping. The research work is now published in the peer-reviewed Journal ACM Transactions on Graphics.

The research team developed an algorithm that outperforms baseline methods in aligning shapes, producing high-quality shape maps with reduced distortion. It excels in challenging mapping scenarios where the input shapes are geometrically distinct, such as mapping a smooth rabbit to a LEGO-style rabbit composed of cubes. According to the authors, the applications of this technique extend beyond graphics, with potential uses in transferring motions, textures, annotations, and physical properties between different 3D shapes. Its impact is not limited to visual computing but can also benefit computational manufacturing and engineering. The algorithm’s design involved extending surface-based algorithms to enable volumetric mapping, but it became apparent that new mathematics and algorithms were required. Many existing mapping algorithms aim to minimize an “energy” that quantifies shape deformation when it is displaced, stretched, squashed, or sheared into another shape. However, these energies lack symmetry, which is critical for producing reliable and realistic mappings. Symmetric methods do not depend on the order of input shapes; mapping a horse onto a giraffe should yield the same matchings as mapping a giraffe onto a horse. The researchers created a mathematical framework to analyze energy behaviors and determine the appropriate choice for creating a symmetric map between two objects. The resulting algorithm combines energy functions for both shapes, ensuring symmetry throughout the mapping process.

The new algorithm takes as input two shapes represented as tetrahedral meshes and computes bidirectional maps, indicating how each corner of each tetrahedron should move to align the shapes. By using suitable energies and maintaining symmetry, the algorithm produces accurate alignments with higher quality and less distortion compared to alternative volume-based approaches. The volume information proves valuable even when only concerned with the outer surface map. While the algorithm demonstrates significant advancements, there are still limitations to address. It may struggle with shape alignments requiring significant volume changes, such as mapping a shape with a filled interior to one with a cavity inside. The authors plan to work on improving the algorithm’s efficiency and extending its applications to medical scenarios, incorporating MRI signals alongside shape data. This extension can bridge the mapping approaches used in medical computer vision and computer graphics.

In conclusion, Professor Justin Solomon and colleagues at MIT proposed algorithm for mapping between volumes represented as tetrahedral meshes presents a significant breakthrough in shape correspondence techniques. By leveraging insights from surface mapping and deformation, combined with the use of symmetrized distortion energies, the algorithm achieves accurate and high-quality alignments with minimal distortion. This advancement opens doors to improved shape transfer, motion mapping, and texture alignment in various fields, offering benefits for both research and practical applications.