2024/10/19 by Xianghao Xu, Srinath Sridhar, Xu, Xianghao +3
Engineering · #Additive Manufacturing and 3D Printing Technologies #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Graphics (cs.GR) #Innovations in Concrete and Construction Materials #Modular Robots and Swarm Intelligence
paper · pdf · doi:10.48550/arxiv.2410.15199
openalex publication_date 2024/10/19 · openalex created_date 2024/11/06 · openalex updated_date 2026/07/28
We propose a zero-shot text-driven 3D shape deformation system that deforms an input 3D mesh of a manufactured object to fit an input text description. To do this, our system optimizes the parameters of a deformation model to maximize an objective function based on the widely used pre-trained vision language model CLIP. We find that CLIP-based objective functions exhibit many spurious local optima; to circumvent them, we parameterize deformations using a novel deformation model called BoxDefGraph which our system automatically computes from an input mesh, the BoxDefGraph is designed to capture the object aligned rectangular/circular geometry features of most manufactured objects. We then use the CMA-ES global optimization algorithm to maximize our objective, which we find to work better than popular gradient-based optimizers. We demonstrate that our approach produces appealing results and outperforms several baselines.