Abstraction Discovery in Shape Programs

Abstract

A central challenge in neurosymbolic graphics is the automated discovery of programmatic abstractions from hierarchical 3D shape representations. While procedural models provide structural clarity, real-world objects often exhibit complex relationships that rigid symbolic templates cannot capture. Traditional approaches rely on a greedy parametric search over a fixed combination of variables, creating a "symbolic bottleneck" where non-linear dependencies remain uncompressed. This thesis introduces a neurosymbolic framework that replaces discrete symbolic matching with the neural modeling of parametric data extracted from a functional DSL. By utilizing parameter clustering and intrinsic dimensionality estimation techniques, the framework identifies the intrinsic degrees of freedom within the data and models them using neural networks like Autoencoders to learn non-linear dependencies between parameters for high-fidelity reconstruction, and Variational Autoencoders for increased latent interpretability and editability. When integrated into an E-graph for structural refactoring, this approach successfully resolves complex parametric variations that symbolic solvers might miss. Evaluated on diverse 3D shape categories, our framework demonstrates the power of combining neural flexibility with symbolic structure to achieve improved program compression as measured by the minimum description length principle.

Motivation

The core challenge in my work is the inefficiency of current greedy search methods for finding abstractions. When the system attempts to fill parameter slots, it considers constants, direct variable mappings, and combinations of variables with arithmetic operators. This manual exploration of every possible combination leads to a combinatorial explosion, where the search space grows too fast for the system to handle effectively.

Instead of relying on these traditional methods to guess relationships through math, my thesis explores a different path. I used autoencoders to learn the underlying relationships between parameters directly from the data. This approach allowed the system to identify meaningful structures automatically, bypassing the limitations of exhaustive search.

Results

Comparison of complexity reduction across methods. Each row (a–j) shows an input model from PartNet-Sym and its reconstructions using PCA, AE, and VAE across two hierarchical levels (L1, L2). The values below each model represent structural node counts and total parameter counts. The results demonstrate progressive compression from L1 to L2 and clear performance gains with latent models. Rows (c, f, h) highlight failure cases where aggressive compression causes geometric disconnection of chair components.

Comparison of models showing structure and parameter reduction

Latent space navigation. The VAE manifold enables smooth geometric transitions, ensuring stable variations during latent space traversal. Unlike standard Autoencoders, which can produce disjoint artifacts, the VAE preserves geometric stability by learning a continuous and stable latent manifold.

Latent space visualization and traversal

Software. Developed an interactive tool to demonstrate the practical application of these learned abstractions. It allows for the live manipulation of latent codes, enabling users to edit 3D shapes and their corresponding hierarchical abstractions in real time.

Interactive latent code manipulation.

Live editing example.