GeometryDB and Proximity Deformer

Meta | Software Engineer, Digital Humans | 2024–2026

A graph-based C++ procedural execution framework for real-time deformation and simulation workflows, and a real-time proximity deformer built on top of it.

Highlights

  • Designed and implemented GeometryDB: a parallel, attribute-level DAG framework in C++ with Python bindings.
  • Built a real-time proximity deformer on top of GeometryDB with automatic bindings generation.
  • Built a Rerun-based visualization suite as first-class graph nodes.

Tags

C++, Python, Deformers, Cloth Sim, Real-Time, Node Graph

Contents

Introduction

GeometryDB is a procedural node graph framework I built at Meta Reality Labs Research. This is the story of how it came to be — the requirements, the design decisions, and the lessons I brought from building similar systems at Pixar, ILM, and Epic. Its first use case was a real-time proximity deformer, which I use throughout as a concrete thread to follow.

Context

I joined Meta Reality Labs Research in 2024, working in a lab focused on next-generation digital human and creature simulation.

The simulation group at the lab built a very capable real-time cloth simulator running what’s basically XPBD on a GPU. Despite our graphics cards being Very Fancy (you wouldn’t believe what the box under my desk cost), we couldn’t get the resolution of the simulation mesh high enough to escape artifacts. This is an unsurprising tension: the size of the triangles in your simulation mesh determines the degrees of freedom the material has — sheet metal vs. cardboard vs. leather vs. silk. When a lower resolution cloth mesh bends and bunches, it can’t help but look blocky. There just aren’t enough geometric degrees of freedom to faithfully represent how cotton or silk folds when you push a long sleeve shirt’s sleeve up your arm.

My first project was to build a proximity deformer — a way to decouple the render mesh from the simulation mesh and recover some of that lost fidelity. It would run within the lab’s production runtime environment, which had access to substantial dedicated compute targeting hardware capabilities beyond what’s available to consumers today.

That environment wasn’t the right place to develop individual components. It had long startup times and wasn’t designed for rapid iteration — behavior wasn’t always fully deterministic, which made validating component logic in-situ unreliable. Each piece of what I needed to build really needed to be unit tested in isolation.

But rolling custom test scaffolding for each component would mean a lot of elaborate throwaway code — infrastructure that existed only to exercise the thing being built, never making it into the final product. If you’re going to build test infrastructure anyway, it pays to make it reusable. A shared framework with common primitives means each new component gets testing almost for free.

We need a deformer

The standard VFX strategy for this is to not render the simulation mesh. You have a higher resolution render mesh that your simulation mesh pulls around. A child-parent relationship, where the child follows the parent. Vertices of the higher resolution child mesh get bound to nearby faces/vertices on the parent mesh. We typically call this kind of deformer a proximity deformer.

This decouples the resolution of the sim mesh, which operates on triangles as large as you can get away with, from the topology of the render mesh, which can be a much higher resolution and is better off being quads. Since there’s a difference in resolutions, we can smooth out the render mesh where the simulation mesh gets crunchy.

There are lots of different methods out there for deforming one surface from another, and they all make different tradeoffs between accuracy, behavior, and speed. I settled on two: one for performance and one for behavior.

For performance we’d do barycentric linear bindings, which would give us a linear smoothing of rough edges, with quickly diminishing returns. Every child vertex gets bound to 3 vertices on the parent mesh, with weights that add up to 1.0 that tell you how close to each it is. Basically, it’ll knock the edges off, given that the render mesh resolution lands in the Goldilocks zone of around twice the density of the sim mesh, and then drop anything more on the floor. That’s not great, but this is as fast as we can go.

The deformer emphasizing behavior was Procrustes method. Instead of binding each child vertex to a triangle on the parent, it binds it to a region of nearby particles within some topological distance, with weights diminishing smoothly as they get further away. The child gets affected more by nearby parents, but still feels the tug of the neighborhood. That’ll give us a much smoother result, but with higher computational costs and more complexity. There’s an iterative solve that needs to happen at evaluation time, and that makes this method possibly problematic for real-time scenarios. Since each child is driven by a larger area of the parent, so long as the parent region stays well behaved the child surface will look okay. But if the parent creases or has other issues, Procrustes method repeats these issues multiple times on the child, making it worse. So, there’s some nuance there.

Bindings Complicate Things

In either case, we’ve got setup work to do that maps child vertices to their parents, which we call bindings. The barycentric linear deformer needs barycentric coordinates of child vertices to parent triangles. Procrustes bindings start with these barycentric coordinates and expand topologically outward, gathering all vertices within some distance.

There was some discussion as to who would be responsible for these bindings. We could make it techart’s responsibility to generate them, and indeed that should be an option. But I didn’t like this, as a big part of what the system needs to do is switch geometry assets at runtime. Putting it on TDs to keep bindings up to date whenever topology of some garment changes probably isn’t the best use of their time. It also made the system fragile, as it becomes really easy to mix up the bindings. Generating them on the fly means they’re always up to date, and things will never break due to stale bindings. So that’s the way I went.

Another factor that went into this decision is the fact that creating even simple barycentric bindings isn’t necessarily easy. If the parent and child surfaces are well aligned, relatively flat, and lack any topologies that lead to ambiguity as to which parent triangle a child particle should belong to, then a naive projection can work. You throw all your parent triangles into a bounding volume hierarchy and do intersection detection against it with your child vertices. When you intersect more than one, you compute the barycentric coordinates for each, and take the one that ends up being closest. Easy.

But that’s rarely the case. Fingers are a good example of where things go wrong. A child vertex that lands in the valley between two fingers can be close to multiple parent triangles, and figuring out which one is the right one is non-trivial. Doing things like trying to minimize topological distance to neighbors and identifying discontinuities only works if you build pretty sophisticated systems that solve regions at a time. Get it wrong and “snags” happen, where parts of one finger get stuck to its neighbor. I think the right way to do this is to walk the surface in 2D space, like we did for skin sliding simulations.

So even simple bindings can be complicated. Maybe we can get by with doing simple projections at first, but particularly with Procrustes Method, the behavior of the deformation is going to be limited by the quality of the bindings. There’s going to be future work needed here.

Form Follows Function

Here are the requirements gathered thus far:

There are some additional complications that are really more software development concerns:

Development environment vs. deployment environment

As covered in Context, the production deployment environment wasn’t suitable for developing and validating individual components. That adds another requirement to the list: a unit-tested development workflow, independent of the deployment system, where each piece gets validated every time a change is made.

Geometry libraries should be procedural

We’re going to be doing real-time operations on geometry, so we could really use a geometry library that’s written to be real-time. Well, I don’t think that really exists. Most geometry libraries require you to hand them data in their own mesh format, which is already a deal-breaker because that doesn’t fit into our real-time constraint. Then it’s up to you to feed the data in and out of their system to get the operations you need done, all the while you could very well be redoing things that don’t need to be done. But then the input topology changes, or you figure out that you need a spatial data set on edges rather than vertices, or you need weighted face normals specified per-vertex rather than unweighted normals, etc. Building bespoke logic around whether this or that structure needs to be rebuilt is where software
systems like this get slow and fragile.

So here are some more requirements:

Cognitive sandboxing

Software doesn’t exist in a vacuum. It has an audience. The audience for this system is a bunch of disparate research groups. These research groups often bring interns in to do work on short timeframes. It’d be great if we could insulate them and researchers in general, from having to worry about the systems in which their work gets deployed. We can sandbox them, so there is less they need to know, and they can build to a common interchange API, that increases their impact. That makes everyone else around them better at their own jobs.

Documentation Matters

Where does user documentation live? I’m a firm believer that forcing people who write software to write user documentation gets them to write better software. It’s also been my observation that web sites and wikis are where user documentation goes to die. I believe that user documentation should be integrated into the functionality it aims to describe, as closely as possible. Ideally it should be written and versioned in the same repository as the code itself.

In GeometryDB, documentation strings are constructor arguments on interface and attribute classes — by convention, the full API surface of an interface is declared in its initializer list, with docs sitting right next to what they describe. No wiki required.

That gives us three more requirements:

Let’s Build a Procedural Node Graph

To me, this is begging for a procedural node graph system. I could write a library that does this in the traditional C++ fashion, but over the years I’ve come to understand that this is how throw-away code is born. It’s hyper focused on immediate needs, which is important, but without thinking about how it’ll grow and change in the future, you’re pretty much guaranteeing that it’s going to have a short life.

Software Design is Social Engineering

Pixar’s Presto dominates my experience with animation deformer systems. One of the downsides I saw in that system was a lot of deformers did significant redundant work. Lots of transforming points from one space to another, and back again, lots of generating different datastructures that may have already been generated elsewhere, etc. Presto’s execution system is intended to manage these things, and likely does more now than the last time I looked under the hood, but the system didn’t make discoverability easy, and its structure and its boilerplate didn’t encourage building fine granularity. Software design is social engineering, and my experience with this system was that it caused engineers to lean towards building deformers as standalone monoliths. Granularity, shared computation, and parallelism were typically nails hit only after production folk complained about things being too slow.

I’ve become opinionated about geometry libraries and how generalized asset construction should be done. For instance, I’ve made the case that doing a good job at generating deformer bindings is non-trivial. It’s going to require a lot of operations on triangle and segment meshes, and it’s going to be an ongoing iterative process. How do we build something that’s not going to become fractured, cumbersome, slow, and brittle along the way?

After architecting PAL at Pixar, working on Dataflow at Epic, and building other systems at ILM, I’ve come to believe that a well architected procedural node graph framework solves a lot of technical and organizational problems that exist in collaborative software environments. It’s not a magic bullet for all situations. There are potential downsides, and it takes real planning and foresight to avoid them. But when done well, a modular procedural node graph is a fantastic force multiplier, as it enables code reuse and collaboration across the org that embraces it.

Another huge upside I’ve noticed is that coding agents do extremely well when they’re cognitively sandboxed at the right level of abstraction, just like the researchers and interns I mentioned earlier. At least for today’s coding agents (March 2026), my theory is that they chase fewer dead ends and get lost in irrelevant side quests less, which frees up more of their “thought” capacity for what you actually want them to do. Narrowing the scope prunes the search space. There are fewer plausible paths to explore, so less opportunity to confidently pursue the wrong ones.

GeometryDB

The immediate problem I had was every part of what I needed to build required its own scaffolding: input/output wiring, dependency tracking, evaluation order, visualization, Python access for scripting and tuning. Without a shared framework, each piece was left to rebuild that infrastructure from scratch, and the resulting code was going to be hard to share, hard to maintain, and hard to reuse.

GeometryDB is a procedural execution framework I built around a directed acyclic graph. Each node in the graph is a C++ object that declares its inputs and outputs. The framework handles:

Let’s start with the core concepts.

The Database

The first thing we need is a data storage concept, and here GeometryDB takes some inspiration from Dataflow in UE5 and Pixar’s OpenUSD — both of which I worked on.

Dataflow uses a Geometry Collection (GC) object for its primary data store. It passes GC objects through the graph where different nodes author changes in the GC database via schemas called interfaces. Different interfaces can operate on the same underlying data in different ways, depending on the intent of the interface. Dataflow also has attributes that control behavior of nodes in the graph, which aren’t stored in the GC.

Geometry Collection Foreign Keys

The GC object itself has a concept of a foreign key, where you could relate one data field to another, like a bucket of indices references another bucket of points. If you, say, delete a point, then the GC updates the indices. I like this idea of the underlying system maintaining consistency, but on the other hand, it muddies some waters. Shouldn’t the logic be stored in the graph? How do we keep the database from doing reindexing work it doesn’t need to do, or doing it at the wrong times? To avoid this ambiguity, GeometryDB keeps all this in higher level logic, leaving it up to interfaces to maintain consistency.

USD is a formalized scene description library, where geometry data and configuration attributes live within the same structure. A USD scene may be comprised of scene description data written in different files.

Scene Description vs. Data Flow

PAL’s original design was that at any point time the graph data represented a full snapshot of that moment. Since the intent was to abstract away an underlying process, it made sense to group attributes in terms of inputs to that process, or its outputs. This made the later addition of runtime proceduralism to PAL a natural fit. Time could implicitly flow through the graph by tracking validity. But before then it was awkward when some PalObjects had only inputs, which was common, so we had the ability to skip the “input/output” designation.

USD reminds me a lot of PAL. PAL predated USD at Pixar, and I feel like (but don’t know for sure) it took a page out of this book, and dropped the input/output concept altogether. I think that’s fine for pure scene description, but it’s limiting for describing a dataflow graph. They later grafted the “input/output” designation on for UsdShade, and there are contextual hints with relationships, but I think this is a real limitation of USD. It describes what the things in the scene are, but not how data might flow through it.

The GeometryDB “database” is the structure that owns all attributes, which own views of data or the data itself. Like USD, the scene can be distributed across multiple db instances, which map to their own storage formats — ephemeral or serialized. Like Geometry Collection, attributes are organized by table, and data fields are related by intent, like the GC foreign key concept. Attributes can be stored grouped in different ways, but the default behavior arranges them in tables named by the node/interface they originated from. Because this is a procedural node graph, the tables are split into groups of inputs and outputs. That gives us a way to uniquely identify attributes by their path: “<dbName>/<interfaceName>/[input|output]/<attributeName>”.

Attributes

Attributes are the real currency of the system. Attributes control access to data, track the validity of that data, and have the means to update it when access to data is requested. If we want to keep the system fast, we need the system to prioritize:

Attributes may or may not own the memory they use. They may be given a view that’s owned by another attribute or host application, or it’ll take ownership of memory transferred to it via move semantics. The default value assignment operator takes a view of data passed to it. Yes, that means that those views can quietly dangle when the input data or output buffers go out of scope. Yes, this is dangerous. The system has some RAII facilities that help detect these situations, but we’re really making a design decision to favor speed over safety here. That puts the onus on the API user to make sure the input memory doesn’t disappear during graph evaluation.

Attributes are typed based on the value they hold, and you can create an attribute that holds any data type that you may want to pass around in the graph.

Callback Attributes

PAL had a concept of a callback attribute, which is an attribute that holds a function pointer. You could use these to add event handler hooks, or you could use them as a way to invoke some process. I think that concept is useful here too.

Don’t force your math library on me!

GeometryDB standardized on a specific math library for vector and matrix types because they were basically de facto standards. Still, it’d be better if the core library didn’t have such deeply ingrained dependencies and stayed math library agnostic, which is a lesson I learned on PAL. This is how I think this can work:

  • Attribute data is stored as raw typed memory described by scalar type, dimensions, and stride. No canonical math type, no ownership assumed.
  • Small type adapters (about ten lines each) teach the system how to interpret any math library’s types as raw memory, so USD, Eigen, glm, or whatever else, all plug in without touching core code.
  • Reading data back in a different type than it was written reinterprets the same memory through a typed view via reinterpret_cast<>. No constructor conversions, and no data copies.
  • Implementation code may use something like Eigen::Map internally for SIMD-accelerated math, but this is invisible to API consumers.

Then getting fancier, when multiple operations need to run in sequence and the input memory is unaligned, an aligned data buffer “arena” is created and populated. The full chain executes with optimal SIMD access, and results are copied back. So two memcpys per attribute regardless of operation count. But if the source memory is already aligned, then we don’t have to bother copying to/from the aligned arena.

All attributes have a single user defined updater function, typically implemented by a C++ lambda. If the value of the attribute is dirty and its value has been requested, it invokes the updater before returning the value. Dependencies between attributes are created through their updater functions.

Attributes are marked dirty through their dependencies. If the value of upstream attributes change, then downstream attribute values are marked dirty.

Now we can visualize a typical data flow use case. You’ve got input attributes with local space points and a local-to-global transform, and an output attribute for global space points. When either of the input attribute values change, the output is marked dirty. When the output attribute value is asked for, it’ll run its updater function that’ll pull values from the input attributes, and author its output value.

Commands

All functional behavior in GeometryDB is encapsulated in the Command concept. It keeps track of input and output attributes for updater functions, and is responsible for error detection and reporting. Since this is the concept in the library that does something, it’s a really convenient place to do higher level graph parallelization operations, which are discussed later.

Interfaces

Interfaces in GeometryDB are organizational constructs for attributes, and are the nodes in the graph. They represent high level scene constructs, like “triangle mesh”, “procrustes deformer”, etc.; math operators like “min”, “max”, “average”, “multiply”, etc.; visualization constructs like “Rerun Mesh3D”, etc. The term “interface” is used instead of “node” because the attribute data is stored in a central database, and it’s possible that different interfaces may operate on the same attributes. There’s flexibility here, but also enough rope to hang yourself with if you’re not careful — which is part of why the term “interface” rather than “node” felt appropriate. You could do sophisticated things with scene composition with this mechanism, like USD does. It also provides a way to deprecate functionality without requiring huge or ongoing cleanup efforts. Interfaces don’t really own their attributes, they only provide a contextualization of them.

Rigs

A rig in GeometryDB is a scene encapsulation concept. It’s an interface that contains other interfaces. Rigs may have config attributes that control aspects of sub-scene contained in the rig, or can configure themselves based on, say, the contents of a USD file, or the type of an interface pointed to by a relational attribute. That is to say, that all interfaces have these abilities. The “rig” concept is really just a naming convention for aggregate interfaces.

Rerun Visualization

Because Rerun nodes are first-class interfaces in the graph, a researcher could drop a visualization node anywhere in the pipeline and immediately see what was happening at that stage — no custom debug code required. I built a visualization suite on top of Rerun that handled the fussy details: double-sided mesh rendering, segment mesh wireframe overlay (not natively supported in Rerun), and vector fields — all with configurable color maps driven by any scalar field, like curvature, or the output of your node/edge/face coloring algorithm, or whatever else you might want. The result was that inspecting intermediate geometry state became nearly as easy as adding a print statement.

Parallel Graph Evaluation

Parallelism in GeometryDB took two forms; kernel level parallelism and graph based parallelism, and these generally break down into parallel-for and task based execution. In either case, it’s easy to make things slower by throwing more processes at the problem. Getting kernel parallelism right is generally concerned with having enough parallelizable work to do that overcomes the overhead of parallelizing, and being smart about memory traversal and data layout.

Getting a win out of doing graph evaluation in parallel is a different game altogether. First of all, you need parallelism to exist in the traversal of the graph. This is where having fine grained data dependencies can help. Both PAL and UE5 DataFlow put the functional behavior of the system on the nodes. Graph traversal always meant hopping from scene level node to node. This makes for a common pattern where the Node::exec() function serves as a dispatch containing conditionals that figure out what work should be done. These are chokepoints for parallel evaluation.

Instead, GeometryDB hangs functional behavior on attributes, which naturally bypasses most of the need for conditional dispatch functions. Data flows through direct attribute to attribute connections, and graph traversal isn’t even aware of the scene level nodes. This makes graph traversal more granular as it’s hopping from attribute to attribute. Thus we have more opportunities to exploit parallelism in the graph.

Command Bean Counting

Still, just because two things can be done in parallel doesn’t mean that doing so will always be faster. And unlike kernel parallelism, there are no good compile-time ways to structure our code to make sure we’re not wasting time on the overhead of doing parallel graph evaluation. Generic greedy task scheduling algorithms won’t know about how individual kernels perform ahead of time, so they’ll happily schedule them in parallel, even though doing so may be slower. We have to do this analysis at runtime.

GeometryDB commands are in charge of dispatching computations, and so they’re the ideal place to measure and track their performance. Over time we can aggregate a performance profile for individual command kernels. We can use that data at runtime to better decide which parts of a graph traversal should be done in parallel, and which ones under which workloads are best kept serial.

Proximity Deformer

Now that we’ve got a functional procedural node graph, we can put it to work. Here’s an example of a rig that encapsulates a graph that translates points to the origin, all built from generic reusable components, with minimal memory allocations and data copies.

Center-at-origin sub-graph

We use this subgraph with the proximity deformer. It’s important to generate deformer bindings with vertices at the origin to avoid floating point aliasing issues. The graph below is a much simplified version of the final proximity deformer. There’s a section devoted to generating bindings that only runs when the rest state changes.

Proximity deformer expanded graph

The full proximity deformer also included the Procrustes deformer and its bindings, plus normal deformation — when source normals are modeled rather than computed, you can’t just recompute them after deforming the surface; you have to deform them too.

While I was building all this, there was some doubt that the GeometryDB deformer could be fast enough. Turns out that the overhead amounted to basically nothing, and I’d argue that the framework is part of what enables it to be fast. What’s especially great about frameworks like this is that it’s all modular, making each piece easy to battle-harden in unit tests. While the end result turned into something pretty complicated, it landed without a hitch. Never crashed, and it never popped up as a tall nail in a performance profile. So we doubled the resolution of our render meshes and made them double sided, and essentially paid no performance penalty for it.

Conclusion

GeometryDB ended up being exactly the kind of invisible infrastructure that good tools should be. It allowed you to focus on building something without the complexities of the production environment, the visualization tools meant problems surfaced early, and the framework overhead was low enough that it never became a conversation. The goal was always to get out of the way. It did.