Puppeteer: Rig and Animate Your 3D Models

Chaoyue Song^1,2, Xiu Li², Fan Yang¹, Zhongcong Xu², Jiacheng Wei¹,
Fayao Liu³, Jiashi Feng², Guosheng Lin^1*, Jianfeng Zhang^2*
*Corresponding authors
¹ Nanyang Technological University ² Bytedance Seed ³ A*STAR

arXiv 2025

Project | Paper | Video | Data: Articulation-XL2.0

Puppeteer is proposed for automatic rigging and animation of 3D objects. Given a 3D object, Puppeteer first automatically generates skeletal structures and skinning weights, then animates the rigged model with video guidance through a differentiable optimization pipeline. This comprehensive approach aims to enable fully automated transformation of static 3D models into dynamically animated assets, eliminating the need for manual rigging expertise and significantly streamlining 3D content creation workflows.

• Sep 09, 2025: We uploaded the video for Puppeteer.
• Sep 04, 2025: Release the inference codes and model checkpoints.
• Aug 15, 2025: Release paper of Puppeteer!

We use Python 3.10 with PyTorch 2.1.1 and CUDA 11.8. The environment and required packages can be installed as follows:

git clone https://github.com/ByteDance-Seed/Puppeteer.git --recursive && cd Puppeteer
conda create -n puppeteer python==3.10.13 -y
conda activate puppeteer
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
pip install flash-attn==2.6.3 --no-build-isolation
pip install torch-scatter -f https://data.pyg.org/whl/torch-2.1.1+cu118.html
pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py310_cu118_pyt211/download.html

We provide a complete pipeline for rigging and animating 3D models. Before running the pipeline, visit each folder (skeleton, skinning, animation) to download the necessary model checkpoints. Example data is available in the examples folder.

Given 3D meshes, we first predict the skeleton and skinning weights:

bash demo_rigging.sh

The final rig files will be saved in results/final_rigging. To evaluate the skeleton and skinning components separately, refer to their respective folders.

To animate the rigged model using video guidance, run:

bash demo_animation.sh

The rendered 3D animation sequence from different views will be saved in results/animation. Refer to the animation folder for comprehensive details on data processing and structure.

The code builds upon MagicArticulate, MeshAnything, Functional Diffusion, RigNet, Michelangelo, PartField, AnyMole and Lab4D. We gratefully acknowledge the authors for making their work publicly available.

@article{song2025puppeteer,
  title={Puppeteer: Rig and Animate Your 3D Models},
  author={Chaoyue Song and Xiu Li and Fan Yang and Zhongcong Xu and Jiacheng Wei and Fayao Liu and Jiashi Feng and Guosheng Lin and Jianfeng Zhang},
  journal={arXiv preprint arXiv:2508.10898},
  year={2025}
}