BlenderFORGE project logo

BlenderFORGE: Framework for Optimizing Reactive 3D-Graphics Editing Ability of MLLMs

(NeurIPS 2026)

Zilin Guo1,*, Tianrui Zhang1,*, Yi Rong1,*, Yichen Liu2, Yuxin Guo2, Jingcheng Ni2, Lewei Lu2, Zehuan Wu2, Dan Xu1
1The Hong Kong University of Science and Technology 2SenseTime Research

*Equal contribution

The Hong Kong University of Science and Technology SenseTime
CodeLeaderboard

Abstract

Precise, programmatic 3D editing requires executable interaction and verifiable feedback. We introduce BlenderFORGE, a data–environment–optimization framework for training open-weight multimodal agents to edit Blender scenes through Python code. The framework combines perturbation-based task generation, an instance-isolated Blender sandbox, and a two-stage training pipeline comprising trajectory supervised fine-tuning and multi-turn AgentLoop-RL. It covers four task families: object placement, shape-key editing, lighting adjustment, and material modification. The trained Qwen3-VL-8B agent improves over its base model across all four families. Evaluations on held-out camera tasks, BlenderGym, and a second model backbone provide additional evidence of transfer.

BlenderFORGE framework: perturbation-based task generation, Blender sandbox, trajectory SFT, general SFT, and AgentLoop-RL.
BlenderFORGE integrates task generation, executable interaction, and agent optimization in a shared scene-state space.

Demonstrations

Object Placement

Object translation and rotation from aligned scene views.

Shape-Key Editing

Shape-key coefficient adjustment to match target geometry.

Lighting Adjustment

Illumination color and energy modification.

Material Modification

Shader parameter editing with rendered feedback.

Quantitative Results

Comparison of zero-shot and BlenderFORGE-trained models on the test set. Results include Qwen3-VL-8B and Qwen3.5-9B backbones.

Citation

@inproceedings{guo2026blenderforge,
  title     = {{BlenderFORGE}: Framework for Optimizing Reactive
               {3D}-Graphics Editing Ability of {MLLMs}},
  author    = {Guo, Zilin and Zhang, Tianrui and Rong, Yi and
               Liu, Yichen and Guo, Yuxin and Ni, Jingcheng and
               Lu, Lewei and Wu, Zehuan and Xu, Dan},
  booktitle = {The Fortieth Annual Conference on Neural Information Processing Systems},
  year      = {2026},
  url       = {https://openreview.net/forum?id=czK5BBdtMP}
}

Video