Back
Technology

ShanghaiTech University Researchers Publish End-to-End Dynamic Gaussian Splatting Processing Platform

Generated on: Last updated: 1 sources
View source

ShanghaiTech Team Unveils Unified Platform for 3D Gaussian Splatting

A new processing platform aims to streamline the creation and use of volumetric video, addressing key challenges of high data costs and fragmented tools.

A research team from ShanghaiTech University has developed a processing platform for 3D Gaussian Splatting technology, designed to provide a unified, end-to-end workflow from data acquisition to multi-platform rendering. The research, which includes a large-scale dynamic human motion dataset, was published in the peer-reviewed journal Visual Intelligence on March 27, 2026.

Technical Background

3D Gaussian Splatting is a computer graphics technique that uses millions of points to create realistic 3D scenes. The technology is a core component of volumetric video, which generates digital 3D models that can be viewed from any angle. Researchers have extended the technique to dynamic scenes, creating what are termed 4D Gaussian Splatting representations. These technologies have applications in virtual reality, augmented reality, and immersive media.

Identified Challenges

According to the research team, volumetric video technology faces two primary challenges:

  • High Storage and Transmission Costs: The data volume for temporal sequences, especially dynamic scenes, creates significant demands on storage, transmission, and real-time interaction.
  • Fragmented Tool Chain: The researchers state that existing solutions and research have typically focused on isolated stages of the process, lacking a unified, end-to-end workflow. They note that earlier studies on compression and optimization have been scattered across different code bases with inconsistent data formats.

Dr. Lan Xu, an Assistant Professor at ShanghaiTech University, stated that high storage cost is a pressing issue, particularly for dynamic scenes where the introduction of the temporal dimension dramatically increases data volume.

Platform Description

The proposed framework is designed to address these challenges by integrating the entire processing pipeline. Its key components include:

  • End-to-End Pipeline: The platform provides a complete workflow from data acquisition and standardized preprocessing to dynamic Gaussian reconstruction.
  • Reconstruction Methods: It incorporates various existing 3D and 4D Gaussian Splatting reconstruction algorithms.
  • Standardized Interfaces: The framework offers standardized data preprocessing interfaces and unified data loading mechanisms.
  • Compression Framework: A core contribution is a general-purpose compression framework compatible with outputs from various reconstruction methods. It is designed to reduce the storage footprint of dynamic sequences while maintaining visual fidelity.
  • Cross-Platform Rendering: The team developed a real-time rendering plugin that supports interactive, free-viewpoint experiences on desktop computers, mobile devices, and extended reality (XR) devices.

Accompanying Dataset

As part of the work, the team created a large-scale dynamic human motion capture dataset. To capture this data, they built a dense multi-view acquisition system consisting of 81 synchronized RGB cameras. The system recorded over 130 sequences of diverse human motions, including complex interactions. The video was captured at a resolution of 3840 x 2160 and a frame rate of 30 frames per second, with timecode alignment.

Researcher Statements

  • Professor Jingyi Yu, a senior member of the research team, stated that "volumetric video enables free-viewpoint exploration of immersive virtual environments."
  • Dr. Lan Xu stated that the goal of the platform is "to establish a complete pipeline from data acquisition to practical application, promoting the large-scale adoption of Gaussian Splatting technologies."

Research Team and Funding

The ShanghaiTech University research team includes Shengkun Zhu, Chengcheng Guo, Yuanji Lu, Zhehao Shen, Yize Wu, Yu Hong, Yiwen Cai, Meihan Zheng, Yingliang Zhang, Lan Xu, and Jingyi Yu.

Funding was provided by multiple Chinese institutions, including:

  • National Natural Science Foundation of China
  • National Key R&D Program of China
  • Central Guided Local Science and Technology Foundation of China
  • MoE KeyLab of Intelligent Perception and Human-Machine Collaboration (ShanghaiTech University)
  • Shanghai Frontiers Science Center of Human-centered Artificial Intelligence

Publication and Author Details

The research was published in Visual Intelligence, an international, peer-reviewed, open-access journal that is the official publication of the China Society of Image and Graphics. The Society covers the Article Processing Charges for the journal.

  • Dr. Jingyi Yu is an OSA Fellow, IEEE Fellow, ACM Distinguished Scientist, Director of the MoE Key Lab of Intelligent Perception and Human-Machine Collaboration, and Inaugural Chair Professor and Vice President of ShanghaiTech University.
  • Dr. Lan Xu is an Assistant Professor with the School of Information Science and Technology at ShanghaiTech University.