Φ PhiRIE. Code
EVIDENCE-DRIVEN REAL-TO-SIM v2.0.1

PhiRIE: From Photorealistic Reconstruction to Interactive Environments

  • Runyi Yang1
  • Deheng Zhang1
  • Xiaoye Wang1
  • Kanzhi Wu2
  • Lei Sun1
  • Ajad Chhatkuli1
  • Kunyu Peng3,∗
  • Luc Van Gool1
  • Danda Paudel1

1 INSAIT, Sofia University “St. Kliment Ohridski”.

2 vivo Mobile Communication Co., Ltd.

3 Karlsruhe Institute of Technology

Contact: runyi.yang@insait.ai

∗ Corresponding author

Enter the playground PhiView code
THE FULL DEMO

From reconstruction to interaction.

Watch on YouTube ↗
Gaussian reconstruction of a captured kitchen
01 / OBSERVE Captured environments
Reconstructed bottle reassembled into a Gaussian office scene
02 / CONSTRUCT Editable scene objects
Recorded robot and reconstructed cup in a LIBERO tabletop scene
03 / INTERACT Appearance meets motion
Real project assets. Explore the details below.
REAL SCANS × OBJECT RECONSTRUCTION × INTERACTIONScroll to explore

01 — THE IDEA

A room is more
than its appearance.

A simulator needs objects with metric geometry, contact surfaces, and visual identities that stay connected during motion. PhiRIE builds these pieces through evidence-guided construction.

01Object selected in a reconstructed office

Observe & propose

Start with captured views and metric observations. Specialized generators propose complete object candidates.

Images → candidates
02Generated bottle aligned to its observed geometry

Register & select

Compare proposals in a shared frame. Use alignment evidence to select, retry, or retain a failed attempt.

Evidence → decisions
03Robot grasping a bottle in PhiView

Connect & interact

Link geometry, collision, and Gaussian appearance through object identity. Inspect how they behave together.

Objects → environments
FROM THE PAPER FIGURE COLLECTIONDifferent rooms. A shared representation. Gaussian environments across ScanNet++ and LIBERO. Includes original scene fits, saved background completion, and a recorded robot state.

02 — HANDS ON

Go beyond
the still image.

Three interactive experiences, rendered with WebGPU in your browser. Load a demo, orbit the scene, and explore the controls.

Preview of the captured desk scene

Make the scene your own.

Orbit a captured Gaussian desk and reposition its reconstructed bottle.

3D assets load only when you launch.

Ready to exploreDrag to orbit · scroll to zoom

WebGPU needs a compatible browser and a secure connection. If it is unavailable, the recorded results below remain accessible.

03 — PHIVIEW DEMOS

See the interaction
from start to finish.

Nine demonstrations of shooting, grasping, placing, and navigating reconstructed rooms. Compare the same scenes with the harmonizer switched off and on.

PHIVIEW · SIX SHOTS · COLORFUL TRAILS

Spray bottle / shooting

01 / 09

Shoot the spray bottle and watch its physical response.

Download MP4 ↓

04 — APPEARANCE

Same state.
A different finish.

Slide to compare saved views before and after appearance harmonization. The camera and scene state stay matched.

Harmonization is image postprocessing. These comparisons show appearance, not a change in physics.

Cup scene with harmonizationThe same cup scene without harmonizationWithoutWith harmonizer
↔

Drag the divider to compare

05 — VISUAL ATLAS

A closer look
at the results.

Explore captured environments, reconstruction candidates, background edits, and recorded interactions. Open any image for its full view and details.

06 — THE RESEARCH

Making rooms
simulatable.

Evidence-driven real-to-sim connects candidate construction with interaction evaluation. The manuscript separates construction quality, coverage, and success under a declared reconstruction scope.

Get v2.0.1 ↗

Paper link coming soon.

PAPER FIGURE COLLECTIONMatched harmonization views, preserved from the paper’s figure collection.

07 — DEVELOPMENT

How PhiRIE
has grown.

From the first reconstruction prototype to a modular system for building and interacting with reconstructed environments.

  1. 0.0.0

    Reconstruction prototype

    Captured views, camera geometry, and Gaussian scene reconstruction formed the foundation.

    1. 1.0.0

      Object construction and simulation

      Object discovery, 3D asset generation, metric registration, background completion, and physics export connected reconstructed rooms to simulation.

      1. 2.0.0

        Modular pipeline and PhiView

        Independent feature interfaces, composable pipelines, simulator adapters, and PhiView brought construction and interaction into one workflow.

        1. 2.0.1

          PhiRIE release and demonstrations

          Consolidated setup and documentation, the project page, illustrated components, and demonstrations of manipulation, harmonization, and controllable physical parameters.

RELEASE PLAN

What comes next.

Planned work; release dates will follow validation.

  1. Guided setupSimpler installation and a complete first-scene walkthrough.
  2. Downloadable scene packsPrepared scenes and repeatable shooting, manipulation, and harmonizer demos.
  3. Reproducible evaluationDocumented configurations, benchmark instructions, and results as they are finalized.
Follow the release plan ↗