RIE
Code Paper <cite>

ϕ-RIE: From Photorealistic Reconstruction to Interactive Environments

1 INSAIT, Sofia University “St. Kliment Ohridski”.

2 vivo Mobile Communication Co., Ltd.

3 Karlsruhe Institute of Technology

Contact: runyi.yang@insait.ai

∗ Corresponding author

Enter the playground PhiView code
THE FULL DEMO

From reconstruction to interaction.

Watch on YouTube ↗

NEWS

Code and paper are fully released.

REAL SCANS × OBJECT RECONSTRUCTION × INTERACTIONScroll to explore

01 — PHIVIEW DEMOS

See the interaction
from start to finish.

Nine demonstrations of shooting, grasping, placing, and navigating reconstructed rooms. Compare the same scenes with the harmonizer switched off and on.

PHIVIEW · SIX SHOTS · COLORFUL TRAILS

Spray bottle / shooting

01 / 09

Shoot the spray bottle and watch its physical response.

Download MP4 ↓

02 — THE METHOD

A room is more
than its appearance.

A simulator needs objects with metric geometry, contact surfaces, and visual identities that stay connected during motion. PhiRIE builds these pieces through evidence-guided construction.

ϕ-RIE at a glance. From captured views to editable objects, physical interaction, and harmonized rendering.
Method overview. Reconstruct the scene, build and register objects, complete the background, and connect appearance to simulation.
01Object selected in a reconstructed office

Observe & propose

Start with captured views and metric observations. Specialized generators propose complete object candidates.

Images → candidates
02Generated bottle aligned to its observed geometry

Register & select

Compare proposals in a shared frame. Use alignment evidence to select, retry, or retain a failed attempt.

Evidence → decisions
03Robot grasping a bottle in PhiView

Connect & interact

Link geometry, collision, and Gaussian appearance through object identity. Inspect how they behave together.

Objects → environments

03 — HANDS ON

Go beyond
the still image.

Three interactive experiences, rendered with WebGPU in your browser. Load a demo, orbit the scene, and explore the controls.

Preview of the captured desk scene

Make the scene your own.

Orbit a captured Gaussian desk and reposition its reconstructed bottle.

3D assets load only when you launch.

Ready to exploreDrag to orbit · scroll to zoom

WebGPU needs a compatible browser and a secure connection. If it is unavailable, the recorded demos above remain accessible.

04 — APPEARANCE

Same state.
A different finish.

Slide to compare saved views before and after appearance harmonization. The camera and scene state stay matched.

Harmonization is image postprocessing. These comparisons show appearance, not a change in physics.

Cup scene with harmonizationThe same cup scene without harmonizationWithoutWith harmonizer
↔

Drag the divider to compare

05 — VISUAL ATLAS

A closer look
at the results.

Explore captured environments, reconstruction candidates, background edits, and recorded interactions. Open any image for its full view and details.

FROM THE PAPER FIGURE COLLECTIONDifferent rooms. A shared representation. Gaussian environments across ScanNet++ and LIBERO. Includes original scene fits, saved background completion, and a recorded robot state.

06 — THE RESEARCH

Making rooms
simulatable.

Evidence-driven real-to-sim connects candidate construction with interaction evaluation. The manuscript separates construction quality, coverage, and success under a declared reconstruction scope.

Read the paper ↗

Download PDF ↗ · arXiv:2609.26795 · September 2026

PAPER FIGURE COLLECTIONMatched harmonization views, preserved from the paper’s figure collection.

CITATION

Cite ϕ-RIE.

arXiv:2609.26795 ↗

Download .bib ↓ 
@misc{yang2026phirie,
  title = {{$\phi$-RIE}: From Photorealistic Reconstruction to Interactive Environments},
  author = {Runyi Yang and Deheng Zhang and Xiaoye Wang and Kanzhi Wu and Lei Sun and Ajad Chhatkuli and Kunyu Peng and Luc Van Gool and Danda Pani Paudel},
  year = {2026},
  eprint = {2609.26795},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2609.26795}
}

07 — DEVELOPMENT

How PhiRIE
has grown.

From the first reconstruction prototype to a modular system for building and interacting with reconstructed environments.

  1. 0.0.0

    Reconstruction prototype

    Captured views, camera geometry, and Gaussian scene reconstruction formed the foundation.

    1. 1.0.0

      Object construction and simulation

      Object discovery, 3D asset generation, metric registration, background completion, and physics export connected reconstructed rooms to simulation.

      1. 2.0.0

        Public release: modular pipeline and PhiView

        Independent feature interfaces, composable pipelines, simulator adapters, and PhiView brought construction and interaction into one workflow. Released as the v2.0.0 stable code on 30 September 2026.

RELEASE PLAN

What comes next.

The pipeline code is released as v2.0.0 stable. Next comes a phone-scan walkthrough.

  1. Pipeline code — releasedv2.0.0 stable on GitHub: reconstruction, discovery, generation, registration, background completion, physics, simulator export, PhiView, and robot evaluation.
  2. Phone scan walkthrough — coming soonCapture a room with a phone, reconstruct it with run_video2sim.sh, and open it in PhiView, with a recorded example scan.
Follow the release plan ↗