OmniCam. Read the paper

GEOMETRY-GROUNDED CAMERA INTELLIGENCE

One panorama.
A world in motion.

Turn a scene and a sentence into a camera trajectory.
Geometry gives it structure. Language gives it intent.

Explore the trajectories Read the paper
360° scene context4 camera behaviorsSE(3) pose sequences
OMNICAM / GENERATED VIEW01 — TARGET
“Focus on the Christmas tree.”Instruction summary · completed render
Full 360-degree panorama of the snowy courtyard
THE STARTING POINTA single panorama
OMNICAM / RESEARCH PROJECTMeet the research SCENE → GEOMETRY → MOTION
01 / THE RESEARCH2026 · Preprint

Omni-Camera Trajectory Generation
via Geometry-Grounded Pose Token Learning

Zhenyang Liu1,2,3Chenjie Cao3Yisu Zhang4,3Xuhui Zuo3Xiangyang Xue1,‡Yanwei Fu1,2,‡Tengfei Wang3,†Chunchao Guo3
1 Fudan University2 Shanghai Innovation Institute3 Tencent Hunyuan4 Zhejiang University

† Project Lead ‡ Corresponding authors

Paper PDF ↗Video demos ↘Citation ↘

02 / TRAJECTORY EXPLORER

Same scene.
Different intentions.

Choose a world. Choose a behavior.
Watch the planned path become a new view.

01 / 09 EXAMPLES
INPUT / 360°01
PLANNED TRAJECTORY02

Point-cloud rendering

COMPLETED VIEW03

Geometry-completed rendering

LANGUAGE INSTRUCTION

Pan right across the snowy courtyard with the Christmas tree.

Shortened description
Read full instruction ↗
Selected example videos

Videos are precomputed research examples. Planned views visualize the predicted path; completed views use a downstream rendering pipeline.

03 / THE BIG PICTURE

From understanding a scene
to moving through it.

One model connects camera intent with scene geometry, across generation, reconstruction, and active perception.

FIG. 01 OmniCam translates a single panorama and textual intent into spatially aware SE(3) camera trajectories.
THE IDEA

Camera motion is more than a path through space. It is a decision about where to move, how to orient, and what to attend to. OmniCam brings panoramic geometry and language together in an autoregressive model, with hybrid pose tokens and explicit 3D target anchoring.

04 / INSIDE OMNICAM

Geometry for structure.
Semantics for intent.

Read the method ↗
01

See in every direction.

A panoramic point-cloud encoder captures omnidirectional context from estimated scene geometry.

PANORAMIC GEOMETRY
02

Give motion a language.

Hybrid pose tokens combine absolute rotations, relative translations, and temporally consistent quaternion signs.

HYBRID POSE TOKENS
03

Ground the intention.

Separate geometric and semantic pathways condition the decoder with an explicit, attention-aggregated 3D target anchor.

3D TARGET CONDITIONING
FIG. 02 Nine tokens per frame: four for rotation, three for translation, and two for camera intrinsics.

05 / MEASURED IN MOTION

Better paths.
Stronger target awareness.

Reported evaluation on OmniCaT.
All compared methods below are trained on OmniCaT.

TRAJECTORY ACCURACY43.0%↓

lower absolute trajectory error
vs. the best ATE baseline

SCENE GEOMETRY62.3%↓

lower collision rate
vs. the best collision baseline

TARGET FRAMING52.0%↗

target visibility
vs. 33.6% for GenDoP

A CLOSER LOOK

Absolute trajectory error

Lower is better. Relative improvement over Director3D, the strongest ATE baseline in this comparison.

Trajectory generation on OmniCaT
MethodATE ↓FDE ↓RPE-R ↓RPE-T ↓Collision ↓Visibility ↑Angle ↓
Director3D1.5222.6482.1870.04927.6%29.3%26.5°
GenDoP1.6242.4831.5770.03830.4%33.6%27.6°
OmniCam OURS0.8681.4621.1350.02710.4%52.0%10.9°

Relative ATE and collision reductions use Director3D as the best baseline for each metric. Collision is evaluated against reconstructed navigation meshes; trajectory errors compare with generated reference supervision. See the paper for evaluation details and limitations.

FIG. 03 Qualitative trajectory comparisons. The target bounding boxes show the intended subject.

06 / BEYOND THE TRAJECTORY

A path is just
the beginning.

Camera trajectories provide a control signal for generated video, 3D reconstruction, and robotic perception.

01 / VIDEO + RECONSTRUCTION

Guide the viewpoint.
Reveal the scene.

OmniCam trajectories drive downstream view synthesis, connecting a panorama to a sequence of new perspectives.

+14.1%CLIP score
+15.4%adjacent-frame SSIM

Trajectory + WorldStereo pipeline, relative to GenDoP trajectories on 484 reported examples. Adjacent-frame SSIM depends on motion magnitude.

02 / ACTIVE PERCEPTION

A better view
for the next action.

Object-directed camera motion helps reveal task-relevant regions for robotic perception. These qualitative examples illustrate the downstream use of generated viewpoints.

07 / THE OMNICAT DATASET

Built to explore.
At a different scale.

From panoramic scenes to geometry-grounded trajectories, with semantic targets and hierarchical language annotations.

267,700camera trajectories
9,998panoramic scenes
21.7Mrendered frames
Surround 97,702Target 90,065Reconstruct 43,316Wander 36,617
FIG. 04 Geometry and semantic grounding → trajectory synthesis → rendering and multimodal annotation.
DATASET DESIGN

Geometry, motion, and language.

The paper describes how OmniCaT combines panoramic scene geometry, four camera behaviors, and hierarchical language annotations.

Read about OmniCaT ↗

08 / BUILD ON THIS WORK

A new perspective
starts here.

If OmniCam is useful to your research,
please consider citing our work.

Read the paper ↗
BIBTEX
@misc{liu2026omnicam,
  title  = {OmniCam: Omni-Camera Trajectory Generation
            via Geometry-Grounded Pose Token Learning},
  author = {Liu, Zhenyang and Cao, Chenjie and
            Zhang, Yisu and Zuo, Xuhui and
            Xue, Xiangyang and Fu, Yanwei and
            Wang, Tengfei and Guo, Chunchao},
  year   = {2026},
  note   = {Preprint},
  url    = {https://zhenyangliu.github.io/OmniCam-Website/}
}

Scroll to inspect · Escape to close