Try it in browser Source Code Explore Other Examples

Visualizes an example recording from the NYUD dataset with RGB and Depth channels.

The dataset, known as the NYU Depth V2 dataset, consists of synchronized pairs of RGB and depth frames recorded by the Microsoft Kinect in various indoor scenes. This example visualizes one scene of this dataset, and offers a rich source of data for object recognition, scene understanding, depth estimation, and more.

Logging and visualizing with Rerun

The visualizations in this example were created with the following Rerun code:

Timelines

All data logged using Rerun in the following sections is connected to a specific time. Rerun assigns a timestamp to each piece of logged data, and these timestamps are associated with a timeline.

rr.set_time_seconds("time", time.timestamp())

Image

The example image is logged as Image to the world/camera/image/rgb entity.

rr.log("world/camera/image/rgb", rr.Image(img_rgb).compress(jpeg_quality=95))

Depth image

Pinhole camera is utilized for achieving a 3D view and camera perspective through the use of the Pinhole.

rr.log(
    "world/camera/image",
    rr.Pinhole(
        resolution=[img_depth.shape[1], img_depth.shape[0]],
        focal_length=0.7 * img_depth.shape[1],
    ),
)

Then, the depth image is logged as an DepthImage to the world/camera/image/depth entity.

rr.log("world/camera/image/depth", rr.DepthImage(img_depth, meter=DEPTH_IMAGE_SCALING))

Join us on Github

import rerun as rr  # pip install rerun-sdk
rr.init("rerun_example_app")

rr.connect()  # Connect to a remote viewer
# rr.spawn()  # Spawn a child process with a viewer and connect
# rr.save("recording.rrd")  # Stream all logs to disk

# Associate subsequent data with 42 on the “frame” timeline
rr.set_time_sequence("frame", 42)

# Log colored 3D points to the entity at `path/to/points`
rr.log("path/to/points", rr.Points3D(positions, colors=colors