SENSOR FUSION DATA

Robot Sensor Recordings

High-frequency proprioception, inertial, and audio streams with sub-millisecond synchronization across embodiments.

<1MS SYNCJSONL · NPZ · WAVStreamed with every collection run

1,000Hz

IMU capture

920Hz

Proprioception

<1ms

Hardware sync

44.1kHz

Audio capture

Download Package Contents Build process Verification Audit trail Use cases Delivery Further reading Request a sample

01Download

Inspect a real sample

Representative records in the delivery format, ready to inspect before licensing the full dataset.

Booster T1 proprioception sample (23 joints)

Representative of the real line. Positions in rad, velocities in rad/s, efforts in Nm.

joints.jsonlrepresentative

{
  "timestamp": 3.0976,
  "unix_timestamp": 1766526822.3984,
  "positions":  { "Waist": -0.0048, "Left_Hip_Pitch": -0.3126, "Left_Knee_Pitch": 0.4141, "Head_pitch": 0.3603 },
  "velocities": { "Left_Knee_Pitch": -0.00929, "Right_Knee_Pitch": 0.00759 },
  "efforts":    { "Left_Hip_Yaw": 2.9137, "Left_Knee_Pitch": -4.2377 }
}

LimX Tron IMU sample (6-DOF + orientation)

Representative of the real line. Acceleration in m/s^2, gyro in rad/s, plus a quaternion and a nanosecond stamp.

imu.jsonlrepresentative

{
  "timestamp": 0.009156,
  "unix_timestamp": 1760757712.1551,
  "stamp_ns": 10578143272597,
  "accel": [9.4843, 0.0972, 2.7084],
  "gyro": [0.000252, -0.0000055, -0.00882],
  "quat": [0.2879, 0.5626, -0.2103, 0.7459]
}

LimX Tron odometry sample (base pose + twist)

Representative of the real line. World position and orientation plus linear and angular velocity.

odometry.jsonlrepresentative

{
  "timestamp": 0.008726,
  "unix_timestamp": 1760758026.1290,
  "pose_position": [0.7061, 0.3026, 0.1516],
  "pose_orientation": [0.5434, -0.2517, 0.7253, 0.3395],
  "twist_linear": [0.00143, 0.00146, 0.00063]
}

02Schema

Record shape

Every field, its type, whether it can be null, and a representative value.

joint_sample_rate_hz

required

Measured joint sampling rate (75 to 920 Hz across platforms).

Type: float64
Unit: Hz

Example 79.23

positions

required

Per-joint angular position (named joints).

Type: map<joint,float>
Unit: rad

Example {"Left_Knee_Pitch":0.4141}

velocities

required

Per-joint angular velocity.

Type: map<joint,float>
Unit: rad/s

Example {"Left_Knee_Pitch":-0.0093}

efforts

required

Per-joint torque / effort.

Type: map<joint,float>
Unit: Nm

Example {"Left_Knee_Pitch":-4.24}

imu.accel

required

Linear acceleration (6-DOF IMU).

Type: float[3]
Unit: m/s^2

Example [9.4843, 0.0972, 2.7084]

imu.gyro

required

Angular velocity.

Type: float[3]
Unit: rad/s

Example [0.00025, -0.0000, -0.0088]

imu.quat

required

Orientation estimate (quaternion).

Type: float[4]

Example [0.288, 0.563, -0.210, 0.746]

imu.avg_rate_hz

required

Measured IMU sample rate.

Type: float
Unit: Hz

Example 97.3

odometry.pose_position

required

Base position in world frame.

Type: float[3]
Unit: m

Example [0.7061, 0.3026, 0.1516]

odometry.twist_linear

required

Base linear velocity.

Type: float[3]
Unit: m/s

Example [0.0014, 0.0015, 0.0006]

audio.sample_rate

required

Audio sample rate (wav).

Type: int
Unit: Hz

Example 16000

timestamp

required

Episode-relative timestamp on every sample.

Type: float64
Unit: s

Example 3.0976

unix_timestamp

required

Absolute Unix timestamp on every sample, for cross-stream alignment.

Type: float64
Unit: s

Example 1766526822.3984

stamp_ns

nullable

Nanosecond hardware stamp on IMU and motor streams.

Type: int64
Unit: ns

Example 10578143272597

Field	Type	Constraint	Description
joint_sample_rate_hz	float64 · Hz	required	Measured joint sampling rate (75 to 920 Hz across platforms). e.g. 79.23
positions	map<joint,float> · rad	required	Per-joint angular position (named joints). e.g. {"Left_Knee_Pitch":0.4141}
velocities	map<joint,float> · rad/s	required	Per-joint angular velocity. e.g. {"Left_Knee_Pitch":-0.0093}
efforts	map<joint,float> · Nm	required	Per-joint torque / effort. e.g. {"Left_Knee_Pitch":-4.24}
imu.accel	float[3] · m/s^2	required	Linear acceleration (6-DOF IMU). e.g. [9.4843, 0.0972, 2.7084]
imu.gyro	float[3] · rad/s	required	Angular velocity. e.g. [0.00025, -0.0000, -0.0088]
imu.quat	float[4]	required	Orientation estimate (quaternion). e.g. [0.288, 0.563, -0.210, 0.746]
imu.avg_rate_hz	float · Hz	required	Measured IMU sample rate. e.g. 97.3
odometry.pose_position	float[3] · m	required	Base position in world frame. e.g. [0.7061, 0.3026, 0.1516]
odometry.twist_linear	float[3] · m/s	required	Base linear velocity. e.g. [0.0014, 0.0015, 0.0006]
audio.sample_rate	int · Hz	required	Audio sample rate (wav). e.g. 16000
timestamp	float64 · s	required	Episode-relative timestamp on every sample. e.g. 3.0976
unix_timestamp	float64 · s	required	Absolute Unix timestamp on every sample, for cross-stream alignment. e.g. 1766526822.3984
stamp_ns	int64 · ns	nullable	Nanosecond hardware stamp on IMU and motor streams. e.g. 10578143272597

03What's included

Proprioception

Joint positions, velocities, and efforts at 75-920Hz, aligned to vision and action streams.

Inertial (IMU)

1000Hz 6-DOF inertial measurement for dynamics, balance, and contact modeling.

Time-Sync Fabric

Sub-millisecond hardware synchronization across all sensor and vision streams.

04Methodology

How it is built

01
High-frequency multi-sensor capture
Proprioception (joint and motor position, velocity, effort), IMU (acceleration, gyro, orientation), odometry, and audio are captured per episode at native rates.
02
Per-sample timestamping
Every sample carries both an episode-relative timestamp and an absolute Unix timestamp; IMU and motor streams additionally carry nanosecond stamps for fine alignment.
03
Cross-stream alignment
Streams are kept at native rates rather than resampled and aligned on a shared timeline. The format records a sync-tolerance budget; the architecture targets sub-millisecond hardware sync.
04
Per-stream accounting
Each stream sample count and average rate are recorded in metadata, enabling rate verification - for example motors at 915 Hz, IMU at 97 Hz, odometry at 96 Hz on one platform.
05
Integrity and packaging
Per-file SHA-256 checksums and total size are recorded, and streams are bundled in the ORTF episode structure with a manifest describing each stream type, rate, and encoding.

05Evals

How we validate

What each evaluation measures and how it is run. Where no benchmark is published, we show the methodology and say so.

Sample-rate verification

Measures

Whether each stream actually achieved its target rate.

Method

Compare the recorded sample count and duration against the declared average rate and flag drops.

Result

Methodology-stage. Per-stream measured rates are recorded; no aggregate published metric.

Cross-stream sync tolerance

Measures

Whether modalities fall within the synchronization budget.

Method

The format defines a maximum acceptable inter-modality timestamp delta, and the validator checks timestamp monotonicity.

Result

Methodology-stage. Tolerance is a spec field; no measured sync-error figure is published.

Sim-to-real sensor grounding

Measures

Whether real high-frequency traces match simulated dynamics.

Method

Real joint and velocity traces are compared against MuJoCo replay.

Result

Real measured: velocity tracking 0.41 rad/s MAE and a dynamic gap score of 0.065 - the real velocity stream is what grounds this. See the sim2real paper.

06Graders

Ground truth

What correct means for this data, and how it is established.

Ground truth

The physical sensor readings are themselves the ground truth. For simulation grounding, the real high-frequency velocity and position traces are the reference the simulation is scored against.

How it is established

Stream integrity is graded by SHA-256 checksums and per-stream sample-count and rate accounting; temporal validity by timestamp monotonicity; sim-to-real fidelity by MAE and RMSE against the real trace.

Agreement

No inter-rater agreement applies - sensor data is machine-measured. Sim-grounding correctness is anchored to the real trace.

07Application

Sensor Fusion Models

Train multi-modal encoders on tightly-synced vision, proprioception, and inertial data.

Sim-to-Real Calibration

Ground simulators against real high-frequency sensor traces to close the reality gap.

Cross-Embodiment Transfer

Aligned sensor schemas across platforms for policies that generalize between robots.

08Environment & integration

How you load it

Delivery

S3, REST API

Formats

JSONL streams (proprioception / IMU / odometry / motors), NPZ packed arrays, WAV audio

Auth

Org-scoped API keys, tenant isolation, and signed artifacts with checksums and access logging.

Cadence

Streamed with every collection run, emitted alongside each episode.

quickstart.sh

# Read and align a proprioception stream
# 1. Load feed/proprioception/joints.jsonl - each line is one timestamped sample
# 2. Each sample carries positions / velocities / efforts maps
# 3. Align across streams on unix_timestamp (or stamp_ns where present)
# 4. Verify rate via sample_count / avg_rate_hz in the metadata

09Related research

Sim2Real Gap in Bipedal HumanoidsRead →

Request a sample