SENSOR FUSION DATA
Multi-Modal Sensor Streams
High-frequency proprioception, inertial, and audio streams with sub-millisecond synchronization across embodiments.
See the data
Representative records in the exact shape we deliver. Real provenance and full slices are shared under license.
Booster T1 proprioception sample (23 joints)
Representative of the real line. Positions in rad, velocities in rad/s, efforts in Nm.
{
"timestamp": 3.0976,
"unix_timestamp": 1766526822.3984,
"positions": { "Waist": -0.0048, "Left_Hip_Pitch": -0.3126, "Left_Knee_Pitch": 0.4141, "Head_pitch": 0.3603 },
"velocities": { "Left_Knee_Pitch": -0.00929, "Right_Knee_Pitch": 0.00759 },
"efforts": { "Left_Hip_Yaw": 2.9137, "Left_Knee_Pitch": -4.2377 }
}LimX Tron IMU sample (6-DOF + orientation)
Representative of the real line. Acceleration in m/s^2, gyro in rad/s, plus a quaternion and a nanosecond stamp.
{
"timestamp": 0.009156,
"unix_timestamp": 1760757712.1551,
"stamp_ns": 10578143272597,
"accel": [9.4843, 0.0972, 2.7084],
"gyro": [0.000252, -0.0000055, -0.00882],
"quat": [0.2879, 0.5626, -0.2103, 0.7459]
}LimX Tron odometry sample (base pose + twist)
Representative of the real line. World position and orientation plus linear and angular velocity.
{
"timestamp": 0.008726,
"unix_timestamp": 1760758026.1290,
"pose_position": [0.7061, 0.3026, 0.1516],
"pose_orientation": [0.5434, -0.2517, 0.7253, 0.3395],
"twist_linear": [0.00143, 0.00146, 0.00063]
}Record shape
Every field, its type, whether it can be null, and a representative value.
| Field | Type | Constraint | Description |
|---|---|---|---|
| joint_sample_rate_hz | float64 · Hz | required | Measured joint sampling rate (75 to 920 Hz across platforms). e.g. 79.23 |
| positions | map<joint,float> · rad | required | Per-joint angular position (named joints). e.g. {"Left_Knee_Pitch":0.4141} |
| velocities | map<joint,float> · rad/s | required | Per-joint angular velocity. e.g. {"Left_Knee_Pitch":-0.0093} |
| efforts | map<joint,float> · Nm | required | Per-joint torque / effort. e.g. {"Left_Knee_Pitch":-4.24} |
| imu.accel | float[3] · m/s^2 | required | Linear acceleration (6-DOF IMU). e.g. [9.4843, 0.0972, 2.7084] |
| imu.gyro | float[3] · rad/s | required | Angular velocity. e.g. [0.00025, -0.0000, -0.0088] |
| imu.quat | float[4] | required | Orientation estimate (quaternion). e.g. [0.288, 0.563, -0.210, 0.746] |
| imu.avg_rate_hz | float · Hz | required | Measured IMU sample rate. e.g. 97.3 |
| odometry.pose_position | float[3] · m | required | Base position in world frame. e.g. [0.7061, 0.3026, 0.1516] |
| odometry.twist_linear | float[3] · m/s | required | Base linear velocity. e.g. [0.0014, 0.0015, 0.0006] |
| audio.sample_rate | int · Hz | required | Audio sample rate (wav). e.g. 16000 |
| timestamp | float64 · s | required | Episode-relative timestamp on every sample. e.g. 3.0976 |
| unix_timestamp | float64 · s | required | Absolute Unix timestamp on every sample, for cross-stream alignment. e.g. 1766526822.3984 |
| stamp_ns | int64 · ns | nullable | Nanosecond hardware stamp on IMU and motor streams. e.g. 10578143272597 |
Proprioception
Joint positions, velocities, and efforts at 75-920Hz, aligned to vision and action streams.
Inertial (IMU)
1000Hz 6-DOF inertial measurement for dynamics, balance, and contact modeling.
Time-Sync Fabric
Sub-millisecond hardware synchronization across all sensor and vision streams.
How it is built
- 01
High-frequency multi-sensor capture
Proprioception (joint and motor position, velocity, effort), IMU (acceleration, gyro, orientation), odometry, and audio are captured per episode at native rates.
- 02
Per-sample timestamping
Every sample carries both an episode-relative timestamp and an absolute Unix timestamp; IMU and motor streams additionally carry nanosecond stamps for fine alignment.
- 03
Cross-stream alignment
Streams are kept at native rates rather than resampled and aligned on a shared timeline. The format records a sync-tolerance budget; the architecture targets sub-millisecond hardware sync.
- 04
Per-stream accounting
Each stream sample count and average rate are recorded in metadata, enabling rate verification - for example motors at 915 Hz, IMU at 97 Hz, odometry at 96 Hz on one platform.
- 05
Integrity and packaging
Per-file SHA-256 checksums and total size are recorded, and streams are bundled in the ORTF episode structure with a manifest describing each stream type, rate, and encoding.
How we validate
What each evaluation measures and how it is run. Where no benchmark is published, we show the methodology and say so.
Sample-rate verification
Measures
Whether each stream actually achieved its target rate.
Method
Compare the recorded sample count and duration against the declared average rate and flag drops.
Result
Methodology-stage. Per-stream measured rates are recorded; no aggregate published metric.
Cross-stream sync tolerance
Measures
Whether modalities fall within the synchronization budget.
Method
The format defines a maximum acceptable inter-modality timestamp delta, and the validator checks timestamp monotonicity.
Result
Methodology-stage. Tolerance is a spec field; no measured sync-error figure is published.
Sim-to-real sensor grounding
Measures
Whether real high-frequency traces match simulated dynamics.
Method
Real joint and velocity traces are compared against MuJoCo replay.
Result
Real measured: velocity tracking 0.41 rad/s MAE and a dynamic gap score of 0.065 - the real velocity stream is what grounds this. See the sim2real paper.
Ground truth
What correct means for this data, and how it is established.
Ground truth
The physical sensor readings are themselves the ground truth. For simulation grounding, the real high-frequency velocity and position traces are the reference the simulation is scored against.
How it is established
Stream integrity is graded by SHA-256 checksums and per-stream sample-count and rate accounting; temporal validity by timestamp monotonicity; sim-to-real fidelity by MAE and RMSE against the real trace.
Agreement
No inter-rater agreement applies - sensor data is machine-measured. Sim-grounding correctness is anchored to the real trace.
Sensor Fusion Models
Train multi-modal encoders on tightly-synced vision, proprioception, and inertial data.
Sim-to-Real Calibration
Ground simulators against real high-frequency sensor traces to close the reality gap.
Cross-Embodiment Transfer
Aligned sensor schemas across platforms for policies that generalize between robots.
How you load it
Delivery
S3, REST API
Formats
JSONL streams (proprioception / IMU / odometry / motors), NPZ packed arrays, WAV audio
Auth
Org-scoped API keys, tenant isolation, and signed artifacts with checksums and access logging.
Cadence
Streamed with every collection run, emitted alongside each episode.
# Read and align a proprioception stream# 1. Load feed/proprioception/joints.jsonl - each line is one timestamped sample# 2. Each sample carries positions / velocities / efforts maps# 3. Align across streams on unix_timestamp (or stamp_ns where present)# 4. Verify rate via sample_count / avg_rate_hz in the metadata
Request access.
Restricted-scope evaluation access for qualified teams. We share real samples, full schema, and provenance under a mutual NDA.