Training Data for Physical AI

Multi-modal robotics datasets from real-world deployments. Collected across multiple morphologies in diverse environments—retail, warehouses, public spaces. Hardware-synchronized sensor streams with human-verified labels.

Hydron8nSanctuaryMaverickRobot PlatformRobot PlatformRobot PlatformRobot PlatformRobot Platform

Collection Methodology

Training physical AI requires data that captures the complexity of real-world interactions: contact-rich manipulation, dynamic locomotion, occlusion, lighting variance, and failure modes that don't exist in simulation.

Hardware-Synchronized Sensor Fusion

All sensor streams synchronized at the hardware level with sub-millisecond precision. Dual RGB cameras (external + egocentric), proprioception at 75-920Hz, IMU at 1000Hz, and audio streams share a unified clock for temporal alignment.

Vision: 30fps H.264Proprioception: 75-920HzIMU: 1000HzAudio: 44.1kHz

Human Verification

Every episode manually reviewed for success/failure classification by domain experts. Scene annotations include object interactions, environmental context, and failure mode categorization. Inter-annotator agreement measured and reported.

Annotation protocol includes first-frame scene description, per-episode success labels, and structured failure taxonomy.

Environment Diversity

Data collected across 10+ industries and environment types: retail stores, warehouses, public spaces, industrial facilities. Varied lighting conditions, floor surfaces, human density, and object distributions capture the long-tail of deployment scenarios.

15+ robot morphologies including humanoids, bipedal platforms, quadrupeds, and mobile manipulators.

Data Specifications

Production-ready formats designed for ML training pipelines.

Sensor Modalities

RGB Vision + Depth1920×1080 @ 30fps
Proprioception75-920Hz
IMU (Gyro + Accel)1000Hz
Audio44.1kHz WAV
Sync Precision<1ms

Data Format

Video EncodingH.264
Sensor LogsJSONL
TimestampsUnix epoch (ns)
AnnotationsJSON
DeliveryCDN + API

Annotation Schema

• Success/failure labels with confidence scores and failure mode taxonomy

• First-frame scene descriptions including object counts, environmental context, task intent

• Per-frame object bounding boxes and segmentation masks (available on request)

• Episode-level metadata: duration, environment type, robot morphology, task category

Training Applications

Data structured for imitation learning, reinforcement learning, and foundation model pre-training.

Imitation Learning

Success-labeled trajectories with complete state-action pairs. Human demonstrations of manipulation tasks, locomotion patterns, and multi-step procedures suitable for behavior cloning and inverse RL.

Reinforcement Learning

Deploy your models on our robot fleet for real-world training and evaluation. Collect task-specific data with your policies, enabling iterative improvement and deployment validation.

Foundation Models

Large-scale multi-modal data for pre-training generalist robot policies. Vision-language-action triplets with natural language task descriptions and environment context.

Evaluation & Benchmarking

Held-out test sets across diverse environments and morphologies. Assess sim-to-real transfer, distribution shift robustness, and long-tail capability.

Get Started

Browse public datasets or request custom data collection for your specific research needs.