Introduction
LeRobotDataset v3 is a file-based multimodal robot-data format designed to store many episodes across Parquet and MP4 files while metadata reconstructs episode boundaries, tasks and feature schemas. Tabular state, action and timestamp data live in Parquet; visual observations are stored as video.
A shared file format does not make two robot datasets behaviorally compatible. Action dimensions, units, camera geometry, control rate, robot morphology and calibration still have to match or be transformed explicitly before datasets are merged for training.
Key findings
- LeRobotDataset v3 stores tabular state, action and timestamp features in Parquet and camera streams in MP4 while metadata describes schema, tasks and episode boundaries.
- Version 3 stores many episodes per data or video file instead of requiring one file per episode, improving large-dataset handling and streaming.
- Metadata can reconstruct episode ranges, feature definitions, FPS, statistics and task information without loading every media file.
- Recording pipelines must close and finalize writers correctly before a dataset is treated as complete or pushed to the Hub.
- Schema compatibility is not action compatibility: units, joint order, camera layout and robot geometry must still be checked before cross-dataset training.
LeRobotDataset v3 structure and compatibility checks
The storage format is separated from the robot semantics needed for training.
| Layer | LeRobotDataset v3 role | What still needs verification |
|---|---|---|
| Parquet | Stores numeric state, actions, timestamps and other tabular features. | Action units, joint order, coordinate frames and control frequency. |
| MP4 video | Stores camera observations efficiently in chunked files. | Camera intrinsics, viewpoint, frame timing and compression effects. |
| Metadata | Describes features, tasks, statistics and episode/file offsets. | Whether metadata includes the robot revision, firmware and calibration needed for reproduction. |
| Streaming | Allows large datasets to be consumed without downloading every file first. | Network/cache behavior and whether training preserves episode boundaries. |
| Migration | LeRobot provides migration paths from older dataset versions. | Converted structure does not repair missing labels or incompatible action semantics. |
Format compatibility is necessary for tooling, but it is not sufficient evidence that two datasets can train one policy without normalization or embodiment mapping.
Evidence classification
- Officially documented specifications, standards or project status stated by the responsible organization.
- Real-system evidence demonstrations or deployments performed on physical hardware under described conditions.
- Company claim a numerical or operational statement reported by the company and not independently audited.
- Simulation or research evidence useful for mechanisms, but not proof of field deployment.
- Insufficient public evidence control mode, trial count, version or operating conditions are missing.
LeRobotDataset v3 storage model
Version 3 is organized around chunked Parquet and MP4 files rather than one physical file for every episode. Episode metadata records where each episode begins and ends inside those files.
This structure reduces file-count overhead and supports larger datasets while preserving episode-level indexing.
What lives in Parquet and MP4
Numeric observations, robot state, actions and timestamps are stored in Parquet. Camera streams are encoded as video, typically MP4, and linked back to episode timing through metadata.
Training code still needs consistent timestamps. A video frame associated with the wrong action can teach the policy the inverse of the intended correction.
Metadata is part of reproducibility
Feature definitions, FPS, task labels, statistics and episode offsets determine how raw files become training examples. Robot revision, firmware, calibration and camera placement are also valuable even when they are not mandatory for the file format.
A dataset card should document those experimental fields because identical feature names can hide different physical meanings.
Recording and finalization
A recorder should verify episode lengths, dropped frames and timestamp monotonicity before publication. LeRobot’s dataset writer also needs to be finalized so open Parquet and video writers are closed and metadata is complete.
Publishing an interrupted recording directory can produce a dataset that looks present on disk but is incomplete for streaming or training.
Cross-dataset training requires semantic alignment
Before mixing datasets, compare action dimensions, units, coordinate frames, control rate, gripper conventions, camera count and task definitions. Rename maps can align feature names but cannot make different robot kinematics equivalent.
For multi-robot training, document the embodiment mapping or model mechanism that handles morphology rather than hiding the conversion in preprocessing.
Versioning and migration
LeRobot provides migration tooling from older formats such as v2.1 to v3.0. Migration changes representation, not the underlying data quality.
Keep the original dataset version and conversion tool version in the experiment record so training can be reproduced later.
Licensing and consent
Code, dataset and captured-video rights can differ. Each Hub dataset has its own license and data-collection context.
Before reuse, verify whether the license permits the intended training, redistribution and commercial use, and whether human recordings have the necessary consent.
Limitations and missing information
- Format versions can change.
- Dataset owners define licensing and consent.
- Standardization does not solve cross-embodiment control.
- Specifications, prices, repositories and deployment status can change after publication.
- Benchmarks from different robots or environments are not directly comparable.
Conclusion
LeRobotDataset v3 makes robot datasets easier to stream, version and inspect by combining Parquet, MP4 and structured metadata. Its biggest limitation is also important: a common container does not create a common robot action space. Reusable datasets need timing, calibration, action semantics, embodiment and licensing documented alongside the files.
Sources and methodology
Rechecked September 26, 2026 against current Hugging Face LeRobotDataset v3 documentation and dataset APIs. Storage-format claims are taken from the official documentation; cross-embodiment and licensing limits are kept as separate engineering checks.
Related TechniaHQRobot guides
Source review notes
Review date recorded July 11, 2026
Supported by the cited sources
- MP4 or image observations are synchronized with numeric features.
- State, actions and metadata are stored efficiently for training and streaming.
Not confirmed or incomplete
- Format versions can change.
- Dataset owners define licensing and consent.
- Standardization does not solve cross-embodiment control.
Likely to change quickly
- Commercial availability, prices, model versions and software access.
- Deployment counts, company partnerships and repository maintenance status.
Share this article
Share the current TechniaHQRobot article page.
Continue reading
Open the latest robotics reporting, Physical AI analysis and hardware notes.