ACME Broadens Social Navigation Data Beyond Single-Site Robots
Eight sites across five countries and seven robot embodiments add 72.1K verified trajectories for testing navigation and prediction under varied crowds.
Underlying Paper
ACME: A Multi-Cultural, Multi-Embodiment Social-Navigation Dataset
Understanding how robots and humans move in shared spaces is essential for designing effective social robot navigation policies and predicting human behavior. However, existing datasets often lack the diversity needed to capture differences in culture, geography, and human-robot interaction-factors that strongly shape appropriate social behavior. To address this gap, we introduce ACME: A Cross-cultural, Multi-Embodiment dataset for social navigation. A large-scale data collection effort across 8 sites in 5 countries, using 7 robot embodiments, ACME is a large and diverse multi-modal dataset aimed at advancing social navigation research, providing 29.35 hours of onboard robot data and 43.5 hours of overhead pedestrian tracking data. Unlike prior datasets, it focuses on capturing goal-driven social navigation behavior in complex social scenarios with explicit robot-crowd interaction through robot speech. To facilitate learning navigation policies and predicting pedestrian trajectories, ACME provides 3D and 2D scene features, odometry, interaction information, and human-annotated pedestrian trajectory labels. We make ACME easy to use by providing both human-readable data for each sensor modality as well as raw binary data. Our qualitative and quantitative analyses show that our dataset captures more challenging scenarios and a broader distribution of pedestrian behavior than previous datasets.
Social navigation datasets often compress a hard robotics problem into a narrow setting: one geography, one robot body, one navigation stack, and a limited range of pedestrian behavior. ACME targets the missing variation. The paper introduces a cross-cultural, multi-embodiment dataset collected across eight sites in five countries, with robot-crowd interaction, synchronized onboard sensing, overhead bird’s-eye-view pedestrian tracking, and human-verified trajectory labels.
Core Contribution
The main contribution is not a new navigation policy. It is a dataset designed to make social-navigation evaluation less dependent on a single lab or campus. ACME combines 29.35 hours of onboard robot data with 43.5 hours of overhead tracking data, yielding 72.1K human-verified pedestrian trajectories in metric space. The authors argue that this scale and diversity expose failure modes that smaller pedestrian and robot-navigation datasets tend to miss: different walking speeds, density regimes, passing-side preferences, and static-versus-dynamic pedestrian mixtures.
Figure 1 summarizes the dataset release: synchronized multi-sensor data, raw ROS2 bags, BEV video, camera calibration for metric trajectory recovery, and human-verified pedestrian paths.
Technical Approach
ACME’s collection design ties together robot-mounted sensors and overhead BEV cameras. The robot-side data include modalities such as 3D LiDAR, RGB or RGB-D streams, and odometry depending on the platform. The overhead view provides pedestrian trajectories that are transformed into metric coordinates through calibration. Position synchronization uses shared tags visible to both robot and BEV cameras, while time synchronization uses QR-code timestamps or event matching where direct integration is unavailable.
The dataset also includes a human verification step rather than relying only on automatic trackers. The verification interface lets annotators inspect video playback and correct tracking errors manually. That matters because the downstream tasks are sensitive to small trajectory errors: a poor label can change a measured closest approach, passing direction, or trajectory-prediction endpoint.
The genuinely new part is the combination of scale, geographic spread, robot embodiments, and explicit social interaction. Prior social navigation datasets in the comparison table often contain one robot, fewer locations, shorter duration, or no overhead pedestrian annotation. ACME is reported with 3,013 robot trajectories and 1,761 minutes in the social-navigation comparison table, alongside human-operated and teleoperated navigation across indoor and outdoor settings.
Results and Analysis
The evidence is strongest where the paper characterizes ACME as a broader and often harder benchmark. Against trajectory datasets with metric human annotations, ACME is much larger: 2,613 minutes and 72.1K trajectories, compared with THÖR at 60+ minutes and 600+ trajectories, TBD at 626 minutes and 10.3K trajectories, and THÖR-MAGNI at 1,416 minutes and about 10K trajectories. The trajectory statistics also show broader variation. Average motion speeds differ by site, with examples such as 1.43 m/s at UBonn, 0.97 m/s at Keio, 1.42 m/s at NUS, 0.97 m/s at Miraikan, and 1.27 m/s at UMich in the open-area cluster.
The benchmark results support a more nuanced claim: ACME exposes challenging conditions for existing models, but the comparison is not uniformly worse across every baseline and dataset. For foundation navigation models, ViNT has 0.85 ± 0.60 m FDE on ACME versus 0.46 ± 0.52 m on SCAND and 0.83 ± 0.49 m on MuSoHu. NoMaD shows 1.44 ± 1.00 m FDE on ACME versus 0.79 ± 0.92 m on SCAND and 1.53 ± 0.88 m on MuSoHu. The direction metrics still show notable degradation relative to SCAND: NoMaD’s total AOE is 93.88° ± 158.00° on ACME, compared with 32.89° ± 85.83° on SCAND.
Trajectory prediction results show a clearer difficulty increase relative to ETH/UCY, with a useful caveat. SocialGAN records 0.34/0.71 ADE/FDE on ETH/UCY and 0.40/0.80 on ACME, while TUTR records 0.22/0.44 on ETH/UCY and 0.31/0.59 on ACME. When ACME is restricted to dynamic pedestrians with average speed above 0.5 m/s, performance drops further; SocialGAN reaches 0.75/1.50 and TUTR reaches 0.56/1.09. That split is informative because ACME contains static and slow-moving pedestrians, which are realistic in museums and crowded public settings but can make aggregate errors look easier than the dynamic-interaction subset.
Limitations
The dataset’s breadth comes with unevenness. The appendix documents missing or degraded streams in several subsets: Miraikan and Keio have 2.2 hours without image streams, CMU lacks released velocity information, UEx needed velocity recovery after a ZED 2i ROS driver issue, and some UBonn camera trajectories may have invalid extrinsics due to a mounting problem. The benchmarks are therefore best read as a strong dataset and evaluation contribution, not as a final ranking of navigation systems across every sensing condition.
Evidence Box
strongKey Claims
- •ACME captures broader cross-cultural and embodiment variation than prior social-navigation datasets
- •Human-verified BEV trajectories enable pedestrian prediction and navigation benchmarking
- •ACME provides a challenging evaluation setting, with trajectory-prediction models performing worse than on ETH/UCY and navigation-model comparisons showing mixed but informative degradation across baselines
Key Results
- •29.35 hours of onboard robot data and 43.5 hours of overhead pedestrian tracking
- •72.1K human-verified trajectories over 2,613 minutes at 10 Hz
- •ViNT FDE 0.85 ± 0.60 m on ACME vs 0.46 ± 0.52 m on SCAND and 0.83 ± 0.49 m on MuSoHu
- •TUTR ADE/FDE 0.31/0.59 on ACME vs 0.22/0.44 on ETH/UCY, rising to 0.56/1.09 on ACME trajectories above 0.5 m/s
Limitations & Caveats
- •Sensor availability and calibration differ across subsets
- •Miraikan and Keio include 2.2 hours without image streams
- •CMU released processed data lacks velocity information
- •Some UBonn camera trajectories may have invalid extrinsics from camera mounting motion
