A dataset for autonomous driving, incorporating lidar, camera and radar sensors, seven simultaneous target vehicles with INS reference poses and 3D scans.

Authors: Philipp Berthold*, Bianca Forkel*, and Mirko Maehlisch (* contributed equally)

Abstract: Datasets are a crucial element in the development of perception algorithms. They relate sensor measurement data to annotated reference information and allow for the deduction of sensor and object characteristics. In autonomous driving, the reference data commonly consist of semantic image segmentation, point-wise associations, or bounding box annotations. The dataset proposed in this work, however, aims to dig deeper into the evaluation of measurement principles and provides scanned 3D models of all vehicles together with a pose and continuous kinematics reference obtained by RTK-GNSS. Combined, the state of the complete dynamic surrounding of the sensor vehicle is known for any point in time. Subsequent reference formats can be easily computed in user-defined granularity. This dataset involves single-object and multi-object recordings with seven target vehicles. In particular, measurement effects such as occlusion, as well as reflections, can be evaluated, as the normals of the shape of the target vehicles are known. We describe the dataset, discuss the technical background of its development, and briefly present exemplary evaluations.

Citation: If you use this dataset, please cite our paper such as

@InProceedings{7V-Scanario_2025,
  author    = {Philipp Berthold AND Bianca Forkel AND Mirko Maehlisch},
  booktitle = {Symposium Sensor Data Fusion (SDF)},
  title     = {{A Multi-Vehicle Dataset with Camera, LiDAR, and Radar Sensors and Scanned 3D Models for Custom Auto-Annotation using RTK-GNSS}},
  year      = {2025}
}

Example videos:

Multi-object scenario (rendered in rviz, without time-synchronization):

3D-Scan of a compact car (succeeded Artec Studio scan registration):

Artifical lighting (may be used for physical data augmentation), shown in Blender:

Heatmap for radar detections (rendered in MATLAB as normalized 360°-view accumulation):

Index:

• Vehicles: Description of the utilized vehicles
• Sensors: Description of the utilized sensors
• Recordings: Description of the recorded single-vehicle and multi-vehicle recordings
• ROS Topics: Description of all ROS topics
• Download: Available material of this dataset (ROS bags, 3D scans, offline INS logs) and how to get all data
• Example ROS code: Manual to run a rough data inspection in rviz

License: on request

The ROS bag contains the sensor data of the ego vehicle and all received WiFi data (INS data from target vehicles). We provide both raw Ethernet data and processed data for the local lidar, radar and vehicle (odometry) sensors and all INS units. This allows for utilizing pre-processed sensor data, and also for the utilization of custom data parsers / decoders. In latter case, the timestamp of the ethernet_msgs/Packet message header should be used for timestamp association. Concerning the camera sensors, raw images (+ camera info) are provided. Extrinsic sensor calibration is provided by the tf_static topic.

Most ROS message type formats are standard ROS messages, the remaining message definitions can be obtained from:

This section describes the download of the measurement recordings (ROS bags), the 3D scans, and additional INS data.

All ROS bags are compressed and can be downloaded separately. Simply download them to /path/to/data/ and execute:

cd /path/to/data && for f in *.tar.xz; do tar -xJf "$f"; done

Download: https://huggingface.co/datasets/UniBw-AD/7V-Scanario / recordings

The scans provide 3D contour and texture information for each vehicle used in this dataset. Each scan is aligned to the coordinate system (translation and rotation) of the output data of the INS of the vehicle. Background: while some INS units use the reference point of the IMU as the zero-point of its coordinate system, others use the centroid of the vehicle. The reference point is selected for best measurability. As the 3D scans are already aligned accordingly, you do not need to perform any object coordinate transformations.

The 3D scans were obtained with the Artec Leo 3D hand scanner and using the Artec Studio for registration. Please contact us if you need the Artec Studio project files including the raw measurements (between 50GB and 200GB per vehicle).

The processed 3D scans are provided as Wavefront Object (.obj-files with texture). You can use software like Blender to convert the objects to a format of your choice.

Download: https://huggingface.co/datasets/UniBw-AD/7V-Scanario / 3D-models/obj.tar.xz

We also provide a downsampled version of each vehicle as Collada file (.dae) for a straight-forward usage in rviz (using Markers) for quick data inspection. Please note that loading all objects simultaneously still requires high computing resources, which might cause lags in rviz.

Download: https://huggingface.co/datasets/UniBw-AD/7V-Scanario / 3D-models/dae.tar.xz

The ROS bags contain the poses of the ego vehicle and all target vehicles (see ROS topic description). This data has been received via long-range WiFi during the recordings. As a stable WiFi connection cannot be retained at high distances, the INS additionally logged their position solution (and some raw data) to a local storage. This data provides continuous positional and kinematic information. In addition, you can also use this data for post-processing (that usually yields a better estimation quality than the live solution). A manual for the installation and operation of the processing software can be found here: OxTS (INS) processing of RD files to NCOM or CSV. The INS data is sorted by vehicle and time.

Download: https://huggingface.co/datasets/UniBw-AD/7V-Scanario / INS/INS.tar.xz (sorted by vehicle and time)

Please contact us if you need the raw data of the local GNSS reference station.

This section outlines the installation of the ROS package that can be used for a rough data inspection (sensor data visualization + 3D models) of the dataset data in rviz. Please note that rviz does not perform time synchronization of the visualized data (e.g., between all sensor measurements and the INS units that provide the poses of all 3D objects), which causes significant deviations of the visualized data. For any kind of data evaluation and processing, you need to use the provided timestamp in the ROS message headers for time synchronization. The rviz rendering should only be used for rough data inspection.

1. Add object_msgs (see dependencies) to your catkin workspace.
2. (optionally, to use raw data) Also add the other packages from dependencies.
3. Add the contents of the folder ros of this repository to your catkin workspace.
4. Download the Collada files and unpack them to the meshes folder in the ROS package, i. e., to scanario/meshes/.
5. Compile your workspace using catkin build.
6. Start the launch file roslaunch scanario inspect.launch bag:=<path_to_bag>, where <path_to_bag> corresponds to the ROS Bag (path and filename) you would like to investigate, for example, roslaunch scanario inspect.launch bag:=/mnt/data/multi-vehicle/09_highway/2024-12-05-12-47-21.bag. The bag will be played, and an example rviz layout is started.
7. To view the 3D models, activate "mesh" (to be found in Tab Displays -> Target Objects -> Marker -> Namespaces). Please note that the initial loading of the 3D models will take up to a minute.