OVERVIEW    PARTNERS    TUMTRAF DATASET    RESEARCH

 


 

TUMTraf Dataset: Multi-sensor data sets for mobility research

High resolution data sets

With a constellation of 7 sensor stations equipped with more than 60 state-of-the-art and multi-modal sensors, and with a road network coverage of approximately 3.5 kilometers, the TUMTraf Dataset offers mobility researchers, industry partners, public authorities and policy makers high-resolution curated data sets created by capturing real-world traffic along freeways, country roads and urban traffic intersections. The data sets contain on the one hand labelled, time-synchronized and anonymized multi-modal sensor data covering area-scanning cameras, doppler radars, lidars and event-triggered cameras for a variety of traffic and weather-related scenarios, and on the other hand abstract digital twins of the traffic objects with position and trajectory-related information.

Current Releases and Registration

The TUMTraf Dataset is available for download upon registration. Please REGISTER HERE to receive the download links. The complete set currently includes the following major releases containing more than 27GB of data.

Data set Description
R3 Spatiotemporal synchronized event-based and RGB camera data set.
R2 Four road scenarios of a busy intersection under different weather conditions with data from two cameras and two LiDARs.
R1 3 different traffic scenarios from the autobahn.
R0 Multiple sets of randomly selected camera and lidar sequences from the autobahn.

Release notes

January 2024: In the R3 release TUMTraf Event, synchronized image material between an Event-Based and an RGB camera, as well as a combined representation, is published. The data set contains over 21,900 2D bounding boxes in various traffic scenarios during the day and at night. TUMTraf Event enables, among other things, the research into data fusion of both sensor systems using a stationary intelligent infrastructure.

June 2023: In the R2 release, the focus is on traffic at a busy intersection, which we captured using labeled LiDAR point clouds and synchronized camera images. The annotations are available as 3D bounding boxes with track IDs. This release includes 4,800 camera images and 4,800 point clouds, which are annotated with a total of over 57,400 3D bounding boxes. Here, the dataset includes objects from ten classes that perform complex driving maneuvers, such as left and right turns, overtaking maneuvers, or U-turns. In addition to the dataset, we also provide the calibration data of the individual sensors so that data fusion is possible. Furthermore, we would like to refer to our TUMTraf-Devkit, which makes the processing of the data much easier. More details can be found at https://github.com/tum-traffic-dataset/tum-traffic-dataset-dev-kit.

May 2022: The R1 release comes with time-synchronized multi-sensor data recorded for 3 different traffic scenarios from the A9 autobahn. They include labelled ground truth data for a couple of typical extreme weather situations that usually occur on the autobahn during winter. Heavy snow combined with strong gusts of wind and dense fog have always posed challenges for vision-based driving assistance and automation systems. With this release we offer researchers and engineers a new set of extreme weather ground truth data for the development of robust and weather-proof vision based systems. The R1 release now also includes an extended version of the high speed crash incident on the autobahn which was made available as part of the previous release. The time periods before and after the crash can now be investigated in more detail with this extended version.

March 2022: The R0 set contains labelled multi-sensor data with a mix of random and sequential traffic scenarios from the A9 autobahn. The data from this set can be used as ground truth for the development and verification of AI-based detectors, tracking and fusion algorithms, and to understand and analyze the occurrence and the after-effects of a typical high speed crash incident on the autobahn.

Roadmap

The upcoming releases will contain amidst others, digital twins with information about trajectory and position, new traffic scenarios, longer sequences, and new locations including an urban traffic intersection. Keep watching this space for more.

Citation

  • The TUMTraf Dataset (previous A9 Dataset) was first presented in the scientific paper A9-Dataset: Multi-Sensor Infrastructure-Based Dataset for Mobility Research at the 33rd IEEE Intelligent Vehicles Symposium (June 5 – June 09, 2022 in Aachen, Germany).
  • The publication TUMTraf Intersection Dataset: All You Need for Urban 3D Camera-LiDAR Roadside Perception explains the R2 release and received the “Best Student Paper Award” at the IEEE ITSC 2023.
  • The scientific work TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras contains details and background information about the R3 release and is currently in a peer review process.



    We request users to cite this paper in your work.

  • Release R3 – Spatiotemporal synchronized event-based and RGB camera data set


    The TUMTraf Event Dataset contains synchronized images from an Event-Based and an RGB camera, as well as a combined representation with 21,900 2D bounding boxes. The dataset considers different traffic scenarios during the day and at night. TUMTraf Event enables, among other things, the research of the data fusion of both sensor systems using a stationary intelligent infrastructure. In this regard, our work allowed us to combine the strengths of both sensor systems while compensating for the weaknesses simultaneously. Therefore, our fusion led to a noticeable reduction in false positive detections of the RGB camera and, thus, increased detection performance. The results are shown in the video below:



    Release R2 – Complex traffic scenarios at a busy intersection


    R2_S1 and R2_S2 These sequences each contain 30 seconds of traffic events at a busy intersection, which were recorded and synchronized with two LiDARs and two cameras. The recordings were made during the day. The objects are labeled with 3D bounding boxes and unique track IDs. This enables continuous object tracking and data fusion.
    R2_S3 In contrast to R2_S1 and R2_S2, this sequence contains contiguous 120 seconds of traffic events at a road intersection, which were also recorded by 2 LiDARs and 2 cameras. The recordings were made during the day. The objects are labeled with 3D bounding boxes and unique track IDs, which allow continuous object tracking and data fusion.
    R2_S4 This sequence also contains a 30-second recording of the traffic event, which was created on a rainy night. Like the other datasets, the objects are also marked with 3D bounding boxes and unique track IDs.

    Release R1 - Traffic Scenarios from the Autobahn


    R1_S1 This set contains a 30s long multi-sensor sequence recorded in winter under heavy snow conditions. The data consists of time-synchronized images recorded at 10fps from 4 cameras observing a 400m long test stretch from multiple perspectives. The traffic objects are labelled with 3D bounding boxes and unique ids within each sensor frame to enable subsequent object-tracking and data-fusion.
    R1_S2 This set contains a 30s long multi-sensor sequence recorded during heavy fog conditions.
    R1_S3 This set contains a 60s long multi-sensor sequence covering the time period before, during and after the occurrence of a traffic accident.

    R1_S1 R1_S2 R1_S3

    Release R0 - Random Camera and Lidar Sequences from the Autobahn


    R0_S1 This set contains a random selection of around 600 images from four cameras equipped with two types of lenses. They are are mounted on overhead gantry signs along the A9 autobahn. The traffic objects are labelled with 3D bounding boxes and classified into one of 7 categories.
    R0_S2 This set contains a 25s sequence of images from a camera on the A9 autobahn and also captures a number of lane change maneuvers. Apart from 3D bounding boxes and object classes, unique track-ids are also assigned to the traffic objects.
    R0_S3 and R0_S4 These sets contain two different sequences of lidar point-cloud frames from overhead gantry bridges on the A9 freeway and are accompanied by labels including 3D bounding boxes and object classes.

    R0_S1 R0_S2 R0_S3 and R0_S4

    Further Articles

    February 10,2022

    TUMTraf Dataset: Frames and sequences of the A9 testbed

    MORE >

    February 10,2022

    TUMTraf Dataset: Frames and sequences of the A9 testbed

    MORE >

    July 31,2021

    Predicting movements by autonomous agents

    To predict movements, many trajectory options must be taken into consideration and prioritized. The FloMo model works with probabilities and has been trained on the basis of three datasets. Its performance is improved by a proposed method of stabilizing training flows.

    MORE >