Why Our Results Are Robust, Reproducible and Comparable
Measurement technology behind ADAS benchmark
Advanced driver assistance systems such as adaptive cruise control, parking assistance and lane keeping respond to differences of just a few meters, fractions of a second and small degrees of vehicle movement. To evaluate these systems objectively, measurement technology must operate with the same level of precision. Our ADAS benchmark is therefore based on a highly precise measurement chain that synchronously captures vehicle motion, position, distance and dynamics with centimeter-level accuracy.
A valid evaluation framework emerges from the combination
No single measurement system can capture all parameters required to fully describe vehicle motion. For this reason, we combine multiple independent instruments into an integrated reference framework:
- Evaluation software
- Optical measuring tool (photogrammetry)
- Digital reference maps (ground truth)
- Differential GPS
- Integrated inertial system with GNSS coupling
Only through this combination does a comprehensive and objective overall picture emerge — forming a reliable basis for the evaluation of advanced driver assistance systems.
Evaluation software for objective KPI-based ADAS assessment
The extensive measurement data collected during test drives is processed with the evaluation software MXeval and transformed into meaningful key performance indicators (KPIs). This enables the behavior of advanced driver assistance systems to be objectively described, assessed and compared.
Examples of KPI evaluations derived from test drives:
-
ABS Vergleich
-
KPI Barplots
-
KPI Correlation mit Cluster
-
Radschlupf Analyse
-
Lateral Guidance: Free Run Auswertung
-
Heatmap mit Querbeschleunigung
-
GPS Geschwindigkeits Heatmap
Evaluation process with MXeval
- Consolidation of raw measurement data
All recorded vehicle dynamics parameters — such as steering torque and angle, vehicle position within the lane, or speed profiles — are synchronized within the MXeval software environment. - Derivation of KPIs (Key Performance Indicators)
The vehicle measurement data is translated into quantitative metrics that describe functional performance. These include, for example, steering intervention intensity of a lane-keeping system, time-to-intervention, or vehicle stability within the lane. - KPI classification (Targeting)
The derived metrics are compared against predefined target values from our evaluation libraries and systematically classified. - Live feedback in the vehicle
The evaluation process is triggered automatically and is available directly within the test vehicle. This enables immediate feedback on both the performance of the driver assistance system and the quality and validity of the executed test. - Interpretation through visualization
Results are displayed in diagrams with color coding. Deviations, strengths and weaknesses can thus be identified at a glance. - Integration of objective and subjective data
In parallel, subjective driver impressions are collected via MXevalApp and later linked to the objective measurement results. This creates a holistic assessment: not only technical performance, but also perceived behavior and comfort are incorporated into the evaluation.
Core benefits of MXeval in ADAS benchmark
- Measurement data is transformed into clear and comparable results (KPIs).
- Benchmarking based on standardized criteria.
- Objective vehicle measurements are linked with subjective driver feedback.
- Decision-making is accelerated.
Optical measuring tool for capturing steering wheel movement
In the ADAS benchmark, steering wheel movement is captured entirely optically using MXoptiCal — manufacturer-independent and without access to in-vehicle CAN data. This allows steering interventions of different parking assistance systems to be quantitatively compared, for example in terms of efficiency, dynamics, or the number of required corrective maneuvers.
Camera and coded reference markers for photogrammetric measurement of steering wheel movement:
Measurement principle with MXoptiCal
A coded reference marker mounted on the steering wheel is captured throughout the entire parking maneuver by an interior-mounted camera. The subsequent software-based analysis provides precise measurement data on steering wheel movement:
- absolute and relative rotation angles,
- total number of steering wheel revolutions,
- steering speed (revolutions per second),
- differentiation between steering movements at standstill and during driving.
The core benefits of optical measurement with MXoptiCal for an objective ADAS comparison
- Manufacturer-independent data acquisition.
- No integration into the vehicle network required.
- High reproducibility of measurements.
- Direct capture of the actual mechanical movement.
Digital maps as a geometric reference (ground truth)
High-resolution digital maps provide the geometric layout of the road and serve as a geometric reference (ground truth) for the evaluation of advanced driver assistance systems. They include, among other elements:
- lane widths,
- curve radii,
- lane centerlines.
This enables not only the determination of the vehicle’s absolute position, but also its relative position within the lane. This geometric ground truth is essential for the objective and reproducible evaluation of advanced driver assistance systems.
Digital reference map and real road view in comparison
Differential GPS for centimeter-level vehicle positioning
For absolute positioning, we use Differential GPS (D-GPS). In contrast to standard GPS, which typically achieves an accuracy of several meters, D-GPS enables positioning with centimeter-level accuracy. A reference station continuously calculates correction data for the received satellite signals, which are transmitted to the test vehicle in real time. Multiple GPS antennas mounted on the vehicle additionally allow precise determination of vehicle orientation and driving direction.
This ensures that we always know:
- where the vehicle is located,
- how it is oriented relative to the road and other traffic participants.
Test vehicle equipped with two roof-mounted antennas (WLAN, GPS/GLONASS/Galileo/BeiDou)
Integrated inertial system with GNSS coupling for describing vehicle motion
The integrated inertial system ADMA-G from GeneSys captures all vehicle movements in real time:
- acceleration and deceleration (longitudinal dynamics),
- cornering (lateral dynamics),
- pitch, roll, and yaw motions (body movements).
This sensor system operates independently of satellite signals and provides continuous motion data even in tunnels, dense traffic, or poor visibility conditions. It therefore forms the dynamic foundation for all subsequent calculations.
In test scenarios involving multiple vehicles, relative parameters such as distance or relative velocity are calculated in real time based on the position and motion data of multiple inertial systems. For this purpose, we use the DELTA add-on, which enables synchronized data exchange between measurement vehicles via WLAN.
Meaningful comparisons are built on reliable data
Our measurement chain delivers objective and technically robust data, from which a consistent overall picture of vehicle behavior is derived. On this basis, advanced driver assistance systems can be evaluated systematically. The high level of measurement accuracy also ensures reproducible test conditions, thereby creating the foundation for benchmarking assistance systems across different vehicles, manufacturers, and test series.
We provide the measurement foundation for a robust ADAS benchmark. Let’s talk about your requirements.