(Image: *Map and photograph of the 200-m test area where the performance of different AV lidars was assessed. Courtesy of Jeffries et al. DOI: 10.1117/1.OE.62.3.031211*)

Efforts to standardize Lidar tests as part of research project

A three-year effort is underway to develop and test standards for automotive lidar performance.

James Davis

22 Jan 2023

Lidars lack a standardized measure of their performance which makes them crucial to autonomous vehicles and advanced driver assistance systems (ADAS).

To address this issue, SPIE in conjunction with Paul McManamon of Exciting Technology, have formed a national group to develop standard tests for lidar technologies over the past three years.

The team published a first-year test report in Optical Engineering and an outline of the more extended three-year project in the journal.

In these tests, eight automotive-grade lidars were measured against a survey-grade lidar on their range, accuracy, and precision. In Kissimmee (Florida), researchers set up a 200-meter path with various targets.

A distinctive feature of the targets was a 10% reflectivity calibrated over a broad spectrum, which differentiated them from previous studies. A lidar detection test was also conducted with highly reflective road signs to determine whether the lidars could detect the target.

It appears that the test results are generally in agreement with what the manufacturer advertises in its datasheets.

However, although all tested devices recorded a mean precision of 2.9 cm, the measured values were not distributed Gaussian.

In simple terms, these devices had a very high probability of reporting extremely inaccurate values (errors more significant than 10 cm). As much as 20 cm could deviate from the actual range in some cases.

Image: *Reflective road signs were placed near the targets to measure the precision and range of automotive lidars under more challenging conditions. Courtesy of Jeffries et al. DOI: 10.1117/1.OE.62.3.031211*

Furthermore, the lidars' target detection performance was adversely affected by the reflective road signs.

“The advertised range performance of lidars pertains to very specific conditions, and performance degrades significantly in the presence of a highly reflective adjacent object” MTU associate professor Jeremy Bos said.

Tests conducted in the first round revealed some important differences in performance between different lidars, which indicates that manufacturers' reports are not reliable. In spite of this, there is still much work and tests to be done.

“The first-year tests were the simplest of them. In the second year, we will duplicate these tests for the characterized lidars while introducing confusion resulting from other automotive lidars approaching from the opposite direction. Additionally, we will measure the eye safety of lidars” Bos said.

Finally, by the third year the team will build up complexity by including weather effects.

MTU associate professor Jeremy Bos conducted testing with the assistance of Ph.D. student Zach Jeffries during the first year of the project. Additionally, Charles Kerschner of the National Geospatial-Intelligence Agency built the Riegl lidar used in the test, and Akhil Kurup, also of MTU, contributed to the research paper.

Take a look at the paper for further information:

Research

Efforts to standardize Lidar tests as part of research project

James Davis

Featured Posts

Pedestrian Safety Concerns Enters the Mainstream