Not all driver assistance systems are safe

Not all driver assistance systems are safe.
Robo-Test reveals shortcomings of driver assistance systems.
Stuttgart, 26.04.2022 – Tests of commercial driver assistance systems show that critical situations are assessed incorrectly and can thus cause accidents. The Stuttgart-based start-up Robo-Test uses an AI-based solution for testing to check driver assistance systems, so-called Advanced Driver Assistance Systems (ADAS) in a simulator. The founding team of Robo-Test has been working on artificial intelligence for many years. With the goal of making autonomous systems safe and explainable, Robo-Test has presented a study on the safety of ADAS solutions. It revealed dangerous gaps in some systems.
The systems tested include MobileEye, a supplier of driver assistance systems to BMW and, at times, Tesla, and the MiniEye system, from a Chinese technology company.
To transparently safeguard autonomous driving, critical scenarios must be tested in which a driver assistance system could make mistakes. So far, (partially) autonomous vehicles have to cover kilometers in the triple-digit millions with test drives to cover as many critical cases as possible. “Theoretically, there are an infinite number of cases, so we use synthetic data and AI algorithms to systematically uncover where cognition gaps lie, i.e., the limits of safe behavior,” said Michael Weyrich, who heads the Institute for Automation Technology and Software Systems at the University of Stuttgart.
Robo-Test relies on AI to combine parameters, such as weather conditions, road conditions or the behavior of other road users. As part of the tests, Robo-Test recreated real accidents involving autonomously driving cars in a simulator. Reproducing errors is important, but the Robo-Test solution finds errors before they cause accidents. Robo-Test does this by systematically permuting test cases based on models and using AI algorithms to locate particularly critical parameters and cases. Based on the behavior of ADAS components in the simulator, targeted test cases are generated to further illuminate the limits of safe driving. “With this AI-based cognitive test, we can specifically identify and drive through critical scenarios. This makes the testing of autonomous systems transparent and efficient,” emphasizes the second founder, Christof Ebert.