Dense Reinforcement Learning for Safety Validation of Autonomous Vehicles

One critical bottleneck that impedes the development and deployment of autonomous vehicles is the prohibitively high economic and time costs required to validate their safety in a naturalistic driving environment, owing to the rarity of safety-critical events. Here we report the development of an intelligent testing environment, where artificial-intelligence-based background agents are trained to validate the safety performances of autonomous vehicles in an accelerated mode, without loss of unbiasedness. From naturalistic driving data, the background agents learn what adversarial manoeuvre to execute through a dense deep-reinforcement-learning (D2RL) approach, in which Markov decision processes are edited by removing non-safety-critical states and reconnecting critical ones so that the information in the training data is densified. D2RL enables neural networks to learn from densified information with safety-critical events and achieves tasks that are intractable for traditional deep-reinforcement-learning approaches. We demonstrate the effectiveness of our approach by testing a highly automated vehicle in both highway and urban test tracks with an augmented-reality environment, combining simulated background vehicles with physical road infrastructure and a real autonomous test vehicle. Our results show that the D2RL-trained agents can accelerate the evaluation process by multiple orders of magnitude (103 to 105 times faster). In addition, D2RL will enable accelerated testing and training with other safety-critical autonomous systems. 

Safe AI Framework for Trustworthy Edge Scenario Test (SAFE-TEST)

It is well-known that scenario generation is essential for testing and evaluation of automated vehicles (AVs), but how to generate realistic and trustworthy scenarios efficiently remains an open question. To address this challenge, in the past few years, University of Michigan’s Center for Connected and Automated Transportation (CCAT) has developed a scenario generation toolbox that can accelerate the development and validation of automated vehicles. This toolbox, which has been implemented at the American Center for Mobility (ACM), includes the augmented reality (AR) testing platform and the naturalistic and adversarial driving environment (NADE). With AR, a real AV can be tested at a test track with interaction from virtual traffic flow. With NADE, the maneuvers of virtual background vehicles will be controlled intelligently, in that most of scenarios are generated from naturalistic driving data, and only at selected moments, adversarial scenarios are generated to challenge the AV under test. The theory behind NADE ensures both the unbiasedness and the efficiency of the testing scenario generation. With this toolbox, every testing mile at ACM can be converted into thousands of equivalent miles on public roads, which can significantly reduce the development costs and shorten the development cycle. In this work, we demonstrate its effectiveness by testing a L4 experiment vehicle at ACM.

Intelligent Driving Intelligence Test for Autonomous Vehicles with Naturalistic and Adversarial Environment

Driving intelligence tests are critical to the development and deployment of autonomous vehicles. The prevailing approach tests autonomous vehicles in life-like simulations of the naturalistic driving environment. However, due to the high dimensionality of the environment and the rareness of safety-critical events, hundreds of millions of miles would be required to demonstrate the safety performance of autonomous vehicles, which is severely inefficient. We discover that sparse but adversarial adjustments to the naturalistic driving environment, resulting in the naturalistic and adversarial driving environment, can significantly reduce the required test miles without loss of evaluation unbiasedness. By training the background vehicles to learn when to execute what adversarial maneuver, the proposed environment becomes an intelligent environment for driving intelligence testing. We demonstrate the effectiveness of the proposed environment in a highway-driving simulation. Comparing with the naturalistic driving environment, the proposed environment can accelerate the evaluation process by multiple orders of magnitude. 

Safety Assessment of Highly Automated Driving Systems: A New Framework

Safety assessment is critical in the development and deployment of highly automated driving systems (ADS). A new framework is proposed for closed-facility testing, which can quantitatively, accurately, and efficiently assess the safety of highly ADS in a cost-effective fashion. To this end, two major problems of closed-facility testing approach are resolved by two pillars of the framework. First, an augmented reality (AR) testing platform is constructed to augment the real ADS interacting with simulated background traffic. Second, a testing scenario library generation (TSLG) method is designed to systematically generate a set of critical scenarios for each operational design domain (ODD). By the important sampling theory, generating a library is transformed as constructing an importance function. A new definition of criticality and critical scenario searching methods are proposed. The framework is implemented in the Mcity test facility at the University of Michigan. Field test results validate the accuracy and efficiency of the proposed framework. In the cut-in case study, the proposed framework can accelerate the safety assessment process by times faster than the public-road testing approach.