Automated Driving System Testing and Evaluation
Dense Reinforcement Learning for Safety Validation of Autonomous Vehicles
One critical bottleneck that impedes the development and deployment of autonomous vehicles is the prohibitively high economic and time costs required to validate their safety in a naturalistic driving environment, owing to the rarity of safety-critical events. Here we report the development of an intelligent testing environment, where artificial-intelligence-based background agents are trained to validate the safety performances of autonomous vehicles in an accelerated mode, without loss of unbiasedness. From naturalistic driving data, the background agents learn what adversarial manoeuvre to execute through a dense deep-reinforcement-learning (D2RL) approach, in which Markov decision processes are edited by removing non-safety-critical states and reconnecting critical ones so that the information in the training data is densified. D2RL enables neural networks to learn from densified information with safety-critical events and achieves tasks that are intractable for traditional deep-reinforcement-learning approaches. We demonstrate the effectiveness of our approach by testing a highly automated vehicle in both highway and urban test tracks with an augmented-reality environment, combining simulated background vehicles with physical road infrastructure and a real autonomous test vehicle. Our results show that the D2RL-trained agents can accelerate the evaluation process by multiple orders of magnitude (103 to 105 times faster). In addition, D2RL will enable accelerated testing and training with other safety-critical autonomous systems.
Safe AI Framework for Trustworthy Edge Scenario Test (SAFE-TEST)
It is well-known that scenario generation is essential for testing and evaluation of automated vehicles (AVs), but how to generate realistic and trustworthy scenarios efficiently remains an open question. To address this challenge, in the past few years, University of Michigan’s Center for Connected and Automated Transportation (CCAT) has developed a scenario generation toolbox that can accelerate the development and validation of automated vehicles. This toolbox, which has been implemented at the American Center for Mobility (ACM), includes the augmented reality (AR) testing platform and the naturalistic and adversarial driving environment (NADE). With AR, a real AV can be tested at a test track with interaction from virtual traffic flow. With NADE, the maneuvers of virtual background vehicles will be controlled intelligently, in that most of scenarios are generated from naturalistic driving data, and only at selected moments, adversarial scenarios are generated to challenge the AV under test. The theory behind NADE ensures both the unbiasedness and the efficiency of the testing scenario generation. With this toolbox, every testing mile at ACM can be converted into thousands of equivalent miles on public roads, which can significantly reduce the development costs and shorten the development cycle. In this work, we demonstrate its effectiveness by testing a L4 experiment vehicle at ACM.
Intelligent Driving Intelligence Test for Autonomous Vehicles with Naturalistic and Adversarial Environment
Driving intelligence tests are critical to the development and deployment of autonomous vehicles. The prevailing approach tests autonomous vehicles in life-like simulations of the naturalistic driving environment. However, due to the high dimensionality of the environment and the rareness of safety-critical events, hundreds of millions of miles would be required to demonstrate the safety performance of autonomous vehicles, which is severely inefficient. We discover that sparse but adversarial adjustments to the naturalistic driving environment, resulting in the naturalistic and adversarial driving environment, can significantly reduce the required test miles without loss of evaluation unbiasedness. By training the background vehicles to learn when to execute what adversarial maneuver, the proposed environment becomes an intelligent environment for driving intelligence testing. We demonstrate the effectiveness of the proposed environment in a highway-driving simulation. Comparing with the naturalistic driving environment, the proposed environment can accelerate the evaluation process by multiple orders of magnitude.
Safety Assessment of Highly Automated Driving Systems: A New Framework
Safety assessment is critical in the development and deployment of highly automated driving systems (ADS). A new framework is proposed for closed-facility testing, which can quantitatively, accurately, and efficiently assess the safety of highly ADS in a cost-effective fashion. To this end, two major problems of closed-facility testing approach are resolved by two pillars of the framework. First, an augmented reality (AR) testing platform is constructed to augment the real ADS interacting with simulated background traffic. Second, a testing scenario library generation (TSLG) method is designed to systematically generate a set of critical scenarios for each operational design domain (ODD). By the important sampling theory, generating a library is transformed as constructing an importance function. A new definition of criticality and critical scenario searching methods are proposed. The framework is implemented in the Mcity test facility at the University of Michigan. Field test results validate the accuracy and efficiency of the proposed framework. In the cut-in case study, the proposed framework can accelerate the safety assessment process by times faster than the public-road testing approach.
Mcity Augmented Reality
This technology develops an augmented reality environment for connected and automated vehicle (CAV) testing and evaluation, in which background traffic is generated in microscopic simulation and provided to testing CAVs. The augmented reality combines the real-world testing facility (Mcity) and a simulation platform, in which movements of testing CAVs and traffic signals in the real-world can be synchronized in simulation, while simulated traffic information can be provided to testing CAVs. Testing CAVs “think” they are surrounded by other vehicles and adjust behaviors accordingly. At the same time, behaviors of simulated vehicles are also influenced by the testing CAVs. Information between real world and simulation is transmitted through DSRC.
1. Feng Y., Yu C., Xu S., Liu H. X. and Peng H. (2018). An Augmented Reality Environment for Connected and Automated Vehicle Testing and Evaluation, 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, , pp. 1549-1554. [PDF]
2. Feng Y. and Liu H.X. (2019). Self-Driving Cars Learn About Road Hazards Through Augmented Reality. IEEE Spectrum. [Link]
3. Feng Y. and Liu H.X. (2019). Real World Meets Virtual World: Augmented Reality Makes Driverless Vehicle Testing Faster, Safer, and Cheaper. Mcity white paper. [PDF]
4. Feng S., Feng Y., Yu, C., Zhang Y., and Liu H.X. (2020). Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology. IEEE Transactions on Intelligent Transportation Systems. [PDF]
5. Feng S., Feng Y., Sun H., Bao S., Zhang Y., and Liu H.X. (2020). Testing Scenario Library Generation for Connected and Automated Vehicles, Part II: Case Studies. IEEE Transactions on Intelligent Transportation Systems. [PDF]
6. Feng S., Feng Y., Sun H., Zhang Y., and Liu H.X. (2020). Testing Scenario Library Generation for Connected and Automated Vehicles: An Adaptive Framework. IEEE Transactions on Intelligent Transportation Systems. [PDF]
7. Feng S., Feng Y., Yan X., Shen S., Xu S., and Liu H.X. (2020). Safety Assessment of Highly Automated Driving Systems in Test Tracks: A New Framework. Accident Analysis and Prevention Volume 144, 105664. [PDF]
8. Feng, S., Yan, X., Sun, H., Feng Y., and Liu H.X. (2021) Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment. Nat Commun 12, 748. https://doi.org/10.1038/s41467-021-21007-8 [PDF] [Link]
9. Liu L., Feng S., Feng Y., Zhu X., Liu H.X. (2022). Learning-Based Stochastic Driving Model for Autonomous Vehicle Testing. Transportation Research Record. 2022; 2676(1):54-64. doi:10.1177/03611981211035756 [PDF]
10. Feng, S., Sun, H., Yan, X., Zhu, H., Zou, Z., Shen, S., and Liu H.X. (2023). Dense reinforcement learning for safety validation of autonomous vehicles. Nature 615, 620–627. https://doi.org/10.1038/s41586-023-05732-2
Liu, H. and Feng, Y. (2017) Simulated Vehicle Traffic for Autonomous Vehicles, US Provisional Patent, Application No. 62/500,299, Filed on May 2, 2017. [PDF]
The paper titled "Dense Reinforcement Learning for Safety Validation of Autonomous Vehicles" was published in Nature and featured on the cover of March 23, 2023 Issue. This research was led by Professor Henry Liu, Director of Center for Connected and Automated Transportation (CCAT), and Director of Mcity.
$5.1 million NSF award will enable next-generation AV testing at Mcity.
News from the University Record
News from University of Michigan News
News from Auto Forbes for Mcity 2.0 opening event on Oct 24th
Prof. Liu presented the latest research on automated vehicles to the Munro Live Team at Mcity
A research paper "Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment" published in the scientific journal, Nature Communications received the SIG ITS Outstanding Paper Award at the INFORMS Transportation Science and Logistics Society Meeting.
A webinar of Scenario Generation Toolbox for Accelerating AV Development and Validation at the American Center for Mobility (ACM) was hosted by ACM.
Learn about the SAFE Test, implemented at the American Center for Mobility (ACM)
A research paper from CCAT Director, Prof. Henry Liu, has been published and chosen as a featured article in the scientific journal, Nature Communications. It proposes a naturalistic and adversarial driving environment (NADE) that will drastically improve the efficiency of autonomous vehicle testing and evaluation.
News from Center for Connected and Automated Transportation
The paper received the SIG ITS Outstanding Paper Award at the INFORMS Transportation Science and Logistics Society Meeting
Professor Henry Liu gave a keynote lecture presentation titled "Testing Autonomous Vehicles in a Naturalistic and Adversarial Environment" at the Purdue Next-Generation Transportation System (NGTS) 2020 conference.
Another news of our Augments Reality Environment for Mcity was posted in engineering.com: Augmented Reality Is Used to Improve Autonomous Vehicle Testing
Our Paper titled "Self-Driving Cars Learn About Road Hazards Through Augmented Reality" written by Yiheng Feng and Henry X. Liu was published on IEEE SPECTRUM.
A white Paper that Prof. Henry Liu and Dr. Yiheng Feng outlined the unique augmented reality technology used to test connected and automated vehicles at Mcity was published in today's University Record.