Imagine setting off into space aboard NASA’s Voyager 2 probe in 1977. How far do you think you would have travelled by 2020?
Over 11 billion miles.
That’s also the same distance autonomous vehicles need to cover to ensure safety. But the miles in these vehicles are reset with every software or hardware change. And the testing requirements become more and more stringent as the level of automation increases.
So, what are the different levels of automation for self-driving vehicles?
These vehicles operate on six levels of automation, starting from level 0 to level 5 (see the figure below). The cars currently on the road today fall between levels 2 (partial automation) and 3 (conditional automation).
The key difference between the higher levels of automation (4 and 5) and the lower ones (2 and 3) is that the former allows vehicles to handle increasingly difficult tasks with minimal to zero intervention from the human occupant. To achieve this, automobiles need to transition from advanced driver assistance systems (ADAS), where the system merely functions as a driving aid, to autonomous driving (AD), where the system takes on the full responsibility of controlling vehicular motion.
Besides, the state-of-the-art nature of AD demands considerably more complex functional testing as compared to ADAS.
During testing, original equipment manufacturers (OEMs) need to ensure the functional safety of the system. With ADAS, the responsibility of operating the vehicle safely falls upon the driver and not the system. But how do we test for safety if the system is entirely responsible for the driving functions?
The key to autonomous vehicle validation
Simulation-based validation is the only truly viable option for AD testing since it offers a significant degree of flexibility, scalability, and affordability.
The challenge with simulation-based validation is knowing which test cases are relevant and how they should be tested – via simulation, proving grounds, or on public roads. Specifically, the key to effective simulation-based AD testing involves the following:
● Generating the right test cases and ensuring maximum coverage of any given scenario
● Integrating life-like human behavior into a realistic environment that truly tests the ability and the adaptability of the software across multiple geographies with varying road rules and behaviors
● Being able to scale up the operation in order to execute many test cases in parallel - quite possibly on the cloud
The benefits of simulation-based validation
Simulation should be supplemented with visits to a proving ground facility as well as on-road, public testing. While simulations can reduce the amount of physical testing required, they will not completely replace it. However, with increased simulation testing, you will be able to reassure the public that sufficient testing has been completed prior to deploying the vehicles on open roads. This will increase consumer confidence and change their perceptions of the technology, which currently, does not inspire much trust.
Simulation-based validation also allows OEMs to scale their testing efforts without increasing spending. Running simulations in parallel, on the cloud, is a real possibility thanks to the emergence of numerous cloud-based simulation platforms and photo-realistic image processing.
A huge benefit of simulation-based validation is the opportunity to test unique edge cases. For example, testing for hazardous and dangerous scenarios, which may be too unsafe to execute on proving grounds, is possible via simulation platforms.
Current limitations of simulation-based validation
While simulation tools and cloud technology allow us to scale up and quickly test many scenarios, the real need is to generate tests for every possible outcome. Specifically, OEMs need to rigorously test scenarios where the software is likely to fail. These are known as ‘edge cases’.
As it stands, scenario and test case generation (edge case identification) can be time-consuming and are often limited to the creativity and ability of the engineer tasked with the job. In order to effectively identify edge cases and ensure maximum coverage of testing, testers will have to deploy artificial intelligence.
On another front, simulating pedestrians, cyclists, and other road users that showcase realistic human behavior is a challenge. Pedestrian and driving behavior, patterns, and styles can be drastically different within the same country, state, and even time of day! Nevertheless, accurately replicating these factors in a simulation through AI models is key to achieving realistic testing conditions.
THE FUTURE IS NOW
Considering that the self-driving industry is relatively young, no best practices have been established for simulation-based validation. This opens a very interesting space for companies, both old and new, to collaborate, innovate, and take the lead in regulating a potentially $7-trillion industry.
It cannot be emphasized enough that employing a simulation-based validation solution is paramount to successfully deploying an autonomous vehicle. Being able to identify edge cases, measure safety parameters, and scale the operation will be an integral part of any AD testing solution.