Stanford
University
  • Stanford Home
  • Maps & Directions
  • Search Stanford
  • Emergency Info
  • Terms of Use
  • Privacy
  • Copyright
  • Trademarks
  • Non-Discrimination
  • Accessibility
© Stanford University.  Stanford, California 94305.
How AI Is Making Autonomous Vehicles Safer | Stanford HAI

Stay Up To Date

Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.

Sign Up For Latest News

Navigate
  • About
  • Events
  • Careers
  • Search
Participate
  • Get Involved
  • Support HAI
  • Contact Us
Skip to content
  • About

    • About
    • People
    • Get Involved with HAI
    • Support HAI
  • Research

    • Research
    • Fellowship Programs
    • Grants
    • Student Affinity Groups
    • Centers & Labs
    • Research Publications
    • Research Partners
  • Education

    • Education
    • Executive and Professional Education
    • Government and Policymakers
    • K-12
    • Stanford Students
  • Policy

    • Policy
    • Policy Publications
    • Policymaker Education
    • Student Opportunities
  • AI Index

    • AI Index
    • AI Index Report
    • Global Vibrancy Tool
    • People
  • News
  • Events
  • Industry
  • Centers & Labs
news

How AI Is Making Autonomous Vehicles Safer

Date
March 07, 2022
Topics
Machine Learning

AI is used to simulate real-world conditions to safety-test autonomous vehicles. Stanford researchers surveyed the algorithms and say they are good, but work remains.

A lot is at stake every time an autonomous car or plane strikes off on its own. Engineers place great trust in the intelligent systems that see and sense the world and help self-controlled vehicles steer clear of virtually every hazard that might come their way.

“Our main challenge as a field is how do we guarantee that these amazing capabilities of AI systems—driverless cars and pilotless planes—are safe before we deploy them in places where human lives are at stake?” says Anthony Corso, a postdoctoral scholar in aeronautics and astronautics and executive director of the Stanford Center for AI Safety.

“The systems themselves are extremely complex, but the environments we are asking them to operate in are incredibly complex, too,” Corso says. “Machine learning has enabled robotic driving in downtown San Francisco, for example, but it’s a huge computational problem that makes validation all the harder.”

Road tests are the ultimate arbiter of safety, but road tests typically come only at the very last stages of the design cycle and are freighted with the same sort of risks to human life that researchers are hoping to avoid. No engineer would want to be responsible for a road test that claimed a life, or even valuable property, in the name of proving that the technology was safe.

Read the full study, "A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems"

In light of these practical challenges, designers of autonomous vehicles have come to rely on simulations to test autonomous vehicles’ ability to steer clear of danger. But, are these simulations up to the challenge? With a paper published in the Journal of Artificial Intelligence Research, Corso and colleagues at Stanford and NASA provide an overview of these “black-box safety validation” algorithms. They find reason for optimism that simulation may one day deliver a necessary level of confidence, but work remains.

Assessing the “Black Box”

The designers of self-driving cars and planes and other computationally intensive autonomous vehicles have turned to the so-called black-box validation applications. Black-box algorithms stand in contrast to their sibling, white-box methods. White-box validation seeks what is known as “formal verification” of a system’s safety—not only finding potential points of failure but, ideally, proving the absolute absence of failure, as well.

This higher standard, however, is impossibly intense computationally speaking and does not scale well to large, complex problems like autonomous vehicles. There is just too much going on to calculate everything to deliver white-box level confidence. But, by cutting a few computational corners, black-box approaches hope to surmount these challenges.

Corso likens it to a video game played in reverse, where the testing algorithm is the player and victory is defined as failure—a crash—but in a simulated world, of course, with no risk to life or property. Only then, when you know when and why a system has failed, can you address such situations in the safety mechanisms built into the vehicle.

“The algorithms take an adversarial approach, trying to find weakness. Our hope is that we don’t find failure. The longer that black-box techniques churn away, running through possible scenarios, trying to create weaknesses and not finding them, the greater our confidence grows in the system’s overall safety,” Corso says of the philosophy that drives the field.

Triangulation

To provide the highest confidence possible, validation algorithms perform a sort of triangulation on failure. The highest tier for the most risk-averse industries like aviation, validation algorithms search for any way a system might fail, an approach known as falsification. “Falsification asks: Can you find me any example where the system fails?” Corso says.

It’s a deliberately low bar set to provide greatest assurance. For self-driving cars, however, that threshold is too low. “With an autonomous car operating in an urban environment, you can always find some pathological situation that’s going to cause a crash,” Corso says. “So we raise the bar a bit.”

The next tier, then, involves finding the failures that are most likely to occur to guide the design team as they make their systems as safe as possible. The third tier is to estimate the probability of various forms of failure to assess how likely any one outcome is over another.

“These techniques kind of build on top of each other to increase confidence in overall system safety,” Corso says.

Toward Safer Systems

The survey does not necessarily make value judgments on the black-box tools reviewed, but rather compares how each addresses the problem, what assumptions the creators have built into their system, and what the relative strengths and weaknesses are in each so that autonomous systems designers can choose the approach that best fits their needs.

However, of the nine systems tested that are currently available for use, Corso notes, only two provide anything more than falsification validation, just one offers most-likely failure testing and another offers probability estimation. So, he says, there’s room for improvement.

Overall, Corso and colleagues can’t yet put the stamp of approval on any of the options, but he can see where the field is headed. The most exciting direction, he says, is “compositional validation,” testing individual components separately—like the visual perception and proximity sensing systems—to learn about how each component fails. Knowing more about how subcomponents fail, Corso says, can be leveraged to improve confidence in the overall system safety.

“A few approaches we mentioned have started to touch on this concept,” Corso says, “But I think it will require a lot more work. In their current state, these whole-system algorithms in and of themselves are insufficient to put a formal stamp of approval on them just yet.”

Stanford HAI's mission is to advance AI research, education, policy and practice to improve the human condition. Learn more. 

Share
Link copied to clipboard!
Contributor(s)
Andrew Myers

Related News

The AI Race Has Gotten Crowded—and China Is Closing In on the US
Wired
Apr 07, 2025
Media Mention

Vanessa Parli, Stanford HAI Director of Research and AI Index Steering Committee member, notes that the 2025 AI Index reports flourishing and higher-quality academic research in AI.

Media Mention
Your browser does not support the video tag.

The AI Race Has Gotten Crowded—and China Is Closing In on the US

Wired
Regulation, Policy, GovernanceEconomy, MarketsFinance, BusinessGenerative AIIndustry, InnovationMachine LearningSciences (Social, Health, Biological, Physical)Apr 07

Vanessa Parli, Stanford HAI Director of Research and AI Index Steering Committee member, notes that the 2025 AI Index reports flourishing and higher-quality academic research in AI.

Here are 3 Big Takeaways from Stanford's AI Index Report
Tech Brew
Apr 07, 2025
Media Mention

Vanessa Parli, HAI Director of Research and AI Index Steering Committee member, speaks about the biggest takeaways from the 2025 AI Index Report.

Media Mention
Your browser does not support the video tag.

Here are 3 Big Takeaways from Stanford's AI Index Report

Tech Brew
Sciences (Social, Health, Biological, Physical)Machine LearningRegulation, Policy, GovernanceIndustry, InnovationApr 07

Vanessa Parli, HAI Director of Research and AI Index Steering Committee member, speaks about the biggest takeaways from the 2025 AI Index Report.

Stanford HAI's 2025 AI Index Reveals Record Growth in AI Capabilities, Investment, and Regulation
Yahoo Finance
Apr 07, 2025
Media Mention

"The AI Index equips policymakers, researchers, and the public with the data they need to make informed decisions — and to ensure AI is developed with human-centered values at its core," says Russell Wald, Executive Director of Stanford HAI and Steering Committee member of the AI Index.

Media Mention
Your browser does not support the video tag.

Stanford HAI's 2025 AI Index Reveals Record Growth in AI Capabilities, Investment, and Regulation

Yahoo Finance
Economy, MarketsMachine LearningRegulation, Policy, GovernanceWorkforce, LaborIndustry, InnovationSciences (Social, Health, Biological, Physical)Ethics, Equity, InclusionApr 07

"The AI Index equips policymakers, researchers, and the public with the data they need to make informed decisions — and to ensure AI is developed with human-centered values at its core," says Russell Wald, Executive Director of Stanford HAI and Steering Committee member of the AI Index.