Using a Driving Test as a Safety Metric (Metrics Episode 4)
The part of the driver test that is problematic for self-driving cars is deciding that the system has maturity of judgement. How do you check that it is old enough to drive?
At some point, companies are done testing and they need to make a decision about whether it’s okay to let their vehicles actually be completely self-driving, driverless cars. The important change here is that there is no longer a human responsible for continuously monitoring operational safety. That changes the complexion of safety, because now there’s no human driver to rely upon to take care of anything weird that might go wrong. That means you’ll need different metrics for safety when you deploy compared to those used during road testing.
One way people talk about knowing that it’s time to deploy is to give the car a road test, just like a human gets a driver’s test. Everyone wants something quick, cheap, and painless to decide whether their self-driving car is ready to go, and a driver test has some intuitive appeal. Indeed, if a car can’t pass a driver test, then truly that’s a problem. But is passing such a test enough?
Let’s go down the list of things that happen in a driver test.
There’s a written test. Well, surely a self-driving car needs to know the rules of the road, just like a person. But for a self-driving car it’s a little more complicated, because it’s not just the rules of the road, but also what to do about conflicting rules or incomplete rules, or how you handle justifiable rule breaking.
For example, if there’s a big truck broken down in the travel lane on a two lane road, do you wait there until the truck is towed away, several hours perhaps? Or do you go around it if there’s no traffic? Well, going over the double yellow line is clearly breaking the usual rules, but in everyday driving people do that sort of thing all the time.
Another thing you need is a vision test. Surely self-driving cars need to be able to see things on the road. For a person it’s typically just whether or not they have the right glasses on. But for a self-driving car it’s more complicated, because it isn’t just seeing that something’s in the road, but also figuring out what’s on the road and what might happen next. It isn’t just about reading road signs.
Another thing, the classic thing people have in mind is a road skills test. Surely a self driving car needs to be able to maneuver the vehicle in traffic and account for all the things that happen. But again, it’s more complicated to know that a self-driving car is ready beyond what you see on a typical road test. Sure, a typical road test covers things like parallel parking and using turn signals. But that’s the easy stuff. Did your driver test cover spin-outs? Did it cover handling a blown out tire with an actual blown out tire? Did you have to deal with loss of brakes? Did you have another car run a red light during the driver test to see how you’d respond? (OK, well that last one actually did happen to me by chance on my own driver test. But I digress.)
Even if you were to address all those types of things, there’s another important piece of a human driver test that people don’t think of as a test.
That’s actually proving you’re a human. You do that by showing your birth certificate. Oh, look: I’m a 16 year old human, and while I may not be the most mature person in the world, society’s determine I’m good enough to be able to handle a driver’s license. Now, what comes with that? It isn’t just that you’re 16. It’s that being 16, or whatever the age is where you are, is a proxy for things like being able to handle ambiguity and reason about consequences well enough. It’s a proxy for knowing that a situation is unpredictable and becoming more cautious, because you’re not sure what happens next. It's a proxy for understanding consequences and personal responsibility. Humans are moderately good at knowing when they’re not sure what’s going on, but things like machine learning are notoriously bad at knowing that they’re out of their element.
It’s also a proxy for handling unexpected events. While humans aren’t perfect, they do remarkably well when completely unexpected, unstructured events happen. They usually figure out something to do that’s reasonable.
In general, being a 16 year old human is a proxy for some level of maturity and experience at handling the unexpected. What we do is we use a driver test to say: okay, this person has basic skills, and because they’re a reasonably mature human, they can probably handle all the weird stuff that happens and they’ll figure it out as they go.
A big problem is it’s unclear how you figure out that a self-driving car has the level of judgment maturity of at least a 16 year old human. We’re not sure how to do that.
What we have known for decades is that you can’t prove a software based system is safe via testing alone. There are just too many complex situations and too many edge cases to be handled. No set of tests can cover everything. It’s just not possible. So, a driver test alone is never going to be enough to prove that a self-driving car is safe. Sure, elements of a driving test are useful. You absolutely want the self-driving car to know the rules of the road, to be able to look down the right away and see what’s there, and to be able to maneuver in traffic. But that is a minimum and insufficient set of things to prove it’s safe enough to responsibly deploy.
The point is you need more than a test. You need good engineering rigor. You need to know the car was designed properly, just like in every other safety critical computer based system. You don’t fly airplanes around to see if they fall out of this sky, and if they don’t, you say it’s safe. In fact, you do a lot of good engineering and you make sure the system is designed to do the right thing, and the testing is just there to make sure that the engineering process did what you thought it did.
So, for self-driving cars, sure, a road test is helpful, but you’re going to need a good engineering process was executed as well. You need to know that the system is going to handle all the things that have to go right, as well as all the things that can possibly go wrong, and that goes far beyond any reasonable driver test.
For the podcast version of this posting, see: https://archive.org/details/metrics-05-driver-test-metrics
Thanks to podcast producer Jackie Erickson.
Categories:
driver test,
metrics
0 comments:
Post a Comment