The personification of the black box

Dr Sam Anthony, CTO and Co-Founder, Perceptive Automata

Autonomous vehicle progress is stalling because AI sees pedestrians as black boxes. But there is a way to solve it

When you use traditional AI techniques in a self-driving car, you end up with a vehicle that sees humans as black boxes. These boxes move around, and sometimes you can attach labels to them – this one is tall, this one is small, this one is holding its arm up – but you don’t understand them.

Navigating around black boxes is hard. They move in unexpected directions, and if you don’t want to hit them you have to be incredibly cautious, assuming they could move in any direction at any time. In fact, there are many situations where it’s simply impossible to figure out how you can get past a black box if you can’t hit it.

It would be better if you could understand them not as black boxes, but as people – people who have ideas and goals and are trying to figure out how to interact with you.

A big problem for autonomous driving systems

When traditional AI models are trained, what they’re learning is a mapping of a label to an image. You take thousands of images and generate numbers for them. How those numbers get generated, or what those numbers mean, isn’t part of the process, but the AI learns what images tend to go with what numbers. What the AI is capable of doing is predicting what the black box attached to that image should say and do. It’s trying to figure out which image is represented by which black box. That’s it.

When you put that system in a car and it sees a pedestrian out in the world, it matches that pedestrian with the black box that fits it best. But it doesn’t really know anything about that pedestrian. It doesn’t have any ability to reason about what’s in that person’s head. Black boxes don’t want to cross the street – black boxes don’t want anything. Black boxes don’t know you’re there. They have no inner life.

Solving the problem with research

At Perceptive Automata, we remove the black box. When we train AI, we do something different. We still take thousands of images, but instead of opaquely, mindlessly applying labels to them, we integrate the personhood of the labellers deeply into the training process. As each of those thousands of images is shown to the AI, what it’s learning is not a set of disconnected numbers. It’s learning what people think about that image. In particular, it’s learning how people would answer questions about what’s in the head of a pedestrian pictured before them.

This isn’t easy to do. To ask people questions about what’s in the minds of pedestrians and get answers that are usable for training AI requires scientific rigour and a great deal of art. You need expertise in visual psychophysics, a field of science dedicated to measuring how people see and respond to the world. You have to be a people expert and know how to understand, study and characterise them.

AI trained this way, like Perceptive Automata’s SOMAI, or State of Mind AI, no longer sees pedestrians as black boxes. When you put our AI in a car, it sees pedestrians as people do. Instead of merely identifying what set of numbers maps most accurately to a given pedestrian, SOMAI is able to answer questions about what’s in that pedestrian’s head. It does this by imagining the people who trained it and hearing their voices. In effect, it answers the question: “If there were 500 people here in the car with you, and you asked them all, what would they say about whether that pedestrian wants to cross in front of your car?”

When you understand pedestrians as people, who have goals and desires that interact with the goals of the vehicle, they stop being black boxes that might move anywhere at any time. They want specific things. Some of them standing at a crossing want to cross. Others maybe don’t. Without a real-time understanding of what’s in pedestrians’ heads, autonomous vehicles will be stuck trying to navigate around black boxes, and the promise of this industry will not pay off.

Learn more and request a demo of SOMAI at perceptiveautomata.com