What Perception Does in a Self-Driving Car

Walk into a crowded store looking for a friend. Your eyes receive an overwhelming amount of visual information — people, racks, signage, reflections, movement. You do not process all of it. You discard almost everything and lock onto a handful of cues: height, hair, the jacket they said they would wear.

That filtering is what perception does. A camera suite produces an enormous stream of pixels every second. Almost none of it matters. The system has to convert that flood into a compact description: where the drivable surface is, where lane markings run, what objects exist, how large they are, how fast they move, and which ones could interact with the vehicle.

Your eyes receive an overwhelming amount of visual information — people, racks, signage, reflections, movement.

Two things make this harder than it sounds. First, the world does not arrive labeled. A plastic bag and a rock can look similar at distance and demand opposite responses. Second, perception has to stay useful in conditions it was not optimized for — low sun, wet asphalt, a truck's headlights, a repainted lane sitting on top of a faded one.

The useful mental model is not a camera recording a scene. It is an aggressive summarizer that throws away most of what it receives and has to be correct about what it kept. Everything downstream inherits those choices. If perception drops something, later stages cannot invent it back.