Kostal and Infineon equip the car with 6th sense

This system – a team effort from Kostal and Infineon – detects whether the driver is showing signs of drowsiness (nodding off) or is distracted. And the car instantly responds – with a vibrating seat or a warning tone, for instance. The less attentive the driver gets, the more attentive the car gets. In order to respond quickly and accurately, the assistance and the emergency braking systems can automatically activate in advance of a potential emergency. “For semi-autonomous or autonomous driving, the assistance system of the vehicle needs to know the state of the driver at all times,” says Frank Blaesing, Head of Innovation Management and Advanced Engineering at Kostal. “It needs to know if and how well the driver is informed about what’s happening on the road at that moment. This information is especially needed when the control over the vehicle needs to be turned back to the driver in a complex traffic situation.” “We want to make driving safer for all road users,” says Jochen Hanebeck, who heads the Automotive division at Infineon Technologies. “Our chips help to reduce the number of serious road accidents. For us, the collaboration with Kostal is a significant contribution to assisted driving, with all its safety and comfort advantages.” 3D camera keeps watch over the eyelids By 2018, cars with the 3D camera system from Kostal will be running off the assembly line. The optical system is only 49 by 29 millimeters (about 2 by 1 inches) in size and is embedded into the dashboard: Through the steering wheel, it “looks” front-facing at the driver’s body and head. It records the exact head position and recognizes the blink of an eye even through glasses or sunglasses. The 3D camera uses infrared light and contains the Real3 3D image sensor chip from Infineon, which records more than 100'000 pixels 50 times a second. For each pixel it simultaneously recognizes the distance and the brightness value. From the depth image of the distance data and the amplitude image of the brightness values, the 3D camera system then identifies 49 predetermined points in the driver's face such as eyes, eyebrows, mouth or tip of the nose. Algorithms use this data to calculate the driver’s attentiveness. If, for example, the system detects signs of fatigue in the driver, it initiates the countermeasures. In addition, knowledge of the exact position of the driver’s head helps to project navigation information in the windshield through head-up displays (HUDs), so that it’s right in the driver’s line of sight, seamlessly fitting into the streetscape in front of him.