Smile, you’re on candid camera when you’re in a modern car with sophisticated driver-assist systems. And in the near future, those cameras will be joined by lidar VCSEL and radar sensors, as well. Yes, the same gadgets that vehicles must use to perceive the environment sufficiently to drive themselves will soon be keeping tabs on you and your passengers inside, too. Ah, progress …  

You may have seen those unflattering black-and-white images showing how a computer “sees” you when it’s trying to figure out what you’re looking at. Most of these are shot with infrared cameras, but Switzerland-based Eyeware has teamed up with Belgium-based semiconductor supplier Melexis to develop the GazeSense system. It uses a 3D camera that employs lidar VCSEL range sensing to more accurately identify an occupant’s face, eyes, and focus of attention. It performs well in a wider range of lighting conditions than the purely infrared cameras primarily used today. The low-resolution image isn’t any more flattering (nobody’s looking at it anyway), but it’s highly accurate, even at an angle. This opens greater possibilities for positioning of the camera (on the dash, rear-view mirror, A-pillar, etc. ). We expect the iPhone 12 Pro’s incorporation of a lidar VCSEL sensor to rapidly bring down the cost and hasten the adoption of such devices. 

Menlo Park-based SRI aims to take driver monitoring to the next level, by endeavoring to look beyond gaze to infer the driver’s holistic state of wellbeing. The system, which has been developed in Toyota’s LQ concept car (first introduced in 2019), incorporates a suite of infrared and 3D cameras to track driver’s eye movements, facial expressions, and skeletal positioning from which it gauges body language. It then strives to identify drowsiness or even potentially dangerous emotional states, like anxiety or boredom, which could affect driving. To counteract these states, the vehicle might blast the air conditioning to help a driver stay alert or maybe suggest an alternate less traffic-choked or boring route if it perceives irritability or boredom. The more miles each driver spends in the car, the more accurately the artificial intelligence recognizes and responds to the driver’s needs. Production (you may be relieved to learn) is not imminent. 

While some are content to simply monitor gaze or emotional state, the Delaware-based radar specialists at Xandar Kardian seek to infer the cabin occupants’ general health by keeping tabs on resting heart rates, respiratory rates, and the general occupancy of the entire vehicle. And the company claims it can do all this with a single sensor. Well, that sensor doesn’t monitor gaze, but by using impulse-radio ultra-wideband (IR-UWB) radar technology it can monitor the driver’s general health, notice drops in resting heart rate or respiration that could indicate drowsiness—information that could be used to turn up the air conditioning or music to rouse the driver. Then, at the end of a journey, it can definitively detect the presence of a living being in the back seat, triggering a mobile-phone alert for the driver. Xandar Kardian claims that its single sensor is more cost-effective than the seven- or eight-seat sensors typically required to serve a cabin monitoring system.