Catching when people text and drive

Departments - Rearview

With their new software, researchers at the University of Waterloo improve road safety by warning distracted drivers.


Computer algorithms developed by engineering researchers at the University of Waterloo in Canada accurately determine when drivers are texting or engaging in other distracting activities.

Cameras and artificial intelligence (AI) detect hand movements that deviate from normal driving behavior, and grades or classifies them in terms of possible safety threats.

Fakhri Karray, an electrical and computer engineering professor at Waterloo, says the information could be used to improve road safety by warning or alerting drivers when they are dangerously distracted. As advanced self-driving features are increasingly added to conventional cars, signs of serious driver distraction could be employed to trigger protective measures.

“The car could actually take over driving if there was imminent danger, even for a short while, to avoid crashes,” says Karray, a university research chair and director of the Center for Pattern Analysis and Machine Intelligence (CPAMI) at Waterloo.

Algorithms at the heart of the technology were trained using machine-learning techniques to recognize actions such as texting, talking on a cellphone, or reaching into the backseat to retrieve something. The seriousness of the action is assessed based on duration and other factors.

The work builds on extensive previous research at CPAMI on the recognition of signs, including frequent blinking, when drivers are in danger of falling asleep at the wheel. Head and face positioning are also important cues of distraction.

Ongoing research at the center now seeks to combine the detection, processing, and grading of several different kinds of driver distraction in a single system.

“It has a huge impact on society,” Karray says, citing estimates that distracted drivers are to blame for up to 75% of all traffic accidents worldwide.

Another research project at CPAMI is exploring the use of sensors to measure physiological signals such as eye-blinking rate, pupil size, and heart-rate variability to help determine if a driver is paying adequate attention to the road.

Karray’s research – done in collaboration with Ph.D. candidates Arief Koesdwiady and Chaojie Ou, and post-doctoral fellow Safaa Bedawi – was recently presented at the 14th International Conference on Image Analysis and Recognition in Montreal, Canada.

University of Waterloo