Researchers have found a simple way to trick self-driving cars into misidentifying street signs, a vulnerability that could put drivers in harms way.
The group’s research paper, entitled Robust Physical-World Attacks on Machine Learning Models, outlines how simple alterations to both a “stop sign” and “right turn sign” can fool a vehicle’s onboard computer.
“We physically realized and evaluated two attacks, one that causes a Stop sign to be misclassified as a Speed Limit sign in 100% of the testing conditions, and one that causes a Right Turn sign to be misclassified as either a Stop or Added Lane sign in 100% of the testing conditions,” the paper says.
In the first experiment, a high resolution image was placed over a stop sign in order to carry out a “poster-printing attack.”
“In contrast to some findings in prior work, this attack is very effective in the physical world,” the paper says. “The Stop sign is misclassified into our target class of Speed Limit 45 in 100% of the images taken according to our evaluation methodology.”
An attack used against a right turn sign also convinced the vehicle’s machine-learning systems that the sign was instead a stop sign or added lane sign.
“Our attack reports a 100% success rate for misclassification with 66.67% of the images classified as a Stop sign and 33.7% of the images classified as an Added Lane sign,” the paper adds.
In what is described as a “sticker attack,” the researchers were able to obtain similar results by adding words and camouflaged stickers to a stop sign.
After generating the words “love” and “hate,” for example, the vehicle falsely interpreted the stop sign as a speed limit sign.
“Sticker camouflage graffiti attacks caused the Stop sign to be misclassified as a Speed Limit 45 sign 66.67% of the time and sticker camouflage art attacks resulted in a 100% targeted misclassification rate.” the paper states.
As noted by Bleeping Computer’s Catalin Cimpanu, countermeasures include everything from cities keeping signs clean to vehicle companies altering their algorithms.
“As self-driving car technologies will become more prevalent, keeping street signs clear of any visual clutter will become a mandatory task of any smart city administration across the globe,” Cimpanu writes. “Researchers say that authorities can fight such potential threats to self-driving car passengers by using an anti-stick material for street signs. In addition, car vendors should also take into account contextual information for their machine learning systems.”
Contact Mikael securely: keybase.io/mikaelthalen