Perception Deception: Physical Adversarial Attack Challenges and Tactics for DNN-Based Object Detection

The presentation discusses the importance of car safety in autonomous driving systems and the potential vulnerabilities in object detection methods. The speaker also introduces their team's efforts in creating a secure distributed computing framework and various open source projects to address these issues.

• Memory safe, trustworthy, and privacy-preserving technology called Meza te has been introduced in the distributed computing framework to address vulnerabilities in object detection methods
• Open source projects such as Wrost SGX, Master Lock, Mesolink, Mesopi, and TV Box have been developed to improve car safety in autonomous driving systems
• The speaker emphasizes the importance of car safety and the potential consequences of not addressing vulnerabilities in autonomous driving systems
• The presentation also discusses the limitations of defense mechanisms such as adversarial training

The presentation highlights the Toyota unintended acceleration case in 2010, which involved the deaths of several people and cost a significant amount of money and resources to investigate. The speaker emphasizes the importance of addressing potential vulnerabilities in autonomous driving systems to prevent similar incidents from occurring in the future.

DNN has been successful for Object Detection, which is critical to the perceptions of Autonomous Driving, and it also has been found vulnerable to adversarial examples. There has been an ongoing debate whether the perturbations to the sensor input, such as video streaming data from the camera, is practically achievable. Instead of tampering with the input streaming data, we added perturbations to the target object which is more practical. Our goal of this talk is to shed a light to the challenges of the physical adversarial attack against computer vision-based object detection system, and the tactics we applied to achieve success. At the same time, we'd like to raise the security concerns of AI-powered perception system, and urge the research efforts to harden the DNN models.The presentation starts with an overview of YOLOv3 to introduce the fundamentals of the state-of-the-art object detection method, which takes in the camera input and produces accurate detections. It is followed by the threat models we design to achieve the physical attack by applying carefully crafted perturbations to the actual physical objects. We further reveal our attack algorithms and attack strategies respectively. Throughout the presentation, we will show examples about our initial digital attack, and how we adapt it to a physical attack given the environmental constraints, for example, an object is seen at various distances and various angles etc., Finally, we wrap up the presentation with a demo to make the audience aware that with a careful setup, computer vision-based object detection can be deceived. A robust, adversarial example resistant model is required in safety critical system like autonomous driving system.