Looking for the perfect signature: an automatic YARA rules generation algorithm in the AI-era

Automatic generation of ER rules for Android using AI algorithms outperforms human-generated rules

• Developed algorithms to automatically generate ER rules for Android using AI
• Ultimate regenerator rules performed better than human-generated ones
• Expert knowledge can be included in rule generation through attributes, attribute values, and optimization results
• Approach is scalable and generates rules in minutes
• Automatic generated signatures improved detection by up to 131% without generating false positives

The speaker demonstrated the use of a tool called Yellow Jumpy to generate a rule for the Sky Go Free family of Android applications. The tool downloaded reports of application analysis from the internet and generated a valid ER rule with a high score and coverage of all input samples. The speaker also tested the efficacy of automatically generated signatures on 1.5 million Android applications collected in 2017, showing significant improvement in detection without generating false positives.

Given the high pace at which new malware variants are generated, antivirus programs struggle to keep their signatures up-to-date, and AV scanners suffer from a considerable quantity of false negatives. The generation of effective signatures against new malware variants, while avoiding false positive detections, is a highly desirable but challenging task, typically requiring a substantial portion of human expert’s time. Artificial intelligence techniques can be applied to solve the malware signature generation problem. The ultimate goal is to develop an algorithm able to automatically create a generalized family signature, eventually reducing threat exposure and increasing the quality of the detection. The proposed technique automatically generates an optimal signature to identify a malware family with very high precision and good recall using heuristics, evolutionary and linear programming algorithms. In this talk I will present YaYaGen (Yet Another YARA Rule Generator), a tool to automatically generate Android malware signatures. Performances have been evaluated on a massive dataset of millions of applications available in the Koodous project, showing that in a few minutes the algorithm can generate precise ruleset able to catch 0-day malware, better than human generated ones.