Follow the White Rabbit: Simplifying Fuzz Testing Using FuzzExMachina

The presentation discusses the effectiveness of feedback-guided fuzzing in finding bugs and vulnerabilities in software programs.

• Fuzz testing involves throwing unpredictable input at a program until it breaks.
• Input specification-guided fuzz testing requires a specification of the program, which may not always be available.
• Feedback-guided fuzzing uses coverage-guided testing to automatically learn about program behavior and has been very effective in finding bugs and vulnerabilities.
• Initiatives like Google's OSS fuzz have exposed thousands of open-source bugs and uncovered bugs in the Linux kernel.
• Continuous fuzzing could have prevented the Heartbleed bug in OpenSSL.

In the past, researchers proposed input specification-guided fuzz testing, which required a specification of the program. However, this had limitations, such as difficulty in obtaining a specification and not testing undocumented features. Feedback-guided fuzzing, which uses coverage-guided testing to automatically learn about program behavior, has been very effective in finding bugs and vulnerabilities. For example, Google's OSS fuzz has exposed thousands of open-source bugs and uncovered bugs in the Linux kernel. Additionally, continuous fuzzing could have prevented the Heartbleed bug in OpenSSL.

Setting up a fuzzing pipeline takes time and manual effort for identifying fuzzable programs and configuring the fuzzer.Usually only large software projects with dedicated testing teams at their disposal are equipped to use fuzz testing in their Security Development Lifecycle. Other projects with limited resources cannot easily use this effective technique in their SDL. This renders the software landscape unnecessarily insecure. Especially less popular software applications are not being fuzzed due to a lack of resources and easy to use tooling.Lowering the required skill level and effort to set up a fuzzing pipeline therefore results in a significant increase of today's software's security. To tackle this challenge, we developed an easy to use framework, FuzzExMachina (FExM), that reduces manual effort to a minimum. Using clever input inference methods and containerization, we automate the fuzzing pipeline from start to end in a scalable fashion. We support acquiring binaries from a variety of sources, including blackbox binaries and source code repositories.In cases for which FExM cannot automatically achieve a high coverage, it drops users to a novel AFL mode, "Afl-TimeWarp", in which they can set up testcases without the need to alter or understand the underlying code. AFL-TimeWarp mode allows to fuzz deeper program states without writing a single line of code, fitting FExM's philosophy to keep it simple for users.To test the viability of our framework, we fuzzed over one hundred packages from the Arch Linux package repository with essentially zero effort. After only a few days, we already found 11 crashes, six of which were exploitable. This shows how FExM permits automated distributed fuzzing of applications; crash exploitability classification; and is equipped with a web front end for navigating security issues in a convenient way. Our work automatically retrofits fuzzing into the security development lifecycle.