Fixing a Memory Forensics Blind Spot: Linux Kernel Tracing

Conference:  BlackHat USA 2021



The presentation discusses the use of eBPF for offensive purposes and how it can be detected using volatility plugins. The speaker walks through various subsystems and shows how they can be abused by malware.
  • eBPF can be used for offensive purposes and can be detected using volatility plugins
  • Various subsystems such as f trace and tracepoint can be abused by malware
  • Proof of concept is shown for each subsystem and a plugin is used to detect the behavior
  • The technical paper accompanying the talk provides more details on the kernel internals
The speaker talks about how they filtered the kernels to only those that were at least 4.9 in their set and wrote a script to query the API to figure out which kernels had all the options present. They found that essentially any kernel that you're going to encounter during your investigations will have these features present which means they can be abused by malware.


The ubiquity of Linux servers across the internet and within cloud instances necessitates that defensive research maintains pace with the introduction of new features to the platform. Unfortunately, these research efforts have not adequately kept pace with advances in Linux kernel development, leaving blind spots for attackers to remain undetected. In this presentation, we document our effort to close a significant blind spot - the Linux kernel's tracing infrastructure. This infrastructure is installed and enabled by default on essentially all Linux distributions and is heavily utilized across a significant number of cloud-centric organizations, such as Facebook, Netflix, Google, GitLab, and Adobe.The provided tracing features have legitimate uses for system monitoring, but also allow for code in userland and the kernel to observe and modify key portions of the operating system. This includes the ability to hook kernel subsystems, such as the networking stack, system call handling facilities, and file system drivers as well as all exported APIs. Current memory forensics techniques provide no means to effectively analyze these tracing features, leaving a significant number of malware capabilities to potentially go undetected. To close this gap, we developed new memory forensic techniques that can analyze the various tracing subsystems and report on potential abuses. These new analysis techniques are embodied in Volatility plugins, as Volatility is the most commonly used analysis framework in the field. To provide capabilities that are useful both now and well into the future, we developed each technique as a plugin for both Volatility 2 and Volatility 3. Our team plans to contribute all the new plugins to the public Volatility repositories upon publication of this paper. This will allow the techniques to be immediately usable in the field as well as provide reference code for future researchers.