Deep neural networks routinely achieve near human-level performances on a variety of tasks, but each new breakthrough demands massive volumes of quality data, access to expensive GPU clusters, and weeks or even months to train from scratch. AI researchers commonly release model checkpoints to avoid the wasteful duplication of these costly training runs, since fine-tuning pre-trained neural networks for custom tasks requires less data, time, and money compared to training them from scratch. While this emerging model sharing ecosystem beneficially lowers the barrier to entry for non-experts, it also gives a leg up to those seeking to leverage open source models for malicious purposes.
Using open source pre-trained natural language processing, computer vision, and speech recognition neural networks, we demonstrate the relative ease with which fine tuning in the text, image, and audio domains can be adopted for generative impersonation. We quantify the effort involved in generating credible synthetic media, along with the challenges that time- and resource-limited investigators face in detecting generations produced by fine-tuned models. We wargame out these capabilities in the context of social media-driven information operations, and assess the challenges underlying detection, attribution, and response in scenarios where actors can anonymously generate and distribute credible fake content. Our resulting analysis suggests meaningful paths forward for a future where synthetically generated media increasingly looks, speaks, and writes like us.