How to Design, Launch, and Monitor Resilient ML Systems

Abnormal Security builds ML products that help protect systems against cyber attacks. Dan Schiebler, Head of Machine Learning at Abnormal Security, discusses best practices for building cybercrime detection algorithms. In this session, Schiebler covesr how to design, monitor, and launch resilient ML systems and how to train ML models on production issues. He talks about the different types of problems that production ML systems can encounter, including features that become unavailable because of upstream data issues, distribution changes, or features that become stale. Schiebler addresses the different types of iteration loops in most companies—online vs offline—and how that plays into testing and training, as well as the company’s ablity to tolerate risk. Historical logs and data also play a key role. Before joining Abnormal, Schiebler worked at Twitter: first as an ML Researcher working on recommendation systems, and then as the Head of Web Ads Machine Learning. Before Twitter, he built smartphone sensor algorithms at TrueMotion.