Container Patching: Making It Less Gross Than the Seattle Gum Wall

The presentation discusses strategies for container security in DevOps, including standardizing on registries, minimizing containers, using scanners for inventory and visibility, and monitoring and measuring progress.

• Standardize on registries to simplify container management
• Minimize containers to reduce vulnerabilities
• Use scanners for inventory and visibility
• Monitor and measure progress to identify bottlenecks and prioritize investment
• Create bugs for relevant images and prioritize based on severity
• Track container parent-child relationships and ownership
• Use existing systems for escalation and dashboarding
• Build a master dashboard for all-up visibility and track images on multiple axes
• Track SLOs over time and use existing systems for tracking
• Automate patching of base images consistently

The speaker emphasizes the importance of tracking SLOs over time and using existing systems for tracking. By doing so, bottlenecks can be identified and investment can be prioritized. Additionally, automating patching of base images can simplify container management and reduce vulnerabilities.

A goal like “Production containers are patched within FedRAMP timelines” is a seemingly impossible task for many organizations. What containers do we have? Who owns them, and how can we get them patched that fast? We’ll talk about our patching strategy of “Prevent, Detect, Fix, Monitor”, discuss the opensource tools available to help in each of those steps, and share lessons learned from our customers and our own patching program. Prevention narrows the funnel: standardized images, slimming images, separating build deps, allowlisting registries, and container promotion policies all help. On detection we’ll cover discovery, recent vuln detection advances, and opportunities to reduce noise. Fixing is about automating ownership discovery, fix sequencing, and release process. Monitoring glues it all together: prioritize fixes and investigate gaps to meet your SLO.