1 / 34

Evolution of Facebook's Vulnerability Management Program

Learn about the evolution of Facebook's Vulnerability Management program, implementation design principles, industry standard technology, architecture mindset, and lessons learned. Discover the journey towards building a highly scalable system to manage vulnerabilities at scale.

bbradberry
Download Presentation

Evolution of Facebook's Vulnerability Management Program

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Vulnerability Management at Scale • Alexandre Fiori • Production Engineer

  2. Overview The evolution of the Vulnerability Management program at Facebook, the birth of the Production Engineering team to support the program, and the pragmatic approach to build a highly scalable system for an ever-growing environment. What’s this about

  3. “Facebook was built on an open source stack. We support and encourage the use and development of open source software and hardware”

  4. Vulnerability Management Timeline • …2015: PCI DSS • 2016: Internal scanning tools • 2017: Broader network scanning • 2018: Reboot, build up • 2019: VMaaS

  5. 2018

  6. New challenge • Scan all infrastructure • Merge scanning technologies • Improve vulnerability matching • Start and track remediation • Manage remediation lifecycle

  7. Architecture

  8. Mindset • Big data • Scalability and reliability • On-line vs off-line pipelines • Concept validation and XFN • Fast prototype and launch

  9. ETL

  10. Collectors • Scan asset inventories and print to standard output

  11. Processors • Pre-process, scan vulnerabilities, post-process

  12. Reporters • Reporters manage escalation lifecycle

  13. Inventory Classes • Hosted software • Installed software • Running software • Network scanners • Hardware

  14. Vulnerability Database • Public vs Proprietary • Multi-vendor system • General purpose datasets • Specialized for ecosystem • Standard format for product->vulnerability matching

  15. Implementation

  16. Design Principles • Command line tools do one thing • Communicate over a text interface • Core functionality shared as libraries • Composable code, tools, and pipelines • Rely on well established UNIX conventions

  17. Industry Standard Technology • MITRE / NIST / NVD • Common Platform Enumeration • Common Vulnerabilities and Exposures • Common Weakness Enumeration • Common Vulnerability Scoring System

  18. Infrastructure • Tools • Services • Data warehouse • Dashboards • Notifications

  19. Internal Pipelines • Tupperware • Container images and packages • First-party vs third-party codebase • Bad Binary Hunter and Buck • Attribution from package to service

  20. Vulnerability Database Tools • nvdsync and $vendor2nvd • vulndb command line tool • Uses NVD CVE JSON 1.0 format • Manages versioned datasets backed by MySQL • Supports vendor snapshots, custom CVEs, and snoozes • vulndb thrift service • CVE lookup and CPE matching

  21. Decoration • CWE and CVSS • Domain and sub-domain • First seen • Backlog vs influx • Owner (on-call ID or UNIX username) • Threat Intelligence

  22. Remediation • Starts from CVE inventory • Depends on decoration data • Supports feedback loop (e.g. snoozes) • Understands release cycles per inventory class • Manages escalation and lifecycle

  23. Lessons Learned • Normalization, aggregation, and blackholes • Per-customer decision trees are burdensome • Handling delays, XFN work, and fine tuning • Tasks and notification updates are annoying • False positives can compromise credibility

  24. 2019: VMaaS

  25. Goals • Self-service system for vulnerability scan • Custom aggregation defined by collectors • Configurable providers and thresholds • Tier-based service, starting from bronze • Default dashboards and reports per tier • Common CVE inventory for all customers

  26. Progress • XFN partnership with early adopters • Migrated “hosted software” inventory class • Total of 10+ collectors in operation • Customers fixing bogus CVEs in the database • Snoozes effectively helping fine-tune reports

  27. Next up • Migrate other inventory classes • Improve data quality and detection speed • Tackle backlog via remediation campaigns • Tackle influx via push-blocking scans • Influence company culture outside security org

  28. Thank you

  29. Q&A

More Related