Building and Maintaining a Successful Vulnerability and Patch Management Program

1. Building and Maintaining a Successful Vulnerability and Patch Management Program Presented to Western Regional Educause April 2, 2008

2. Naval Postgraduate SchoolEstablished in Annapolis in 1909Moved to Monterey in 1951

3. Academic Programs Academic Schools • Graduate School of Business and Public Policy • Graduate School of Engineering and Applied Sciences • Graduate School of Operations and Information Sciences • School of International Graduate Studies Research Institutes • Cebrowski Institute for Information Innovation and Superiority • Wayne E. Meyer Institute of Systems Engineering • Modeling Virtual Environments and Simulation Institute

4. NPS Population • 1700 resident students • Less than 50% are Navy • 30% international officers • Remainder Air Force, Army, US Marine Corps, US Coast Guard, and civilians • 1700 faculty and staff • 880 distance learning students

5. Accreditation Regional • Western Association of Schools and Colleges (WASC) Programmatic • ABET (some programs) • AACSB • NASPAA

6. CSU-Monterey Bay Monterey Peninsula College Monterey Institute of International Studies Hopkins Marine Station – Stanford University Monterey Bay Education Science and Technology Center, University of California National Undersea Research Program (NOAA) Moss Landing Marine Lab (CSU) University of California Sea Grant Extension National Weather Service Monterey Bay National Marine Sanctuary, NOAA Pacific Fisheries Environmental Lab, NOAA Defense Language Institute Fleet Numerical Meteorology and Oceanography Center Monterey Bay Aquarium Research Institute Naval Research Laboratory Defense Manpower and Data Center Naval Postgraduate School Monterey College of Law Chapman College Golden Gate University Research-related Organizations

8. NPS Systems • Multiple Networks – 7 and growing • Web services • Extranet: 326MB per day; 55M hits per day • Intranet: 786MB per day; 88M hits per day • 5425 systems (computer, printer, scanner) connected to NPS network • 6500 campus computer accounts • 800 software applications • 3150 active phone lines

9. Definition: Information Assurance • “Measures that protect and defend information and information systems by ensuring their confidentiality,integrity, availability, non-repudiation and authentication. This includes providing for restoration of information systems by incorporating protection, detection and reaction capabilities.” (DoDD 8500.1)

10. IA Attributes • Confidentiality – Disclosure of Information • Integrity- Unauthorized modification of data • Availability- timely, reliable access to data • Non-repudiation- proof of delivery and identity • Authentication – proof of identity

11. IA Tools • Computer Network Defense (CND) • Vulnerability Management • Alerts, Bulletins, Tech. Advisories – Navy’s IAVM program • Network Vulnerability Detection Tools- Retina/REM • Patch Management Tools • LANDesk • WSUS • Antivirus Tools • Centrally managed Symantec Antivirus • Barracuda Spam Filter • Network Access Control • Bradford Network Appliance • Intrusion Detection • Snort • StealthWatch

12. Motivators Why did NPS create a Vulnerability and Patch Management Program? • Attacks • Welchia and Blaster – 2003 • Other attacks have followed and continue to pose a significant threat. • Mandates • DoD/Navy - Information Assurance Vulnerability Management (IAVM) Process – CJCSM 6510.01 • Best practices

13. Vulnerabilities over the last 10 years Reference: http://nvd.nist.gov/statistics.cfm

14. Scoping the Problem • Our EDU network poses the biggest challenge • Largest network at NPS • Transient systems • Many locally administered systems • New vulnerabilities emerge daily. • A strategy is needed that protects not only servers and network services, but workstations as well.

15. How to manage? • Effectively managing this problem requires NPS to: • Maintain awareness of our vulnerability posture. • Scan regularly to ensure compliance • Obtain Local access to all NPS assets • Update vulnerability audits. • communicate vulnerabilities/remediation to system owners. • Close the loop (feedback and documentation).

16. Where we were – May 2007 • In-house system bridged gap between Foundstone and Remedy • One remedy ticket = One vulnerability on one system • Vulnerability Technician did not work directly with system owners. • Feedback system was almost non-existant (duplicate tickets, false positives) • Vulnerability scanner was not properly configured. • We reached a critical decision point • Foundstone License was soon to expire. • Given the expense to continue with Foundstone, we needed a more cost effective solution.

17. Leveraging our unique position • Is NPS a University or a Naval Command? • We are both! • As a Navy Command, Retina/REM were available to us for free! • Rather than face a coverage gap, we began learning Retina/REM.

18. Configuration • One dedicated system allows us to scan for vulnerabilities. • 1 Dell PE1950 (Windows 2003) • eEye Retina Vulnerability Scanner • eEye Retina Enterprise Management (REM) Console • SQL Server 2000

19. Developing a scan schedule • We recommend a more frequent scan schedule than once a month. • Maintain better awareness of your vulnerability posture. • NPS scans our class B address space weekly. • Avoid scanning a large IP space in one session. • Find a scan schedule that promotes easy troubleshooting. • We scan by building, and scan 2-3 buildings per day. • Scan DMZ when load is lowest.

20. Developing a Scan Schedule (cont) • Maximize your coverage • Do your users powerdown at night? • Many of ours do. • We scan DHCP zones during the day and static IP ranges at night.

21. Host-based Vulnerability Scans • Requires local access to the machine • For windows (local or domain admin) • Other OSes (SSH account) • Vulnerability audits are usually dependent upon examination of registry settings, file version, or package. • At NPS – the majority of our systems are Windows-based and belong to our Windows domain – to those machines access is easy… but…

22. Host-based Vulnerability Scans (cont) • Access to Researchers’ systems often presents a challenge. • We have overcome these challenges by: • Establishing collegial relationships with our researchers – we try not to be “the man behind the curtain”. • Establishing a configuration management process that requires systems be rid of medium and high risk vulnerabilities.

23. Knowing who to contact • Scans are of little value if the results are not shared with system owners for remediation. • Determining a system owner for every system can be challenging and difficult to keep updated. • We use SQL triggers to automate the discovery/assignment of enterprise workstations. • Other systems are matched to an owner once a quarter.

24. Communication/Remediation • We choose different remediation Strategy dependent upon asset type: • Enterprise Servers • Enterprise Workstations • Researchers / non-Enterprise administered systems

25. Enterprise Servers • Administered by Server Management and Business Solutions Group • Server management applies OS patches • BSG applies application specific patches • Patching is performed as a part of maintenance – coincides with Patch Tuesdays.

26. Enterprise Workstations • Several tools exist which aide patch deployment to our workstations • Group Policy • LANDesk • Patch Management • Remote Control • Inventory Scanner

27. Enterprise Workstations (cont) • WSUS • Solves problem where LANDesk MS updates caused auto-reboot. • Remote Desktop to the machines • Some failed pushes are easy to fix – but cannot be deployed to all systems. • When all else fails – re-image the machine.

28. Non-enterprise administered systems • It is the responsibility of the administrator to patch their system. • But, what about people who just won’t patch? • We deny their operational requests until our security requests have been met (most common request type is firewall related). • We avoid threatening system disconnect unless absolutely necessary.

29. Closing the loop • Document the feedback you receive. • Sometimes recommended fixes fail. • Occasionally false positives are reported. • Documenting this information provides you with a clearer picture of what your actual vulnerability posture is. • We have created a separate database which contains NPS specific fix information and false positives.

30. Where are we now? April 2008 • For the time being, remedy is not a part of the vulnerability management process – all vulnerabilities are tracked within Retina/REM. • Local access has been obtained on approximately 90% of all network devices. • Retina/REM appears to be a good fit for NPS and we have no licensing worries for the foreseeable future. • Much closer relationships between Network Security and the rest of campus.

31. Summary • Significant milestones in our IA program • Deploying a vulnerability scanner (2003) • Brought visibility to the gap between patched systems an those at risk • Deploying LanDesk “push” (2004) • Shortened timeline between vulnerability and patch • Reduced sysadmin time accomplishing patching • Immediately saw a drop in virus infections. • Adding the WSUS or “pull” component to patching • Reduced the patch time again.

32. Future Work Next step: • Bradford network appliance – compliance appliance Challenges: • Scanning is limited to the access rights to each system. • Research networks protected by a Firewall or behind a separate gateway are no visible without admin rights.

33. Questions? Contact Information Terri Brutzman tbrutzman@nps.edu Jason Cullum jcullum@nps.edu