160 likes | 175 Views
Fermilab: Windows Patching. The Next Generation. Fermilab Windows Patching – First Generation. To be on our network, a limited set of mandatory patches had to be on systems The lab allowed OU managers to use 3 rd party tool if desired
E N D
Fermilab:Windows Patching The Next Generation HEPiX Nov 5, 2007 Fermilab Windows Patching
Fermilab Windows Patching – First Generation • To be on our network, a limited set of mandatory patches had to be on systems • The lab allowed OU managers to use 3rd party tool if desired • Computing Division used and provided monthly patching via SMS – • SMS team created the package • OU Managers define collections and advertise the patches • WSUS used as final ‘safety-net’ for Fermilab defined mandatory patches HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – Rules Changed • The Lab’s rule now is all of Microsoft’s Critical/Important monthly patches are to be installed on all systems before the next patch set is released • Fermilab’s Computer Security Team (CST) monitors systems for key patches. If patches missing – system is blocked from the network. KTF – 10-30-2007 HEPIX Presentation
Windows Patching – Prior to 3rd Qtr 2007 • Tier 1 – Solution selected by Div/Section/Major Experiment • Tier 2 – SMS • Very powerful and flexible in scheduling • Complex in administering • Wealth of details on activity • Tier 3 – WSUS • Similar to Microsoft Windows Update • Used only for Fermilab mandatory patches HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – After 3rd Qtr 2007 • We swapped Tiers 2 and 3 (very few have their own Tier 1) • Now we have a lab-wide schedule for patching • Pre-defined key client groups (OU managers define who belongs to which special groups) • Get user involved - Nag user to reboot • User will receive a pop-up every 2 hours until he/she reboots the computer (if a reboot is required) • Force reboot the day before next patches released • Catch users that are away for long periods of time • Patches are cumulative • Simplify process for systems that were away for long periods of time • Include all Microsoft High Priority patches HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – More like Windows Update HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – Why Did We Change? • Streamline the operation and reduce redundancy • Improvements in WSUS • Mimic Windows Update • Advertise same patches as in Windows update (except Service packs and major application updates like IE) • Reduce overhead on the SMS team and OU Managers • Provide Lab-wide patching schedule • All users patched with same schedule • Easier for user/desktop support team to verify the computer is patched • User can use Windows update to verify they are patched HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – Are we replacing SMS? • SMS does not go away. SMS still plays key roles: • Inventory and reporting • Application installs/Application patches • Mandatory patch rollouts • Service pack installs • Major OS/application upgrades like IE (with a FERMI tailored version) • Leverage SMS to do what it does best • Use SMS and WSUS to deploy special patches when needed HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – WSUS Client Groups • WSUS allows us to define multiple computer groups (client groups) • Client Groups allow us to tailor patching schedule and options to specific groups of computers • WSUS client group setting can be configured via domain Group Policies or registry settings • A batch script can be used for non-domain systems (and long term visitors) HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – Fermilab WSUS Client Groups • General • This is the majority of systems in the domain • Pilot-Testers • Systems used by computer-savvy users, or servers/desktops that can help test the new patches • Kiosks • Multi-user or systems with no assigned user • Manual • Systems that provide a service to multiple users or computers that should be patched manually • Defer • Special case systems that can not be patched at this time (will need special approval) • Additional groups (if needed) • Can create tailored groups for special needs (like control systems) HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching - Process Patch Tuesday -Review- Day before next patches released – force reboot of systems Rollout to Pilot-Testers Begin to ‘NAG’ user to reboot system Rollout to Kiosks FERMI Patch Day Rollout to the lab HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – How we make it work • Lab-wide GPOs set in our domain • OU level GPOs allow managers to define ‘non’ general machines • GPOs use the ‘scope’ feature to control membership • When patches arrive – automatically advertised to ‘pilot’ group • If no issues during testing, the WSUS admins advertise patches to all WSUS groups • Updates set with ‘deadline’ to ensure patches get installed before next patch date HEPiX Nov 5, 2007 Fermilab Windows Patching
Group Policy using ‘Scope’ HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – Global Groups for WSUS Client Groups • Domain Global groups are populated with the computer account names that the special WSUS client groups (kiosks, manual, and pilot-testers) are targeted to. • Group ‘defer’ can only be updated by the Domain Admin staff and OU Managers. • By default, all computers are members of the ‘General’ group • WSUS Client groups not dependent on which sub-OUs computer accounts are in HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching - Results • After converting – 80% of all systems usually fully patched within 9 days after Microsoft releases monthly patches • WSUS smoother operation for our desktop admins • Desktops/laptops throughout the lab are more consistent in what is patched • Long term visitors are easily patched HEPiX Nov 5, 2007 Fermilab Windows Patching
Windows Patching – What happens to systems Off-site? • Like SMS, the WSUS server is blocked from off-site. Clients will not pickup the new patches unless on-site or connecting in via the VPN. • User can invoke ‘Windows Update’ and get latest updates from Microsoft instead of our WSUS server. (Providing user is local admin of their machine). HEPiX Nov 5, 2007 Fermilab Windows Patching