SIA320 Impact of Cloning and Virtualization on Active Directory Domain Services

2. SIA320Impact of Cloning and Virtualization on Active Directory Domain Services Dean Wells Senior Program Manager Microsoft Corporation

3. Session Objectives and Takeaways • Session Objective(s): • Review fundamental Active Directory and Windows concepts • Identity, replication, time, etc. • Provide an understanding of the risks related to cloning & virtualization within Active Directory • Key Takeaways: • Improve comprehension surrounding… • core Active Directory and Windows concepts impacted by cloning & virtualization • best practices when virtualizing DCs and domain members • successfully cloning Windows machines

4. Windows Concepts Machine Identities

5. Computer-identity comprises… • Name • Stored locally, suffixed with a $ • IP address • Network identifier • Name/IP information stored in DNS • SIDs • What are these? • What is that for?

6. What is a SID? • Protocol Documentation – Glossary: • An identifier for security principals in Windows that is used to identify an account or a group • Conceptually, a SID is composed of: • an account-authority portion (typically a domain) • all security principals created in a given domain will share the same authority-portion (SIDprefix) • a smaller integer representing an identity relative to the account authority, termed the relative identifier (RID) S-1-5-21-2127521184-1604012920-1887927527-19221

7. SID assignment • Machine SIDs • How is it assigned? See [MS-SAMR], section 3.1.1.9.2 • How many SIDs does a computer serving as a domain-member have? • Domain SID • Where does that come from? • SID Usage • Authorization • Deployment Scenarios • Lets walk through a few scenarios (some verging on the contrived, I’ll grant ya)

8. Scenario 1 • Start with a domain joined machine M1 • Clone it and boot (e.g. copy its VHD) • At this point, can the clones co-exist? • “Offline” unjoin, rename to M2, rejoin clone (M2) to the domain • And now?

9. Scenario 2 1)LUIGI, a non-DC, is cloned 4) LUIGI is joined to the child domain 2) LUIGI’s clone is renamed & promoted to a DC 5) CHILD\SuperMarioBros has PEACH\Administrator added as a group member 3) A child domain is added (a fresh OS install)

10. Scenario 3 M1 & M2 promoted as first DCs in two forests • Setup a machine M1 • Clone M1 to get M2 • Promote both in different domains in different forests • Result: 2 domains with same SID space • Establish trust between the 2 domains/forests • What happens? M1 is cloned to M2 Trust? Forest1.com SID: S-10 Computer: M1 SID: S-10 Forest2.com SID: S-10 Computer: M2 SID: S-10

11. Scenario 4 • Create domain from machine M1 (dom1.lab) • Install a new machine M2 • Clone M2 to get new machine: M3 • Promote M2 as a replica in dom1.lab • Join M3 to dom1.lab domain hosted by M1 and M2 • Anything wrong here?

12. Windows Concepts Active Directory Replication

13. Update Sequence Numbers (USN) • What’s a USN? • 64 Bit QWORD • Logical clock, per DC (USNs are local to a DC) • Never re-used and SHOULD NEVER rollback • When are USNs assigned? • (i.e. when does the clock tick?) • Assigned to new objects / update transaction • if transaction is aborted  USN skipped, remains unused • Independent from system time

14. Update Sequence Numbers (USN)

15. Object usnCreated =4711 Object usnChanged =4711 Property Value USN Version# Timestamp Originating GUID Orig. USN P1: Value 4711 1 <time> DS1 4711 P2: Value 4711 1 <time> 4711 DS1 P3: Value 4711 1 <time> 4711 DS1 P4: Value 4711 1 <time> DS1 4711 Object creation & metadata DS1 • Add new user on DS1 • DS1 USN increases to 4711 • DS1 object metadata below USN: 4710 USN:4711

16. Object usnCreated =2052 Object usnChanged =2052 Property Value USN Version# Timestamp Originating GUID Orig. USN P1: Value 2052 1 <time> DS1 4711 P2: Value 2052 1 <time> 4711 DS1 P3: Value 2052 1 <time> 4711 DS1 P4: Value 2052 1 <time> DS1 4711 Object replication & metadata DS2 DS1 • User replicated to DS2 • DS2 USN increases to 2052 • DS2 object metadata below USN:2052 USN: 2051 USN: 4711

17. High Watermark Vector • Table per NC per DC • Maintains • replication partners using DC’s DC-GUID • highest known USN from last replication • Used to detect recent changes on replication partners • so that DCs only replicate that which changed since the last replication cycle

18. DC GUID Highest known USN DS1 GUID 4711 DS3 GUID 1217 High Watermark Vector DS1 • DS4’s high-watermark vector • assumes that DS1 and DS3 are its replication partners USN: 4711 DS4 DS2 USN: 3388 USN: 2052 DS3 USN: 1217

19. Database identity • Domain Controllers are machines with machine identities • Name, SID • Domain Controllers host a database with an identity • Invocation ID, stored on NTDS Settings Object • When is it assigned/updated? • Usage of the invocation ID • Replication metadata (UTD Vector)

20. Up-to-dateness (UTD) vector • Table per NC per DC • Used to detect updates already received via another replication route • Maintains • originating DC’s *invocation ID* • highest originating USN • timestamp of last successful replication cycle • Which DCs have an entry in UTD vectors?

21. Replication timestamp Invocation ID Highest originating USN 12:02.31 DS1 GUID 4691 12:02.29 DS2 GUID 2052 12:02.36 DS3 GUID 1216 Up-to-dateness (UTD) vector DS1 USN: 4711 DS4 DS2 USN: 3388 USN: 2052 • DS4’s up-to-dateness vector • assumes that DS1, DS2 and DS3 have all originated writes against the partition DS3 USN: 1217

22. Making the UTD vector “up-to-date” • DC2 initiates replication from DC1 • DC1 determines what changes to send: • Local USN higher than the one stored by DC2 in its high watermark table • Originating USN higher than values in the UTD vector stored by DC2 • At the end of replication: • Increase DC2’s high watermark for DC1 to new DC1’s highest local USN • DC2’s UTD vector becomes the max merge of DC1 and DC2’s UTD vectors

23. Lingering Objects • An object on DC1 is lingering if: • It is not present on DC2 that fully hosts the same NC • It is not “about to” be garbage collected • The creation of that object is not part of any upcoming replication cycle • in other words, USNcreated on DC1 is lower than highest exchanged USN - as stored in High Watermark Vector for DC1 on DC2 • Detection happens when DC2 receives from DC1 an update or deletion event for the object. • Events 1388, 1988 • The fact that an object is lingering doesn’t necessarily make it “wrong”

24. USN rollback • What is a USN rollback? • corresponds to the situation where a USN which had previously been allocated to an update gets re-used • Such a phenomenon breaks the strongest assumption made in our replication algorithm • Detection: • DC2’s UTD vector indicates that it has replicated all originating updates from DC1 up to USN X1 • Next time DC2 pulls updates from DC1, DC1 “thinks” that its highest originating USN is X2<X1. • Since DC1 realizes that it has previously sent out udpates with higher USN than what it’s currently using, it quarantines itself • Event 2095

25. USN rollback USN Rollback Detected

26. USN bubbles… how a USN rollback can turn really bad USN Rollback Detected

27. Application – Backup/Restore • Reset the invocation ID • Use a supported backup/restore solution! • VSS writers, whether in Windows backup or 3rd party solutions • Last resort option… (not formally tested) • Before you apply the snapshot, disable the network adapters on the VM • Apply the snapshot • Set registry value Database Restored from Backup=1 • Reboot • Verify that the DC has a new invocation ID • Re-enable network adapters

28. Improper Backup/Restore • What else can go wrong with an improper backup/restore? • Summary of a real-world case: • 2500 users not able to log on • Users having access to resources they should not have access to anymore • Schema mismatches after Schema Master rolled back • Exchange server failing • RID pool allocated twice after RID master rolled back

29. Application – P2V migration • Is it enough to reset the invocation ID on the newly created Virtual DC? • Online or offline P2V? • Lab creation via P2V • What happens if various DCs are P2V’d at different times and placed in test network? • Recommendations: • Use P2V in SCVMM, it has a few checks in place • Reset the invocation ID • Do not place physical and P2V’d VM on same network… ever!

30. Application – RODCs • Virtualization of RODCs • Can I take snapshots of RODCs and use them? • Mostly… • Can I clone RODCs in a branch site? • No

31. Final Thoughts Security, Performance, Going all virtual, etc.

32. Security considerations • Hosts of domain controllers should be handled with same care as the DCs they host • Possible EoP from host administrator to Domain/Enterprise Admin • As possible, reduce attack surface on host • Server Core • A guest DC has admin privileges over domain members, including hosts if joined to the domain • Possibility: make the host a DC

33. Performance considerations • In testing conducted in a W2K8 Hyper-V environment • Virtual DCs perform at about 90% compared to physical DCs • Is that still true? • No, virtualization technologies improve. We’re now almost at par.

34. Going all virtualA good idea? • Key: Avoid single points of failures • Same messaging for the past 10 years • Do not place all your DCs on the same host (we’ve seen it…) • Diversify host’s hardware if possible • Maintain 1-2 physical DCs per domain?

35. Miscellaneous Considerations • Disk Write Caching (FUA) • Disk write caching setting on guest is honored by the host • Machines running hot • Host running 5 VMs gets (too) hot and shuts down VMs • Antivirus • Runs on the host, “locks” VM files (cannot boot) • KB 961804 • Snapshots and host’s disk space • What if a snapshot takes up the whole disk? • What if snapshot files improperly deleted?

36. Summary

37. Summary • Cloning non Domain Controllers? • Perhaps, risks for 3rd-party software remain an unknown quantity • Best Practice: SYSPREP instead • Cloning Domain Controllers? • ABSOLUTELY NOT • unless it’s the only DC in the entire forest and, even then, concerns remain: It won’t naturally replicate; DirSync; other apps that understand the replication fabric, etc. • Snapshotting Domain Controllers • Writeable: practically guarantees a USN rollback situation • RODCs: perhaps… but untested  the risk is unknown • TimeSync Best Practices • Under investigation: a KB update should appear soon

38. Session Evaluations Tell us what you think, and you could win! All evaluations submitted are automatically entered into a daily prize draw*  Sign-in to the Schedule Builder at http://europe.msteched.com/topic/list/ * Details of prize draw rules can be obtained from the Information Desk.

39. Appendix Time

40. Windows Time • Uses the Network Time Protocol (NTP) • [RFC1305], [RFC2030], [MS-W32T] and [MS-SNTP] • Synchronizes time within a forest • Relies on a hierarchical model • Architecture:

41. Time Synchronization • Any machine • Synchronize time from local CMOS clock on startup • Non-domain members • Synchronize time from time.windows.com by default • Domain Members • Synchronize time from any DC in their domain • Domain Controllers: 4 options: • Domain hierarchy-based sync • Manually specified sync source • All available sync mechanisms • No synchronization

42. Active Directory TimeSync hierarchy

43. TimeSync considerations • Machines sync time with local clock on startup • How does that apply to virtual machines? • By default, domain hierarchy used for time sync • By default, virtual machines sync time with their host • What happens for virtualized domain controller?

44. TimeSync for virtualized DCs • Disable time sync between guest and host • Allows default domain hierarchy to be used • Should have PDC in root use external time source • What about the sync at boot-up? • Need accurate time for the host • Hosts needs to… • Poll for time more regularly than normal clients • Have ideally the same time source • NOT have a guest as time source • Reference: http://technet.microsoft.com/en-us/library/ee522994(WS.10).aspx#BKMK_VM

45. © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.