420 likes | 526 Views
Troubleshooting an integrated XenDesktop, PVS and XenServer environment. Karen Sciberras, Escalation Engineer & Keith Mclaughlin, Lead Escalation Engineer Tuesday, May 11 th 2010. Architecture Overview. XenDesktop Setup Wizard. Pool Management Service. General Issues and Troubleshooting.
E N D
Troubleshooting an integrated XenDesktop, PVS and XenServer environment Karen Sciberras, Escalation Engineer & Keith Mclaughlin, Lead Escalation Engineer Tuesday, May 11th 2010
Architecture Overview XenDesktop Setup Wizard Pool Management Service General Issues and Troubleshooting Tools Agenda
Architecture Overview Citrix Confidential - Do Not Distribute
Architecture Overview Boot Virtual Desktop Clients Provisioning Service PXE Boot Desktop Delivery Controller MAPI VDA Clients XenServer XAPI Boot Virtual Machine Boot Virtual Machines Xen Server Citrix Confidential - Do Not Distribute
Provisioning Service Responsibilities • Creates and manages the image • Creates and manages Domain Machine Accounts • Delivers image to VDA Clients Citrix Confidential - Do Not Distribute
XenDesktop Responsibilities • Creates Desktop Groups and VDA clients • This is easily obtain using the XenDesktop Setup Wizard • Manage the Virtual Machines by: • Handling the Power Management for 'managed' desktops • Maintaining Pools of idle desktops • This is handled by Pool Management Service on DDC
XenDesktop Setup Wizard Citrix Confidential - Do Not Distribute
XenDesktop Setup Wizard • Allows an administrator to quickly create a set of virtual desktop • It is installed on the Provisioning Services Server and communicates with: • XenDesktop DDC (Desktop Delivery Controller) • Provisioning Services • Virtual Infrasturcture (Citrix XenServer, Microsoft HyperV or VMWare ESX) • What is needed: • Virtual Machine Template on the hosting infrastructure (XenServer, ESX or HyperV) • A base OS provided by Provisioning Services
XenDesktop Setup Wizard Select base OS Add Target Device in PVS Each Target Device identified by MAC PVS adds Target Device to AD Obtains list of SIDs for VMs Virtual Infrastructure Creates Desktop Group Add Virtual Machines to Group Maps UUID to SID Provisioning Service (PVS) XenDesktop Setup Tool Connect to XenServer Pool Obtain list of Templates Select a Template Create X number of VMs Desktop Delivery Controller • A MAC address created for each VM • Corresponds to the Virtual NIC of VM
Creating Machine Accounts in the domain Provisioning Service Create VDA Target XenDesktop Setup Wizard VDA Clients MAPI Create VDA Target Add VDA Target Xen Server VDA1 VDA1 SQL Database DomainController Citrix Confidential - Do Not Distribute
What is Pool Management? • Feature that controls the power states of ‘machines’ • Service that contains the logic for the power state transitions • We have three plug-ins which all talk to hypervisors: • XenServer • VMware ESX • Microsoft Hyper-V • Idle pool • Powers on machines in advance so that users don’t have to wait for them • Powers down machines when they are not required
Desktop Life cycles Off Pooled Assigned Suspended 7 Minutes Idle pool 5 Minutes Free Taint Action [immediate] Log on Log off Disconnect In Use Disconnected Reconnect Log off Tainted
Some hidden dials that can be tweaked (with care!!) • LogoffActionDelay (7 mins) / DisconnectActionDelay (5 mins) • Lets the user change their mind without waiting for a new VM to boot... • MUST be > RegistrationTimeout • RegistrationTimeout (3 mins) • How long a desktop gets to register (before showing error) • RegistrationForceShutdownTimeout (3hrs) • How long a desktop gets to register (before we force a power-off) • ShutdownTimeout (10 mins) • How long we give a VM to shutdown gracefully, before pulling the plug
General Issues and Troubleshooting Citrix Confidential - Do Not Distribute
Known Issues • Provisioning Services and Antivirus • Configuration for PVS to handle machine passwords • Virtual machines are not mapped to AD account names • Hypervisor is Overwhelmed
Provisioning Services and Antivirus • Whitelist PVS Filters • Do not scan system drive • Antivirus updates
Provisioning Services and Antivirus best practices • Limit Antivirus updates to the target. • Disable scanning of the write cache location especially if caching on server. • Do not scan I/O in real time. • Exclude scanning low level PVS drivers • BNNS.sys, BNNF.sys, BNPort.sys, and bnistack.sys.BNDevice.exe • Install Antivirus before Provisioning Services test updates on a staging image. • More information can be found: • http://support.citrix.com/article/ctx124185
Known Issues • Provisioning Services and Antivirus • Configuration for PVS to handle machine passwords • Virtual machines are not mapped to AD account names • Hypervisor is Overwhelmed
Configuration for PVS to handle machine passwords • Issue: Users were able to log into their machines but now cannot. • Administrator is able to log into the vdisk in private mode. • Resolution: Points to the AD machine account not being setup properly in AD • AD machine account password has expired • Group Policy setting: Computer Configuration -> Windows Settings -> Security Settings -> Local Policies -> Security Options • Domain member: Disable machine account password changes: Enable
Known Issues • Provisioning Services and Antivirus • Configuration for PVS to handle machine passwords • Virtual machines are not mapped to AD account names • Hypervisor is Overwhelmed
Identifying Virtual Machines • Hosting Infrastructure Identifiers look like this: • Microsoft path/to/VM • VMWare Name-UniqueID • XenServer GUID • XenDesktop uses • Active Directory Machine Identity [Machine SID] • We store this mapping as VM meta-data
VM Meta Data • CTXGuestOSID • This is the Guest OS SID. • Data is written by the DDC Farm Master. • Referenced by the DDC to keep the virtual machine and the Guest SID aligned for the PVS service. • CTXGuestMGTInfo • Data is written by the DDC Farm Master. • Keeps the virtual machine and the Pool management service aligned. • This data identifies which pool a machine belongs to • Allows the DDC to query power state of the virtual machines in that pool.
Virtual machines not mapped to AD account names Desktop in Active Directory VM in XenServer Problems [mouse over]
Known Issues • Provisioning Services and Antivirus • Configuration for PVS to handle machine passwords • Virtual machines are not mapped to AD account names • Hypervisor is Overwhelmed
Hypervisor Overwhelmed • By default, the Pool Management Service will start-up 10% of the default pool size. • In large environments, this may be more than the hypervisor/ Provisioning server will be able to handle. • To prevent this, the pool management can be configured to stagger the start-up of the Virtual Machine. • This is configured in the config file of the Pool Management Service
Hypervisor Overwhelmed • Open C:\Program Files\Citrix\VmManagement\CdsPoolMgr.exe.config • Add setting, for example: • <?xml version="1.0" encoding="utf-8" ?><configuration> <appSettings><add key="LogToCdf" value ="1"/><add key="LogFileName" value ="C:\cdslogs\VMManager.log"/><add key="LogDebug" value="1"/><add key="MaximumTransitionRate" value="20"/></appSettings></configuration> • Restart the Pool Management Service
Tools Used • CDF Tracing • Pool management Logs • Setup Logs • PVS Soap Logs • Network Tracing e.g. Wireshark
Enable CDF Tracing on XenDesktop Only available in XenDesktop CDF trace information can be written to logfiles in plain text Need to edit a text file to enable Can also use CDF Control (CTX111961)
How to Enable Logging • Manually create a directory where to store the log • Edit the config file with the following values: • Configure the value LogToCDF from 0 to 1 <add key=“LogToCDF” value=“1” /> • Add the location where log file will be stored: <add key=“LogFileName” value=“<location.log>” • Restart the service Article CTX117452 provides further information
XenDesktop Setup Wizard Logs INF:(9/30/2009 12:31:02 PM):Retrieving poolName for Pool at address http://10.90.144.102. INF:(9/30/2009 12:31:02 PM):Pool name for PoolMaster at address http://10.90.144.102 is Karen XenEnv INF:(9/30/2009 12:31:02 PM):Cloned Machine XDVDA1 (UUID : 27568153-1d53-c7c1-bcbb-438a761ff565) , MAC : 92:45:30:22:a8:bf INF:(9/30/2009 12:31:02 PM): XenManager.CloneVm method. - Exit INF:(9/30/2009 12:31:02 PM):Vm cloning for desktop XDVDA1 was successful. INF:(9/30/2009 12:31:02 PM):Vm disk provisioning for desktop XDVDA1 beginning. INF:(9/30/2009 12:31:02 PM):Calling ProvisionDisk of diskManager with args: clonedVmName = XDVDA1. INF:(9/30/2009 12:31:02 PM):-> PVSmanager.ProvisionDisk - Entry INF:(9/30/2009 12:31:02 PM):Entering IDiskManager.ProvisionDisk method. INF:(9/30/2009 12:31:02 PM):The VM XDVDA1 will be provisioned using diskTemplate XP machine. INF:(9/30/2009 12:31:02 PM):Adding new machines to provisioning server. INF:(9/30/2009 12:31:02 PM):Adding new machines to provisioning server. INF:(9/30/2009 12:31:02 PM):-> PVSmanager50.GetSharedDiskTemplates - Entry INF:(9/30/2009 12:31:02 PM):-> PVSManager50.GetSites - Entry INF:(9/30/2009 12:31:02 PM):Adding site XD with Id 05d0bb98-2bfa-4663-8b42-1c3c6085688a INF:(9/30/2009 12:31:02 PM):All 1 sites successfully retrieved.
Provisioning Services: Adding Device to Farm 010-04-13 10:49:47,609 [12] DEBUG SoapServer.ServiceMain - ** Start Command 'Add Device' 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - Username: XDS\Administrator 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - deviceName=XD3VDA1 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - collectionId=f368555d-4d66-487c-ad1c-c1bdb04a9bbe 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - deviceMac=7e-d8-03-55-99-e6 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - description= 2010-04-13 10:49:47,625 [12] DEBUG Mapi.CommandType - in CommandAddDevice.Execute 2010-04-13 10:49:47,625 [12] DEBUG Mapi.Command - Add to table Device 2010-04-13 10:49:47,625 [12] DEBUG Mapi.Command - sqlStatement = <INSERT INTO [Device] ([deviceId],[deviceName],[collectionId],[deviceMac],[description]) SELECT DISTINCT @v1,@v2,@v3,@v4,@v5> 2010-04-13 10:49:47,625 [12] DEBUG Mapi.Command - parameter values are <@v1 = 235ab677-f89a-4339-9efe-317c927054c5, @v2 = XD3VDA1, @v3 = f368555d-4d66-487c-ad1c-c1bdb04a9bbe, @v4 = 7ED8035599E6, @v5 = > 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - Command 'Add Device' returned: 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - return code: 0 2010-04-13 10:49:47,625 [12] DEBUG SoapServer.ServiceMain - ** End Command
Provisioning Services: SOAP Logs - Adding Device to Domain • 010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - ** Start Command 'MacroSet' • 2010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - Username: XDS\Administrator • 2010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - MacroSet 'Set Device' • 2010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - deviceId=235ab677-f89a-4339-9efe-317c927054c5 • 2010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - adTimestamp=1271152188 • 2010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - adSignature=8899 • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.CommandType - authGroups = <4c5b1faa-0bc7-478a-a45c-50f3e72d4549> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.CommandType - parameters preVal = <deviceId=235ab677-f89a-4339-9efe-317c927054c5> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.CommandType - record fields preConv = <adTimestamp=1271152188, adSignature=8899> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.CommandType - pre-ValidParms = <deviceId=235ab677-f89a-4339-9efe-317c927054c5> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.CommandType - post-ValidParms = <deviceId=235ab677-f89a-4339-9efe-317c927054c5> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.CommandType - pre-ConvertFields = <adTimestamp=1271152188, adSignature=8899> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.CommandType - post-ConvertFields = <adTimestamp=1271152188, adSignature=8899> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.Command - Set in table Device • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.Command - sqlStatement = <UPDATE [Device] SET [adTimestamp] = @v1,[adSignature] = @v2 FROM • [Device] d WHERE d.[deviceId] = @v3> • 2010-04-13 10:49:48,859 [12] DEBUG Mapi.Command - parameter values are <@v1 = 1271152188, @v2 = 8899, @v3 = 235ab677-f89a-4339-9efe-317c927054c5> • 2010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - Command 'MacroSet' returned: • 2010-04-13 10:49:48,859 [12] DEBUG SoapServer.ServiceMain - return code: 0
Packet Sniffers • SUM306: Citrix Provisioning Services stream process architecture and advanced troubleshooting Today at 2:30-3:20.