1 / 48

Troubleshooting XenDesktop 5 Deployments

Troubleshooting XenDesktop 5 Deployments. Baptiste Duflos, Escalation Manager & Ken Baldwin, Escalation Engineer Tuesday, May 24 th 2011. Introduction and objectives. Case study for MCS fails to create pooled machines. Machine Creation Services introduces:.

brenden-tanner
Download Presentation

Troubleshooting XenDesktop 5 Deployments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Troubleshooting XenDesktop 5 Deployments Baptiste Duflos, Escalation Manager & Ken Baldwin, Escalation Engineer Tuesday, May 24th 2011

  2. Introduction and objectives

  3. Case study for MCS fails to create pooled machines

  4. Machine Creation Services introduces: • Fully integrated provisioning into the XenDesktop 5 console • Desktop lifecycle support and image roll-back capability • Leverages and supports all 3 major Hypervisors Citrix Confidential - Do Not Distribute

  5. Each VM consists of a Difference disk and an Identity disk VMs can be created in pooled or private mode VM VM VM Storage Diff Disk Diff Disk Diff Disk Id Disk Id Disk Id Disk Persistent Identity disk provides AD computer account info Pooled image will reset back to initial state after reboot Master Disk One copy of the base image shared by all VMs

  6. AD Identity Service Active Directory Broker Data Access SQL Data Access Data Access Machine Creation Service Machine Identity Service Infrastructure Service HCL HCL Host Service Configuration Service Hypervisors Hypervisors and Storage Citrix Confidential - Do Not Distribute

  7. Reproducing the error: failed to create Catalog Storage Machine Creation Service SQL Data Access Hypervisors HCL The Catalog could not be loaded due to the following errors: There are no master images associated with this Catalog See CTX127068 for resolutions to this problem Network Citrix Confidential - Do Not Distribute

  8. Troubleshooting Methodology – initial first look • Validate the Hypervisor is configured correctly • Check the image • Check permissions if storage path is not using local attached storage • Validate the Hypervisor permissions - CTX127546 • Try using another virtual image for creation • Check the master image snapshot wasn’t deleted • Verify the Certs and Proxy.xml - CTX125578 • Configure and test multiple host connections Citrix Confidential - Do Not Distribute

  9. Troubleshooting Methodology – Logs and Traces MCS, Controller, and Broker SQL • Service Logging - CTX127492 • SQL Trace - CTX127257 • CDF Control - CTX111961 Citrix Confidential - Do Not Distribute

  10. Machine Creation Service Log Analysis CitrixMachineCreationService:-> Citrix.XDServiceBase.LogicBase.GetRemoteServiceInstances - EntryCitrixMachineCreationService:Returning cached service instancesCitrixMachineCreationService: Citrix.XDServiceBase.LogicBase.GetRemoteServiceInstances - ExitCitrixMachineCreationService:Sorting the ServiceInstances.CitrixMachineCreationService:Using the next service instance http://xd5-lab.local/Citrix/HostingUnitService/IServiceAPI CitrixMachineCreationService:Conversion error in Property Resolver. Exception is System.NullReferenceException: Object reference not set to an instance of an object.at HostingUnitServiceClient.HusClient.TranslateHostingUnit(HostingUnitInternal hostingUnit)at HostingUnitServiceClient.HusClient.GetHostingUnitDetails(Guid uid)at Citrix.DesktopUpdateManager.SDK.SDKLogic.GetHostingUnit(Guid uid)at Citrix.XDServiceBase.PropertyResolver`2.Resolve(TInput toResolve)CitrixMachineCreationService:Exception caught in PostProvTask, HostingUnit not found, not adding prefix Citrix Confidential - Do Not Distribute Citrix Confidential - Do Not Distribute

  11. MCS Log Analysis MachineCreationServiceLog:2:1:Queued task RunTask-580c3ed9-d6ac-44a1-94e2-442e015c531c, current queue length=1, high priority=0, no-op=0" MachineCreationServiceLog:2:1:VMware: Begin copy disk lenir-012603_S4B4-1-baseDisk, task RunTask-580c3ed9-d6ac-44a1-94e2-442e015c531c" MachineCreationServiceLog:2:1:Dequeued task RunTask-580c3ed9-d6ac-44a1-94e2-442e015c531c, current queue length=0, high priority=0, no-op=0" MachineCreationServiceLog:2:1:Queued task RunTask-b5e8b09e-5568-41eb-86e1-2acae9b98358, current queue length=1, high priority=0, no-op=0" MachineCreationServiceLog:2:1:VMware: Begin copy disk lenir-012603_S4B4-1-baseDisk, task RunTask-b5e8b09e-5568-41eb-86e1-2acae9b98358" MachineCreationServiceLog:2:1:Dequeued task RunTask-b5e8b09e-5568-41eb-86e1-2acae9b98358, current queue length=0, high priority=0, no-op=0" MachineCreationServiceLog:2:1:EndCopyDisk: task RunTask-b5e8b09e-5568-41eb-86e1-2acae9b98358" MachineCreationServiceLog:2:1:EndCopyDisk: task RunTask-580c3ed9-d6ac-44a1-94e2-442e015c531c" Citrix Confidential - Do Not Distribute

  12. SQL Profile trace Analysis • On the SQL Profile trace make sure to select the following: • “Security Audit” • “Stored Procedures” • Look through the trace and check for any permission errors or any failures for running a stored procedure • For our case everything looked normal so we need to focus on the CDF analysis Citrix Confidential - Do Not Distribute

  13. Using CDF Control • Parsing the CDF trace and enabling the expert shader feature allows us to quickly find exceptions which are typically highlighted in orange High level failure is: “Failed to copy all master images to all of the Hosts. No machines have been added to the Catalog.” • With CDF Control you can download the public TMF files which will allow you to parse the CDF trace and troubleshoot your issue Citrix Confidential - Do Not Distribute

  14. CDF Trace Log Analysis MachineCreationServiceLog:1:1:Converting to a return code, an exception of type: Citrix.Cds.DAL.DALDataStoreException and message: General database error: XML parsing: line 1, character 331, illegal name character.“ MachineCreationServiceLog:2:1:The DALDataStoreException, has an inner Sql exception with the Number set as 9421.“ MachineCreationServiceLog:1:1:Creating a new provisioning scheme failed with error ServiceStatusInvalidDB.“ MachineCreationServiceLog:1:1:System.InvalidOperationException: ServiceStatusInvalidDB At Citrix.DesktopUpdateManager.SDK.NewProvisioningSchemeSupport.NewProvisioningSchemeLogic.DoCommitScheme(NewProvisioningSchemeWorkflow context) MachineCreationServiceDAL:8:5:DAL >>> WorkflowAddMetadata(2bcc068d-a5b0-42c0-933b-38958a7a74bb, Citrix_DesktopStudio_ExtraWarnings, Failed to copy all master images to all of the Hosts. No machines have been added to the Catalog.)“ c Citrix Confidential - Do Not Distribute

  15. Root Cause Analysis Citrix Confidential - Do Not Distribute

  16. Resolution • This issue resulted in Citrix adding a check in the code for each call to path with improved error handling when illegal characters are discovered in the storage naming scheme. • The change has been checked into XenDesktop 5 SP1. Citrix Confidential - Do Not Distribute

  17. Troubleshooting XenDesktop 5 Session Launch using Pass-through Authentication

  18. Problem Definition • XenDesktop 5 sessions fail to launch when using pass-through authentication Steps to Reproduce: • Launch XenDesktop session from a domain-joined Windows PC • Desktop Viewer opens, and the progress wheel spins.. • VDA Windows logon screen is seen briefly Expected Results: The session logon process completes, and the Windows desktop is presented. Actual Results: The session closes immediately after flashing the Windows Logon screen

  19. Citrix Confidential - Do Not Distribute

  20. Background on the issue • XenDesktop 5 in a POC environment, XenDesktop 4 is already deployed and is in production • XenDesktop 4 sessions prompt for credentials at the Windows logon screen from the same endpoint • Explicit authentication works for both XD4&5 Citrix Confidential - Do Not Distribute

  21. Narrowing Down the Issue Three main components involved in session launch VDA Broker Endpoint SQL Workstation Agent Web Interface Online Plugin XML Service PortICA ICA Settings Controller Desktop Viewer Citrix Confidential - Do Not Distribute

  22. XenDesktop Authentication Methods Explicit Authentication Pass-through Authentication User name and password are presented directly to Web Interface site Allows Broker to validate and authenticate VDA session launch request Useful for non-domain joined endpoint authentication • User identity is verified by IIS using NTLM or Kerberos • Allows Broker to validate the user for desktop enumeration • Requires endpoint device to provide credentials directly to the ICA Server Citrix Confidential - Do Not Distribute

  23. Explicit Authentication XenDesktop 5 Broker SQL XML Services Controller Web Interface WCF HTTP(S) ICA Endpoint VDA Citrix Confidential - Do Not Distribute

  24. Pass-through Authentication XenDesktop 5 Broker SQL ICA File XML Services Controller Web Interface WCF IIS HTTP(S) ICA Endpoint VDA Citrix Confidential - Do Not Distribute

  25. Reproduce the Issue Test Cases Test Results XenDesktop 4 environment using Pass-through authentication XenDesktop 5 environment using Pass-through authentication XenDesktop 4/5 environments using explicit authentication • Reached the Windows logon screen, where I was able to login • Session launch fails at the Web Interface Site • Worked with both XD4 & XD5 Citrix Confidential - Do Not Distribute

  26. Session Launch Fails at Web Interface XenDesktop 5 Broker SQL XML Services Controller Web Interface IIS An error occurred while making the requested connection Endpoint VDA Citrix Confidential - Do Not Distribute

  27. Troubleshooting the Broker • Service Logging - CTX127492 • CDF Control - CTX111961 • XDPing - CTX123278 • Powershell SDK - CTX127254 • WCF Diagnostics- MS732009 Citrix Confidential - Do Not Distribute

  28. Broker CDF Analysis CdsXmlServices:2:1:ProcessCredentials: exception Citrix.Xms.XmlSupport.CredentialsException: ID only credentials received but TrustRequestsSentToTheXmlServicePort=false at Citrix.Xms.XmlSupport.CredentialsProcessor.ProcessCredentials(CommonCredentials RequestCredentials, CredentialType SupportedCredentials, CredentialOptions ProcessingOptions) CdsXmlServices:2:1:GetErrorIdFromCredentialsException: AccessDenied -> not-trusted CdsXmlServices:2:1:Credential Exception, reason AccessDenied: Citrix.Xms.XmlSupport.CredentialsException: ID only credentials received but TrustRequestsSentToTheXmlServicePort=false at Citrix.Xms.XmlSupport.CredentialsProcessor.ProcessCredentials(CommonCredentials RequestCredentials, CredentialType SupportedCredentials, CredentialOptions ProcessingOptions) at Citrix.Cds.Xms.Wpnbr.BaseTransaction.ProcessCredentials(CredentialType SupportedCredentials, CredentialOptions ProcessingOptions) at Citrix.Cds.Xms.Wpnbr.AddressTransaction.HandleRequest(IXmlMultiplexer multiplexer) at Citrix.Xms.XmlSupport.XmlPerf.WrapTransaction(Type t, Action transaction) at Citrix.Cds.Xms.Wpnbr.WpnbrServer.HandleRequest(HttpListenerRequest request, WindowsIdentity identity) CdsXmlServices:2:1:GetErrorIdFromCredentialsException: AccessDenied -> not-trusted Citrix Confidential - Do Not Distribute

  29. Troubleshooting: Broker Components • Searched Citrix KB for XML Service issues in XD5 • Found that XD5 broker requires XML service to trust ID-Only credentials (CTX128328) • Also required for SSO to work through Access Gateway • Configure using XenDesktop 5 Powershell SDK (CTX127254) Citrix Confidential - Do Not Distribute

  30. Session Launch Fails During Session Initialization XenDesktop 5 Broker SQL ICA File XML Services Controller Web Interface WCF IIS HTTP(S) ICA Endpoint VDA Citrix Confidential - Do Not Distribute

  31. Troubleshooting VDA: Session Launch Portica.ICA.IcaClientStack.GetCredentials CdsWorkerAgent:2:1:Validate no credentials returned Portica.BizLogic.TakeOwnershipOfCredentials Portica.GinaServer.SendAutoLogonMessage Utils.Kernel32.UnmanagedBuffer.SafeDisposeObj ThreadID=7, disposing=True, pointer=32C60E8, size=1568, source=Citrix.Portica.GinaServer.SendAutoLogonMessage Portica.GinaServer.ProcessGinaMsg Received message of type: CancelIcaConnection • PortICA Service Logs (CTX118837) • Workstation Agent Service Logs (CTX127492) • CDF Trace Modules: CdsWorkerAgentICA ServiceMF_Session_WfshellMF_DLL_CtxginaMF_Library_System • Portica_DLL_PICACredProviderPortica_DLL_PICADisplayManagerPortica_DLL_PICASessionHelperPortica_Library_picaCPHelper Citrix Confidential - Do Not Distribute

  32. Troubleshooting: VDA Components • Enforce Auto Logon (CTX127392) • Requires credentials to be passed, or the session is canceled • Enabled by default in XD5 for security purposes • Can be manually set on VDA • Create DWORD value on the VDA called 'EnforceAutoLogon' in HKLM\Software\Policies\Citrix, and set it to 0 Citrix Confidential - Do Not Distribute

  33. Troubleshooting Online Plugin Directory must exist, and be writable Endpoint • ICA Logging - CTX115304 • CDFControl - CTX124934 • DebugView - BB896647 • Client Policies - EDocs Enable LogEvidence for CST Citrix Confidential - Do Not Distribute

  34. ICA Log Analysis [KB-Win7-x32RTM] Address=10.54.67.97:1494 AutologonAllowed=ON BrowserProtocol=HTTPonTCP ConnectionBar=1 InitialProgram=#WinXP 32-bit $P8 Launcher=WI LaunchReference=EE2998E87E058B78E1CAF7050FB40E SessionsharingKey=-R7YM1LL1qw5bcb7LTq21sC UseLocalUserAndPassword=On • Desktop Group • ICA Address • Auto-Logon Allowed • Desktop Viewer • Single Sign-On Citrix Confidential - Do Not Distribute

  35. Pass-through Authentication Requirements • Searched Citrix KB for UseLocalUserAndPassword Citrix Confidential - Do Not Distribute

  36. Pass-through Authentication Client Policy Settings Citrix Confidential - Do Not Distribute

  37. Pass-through Authentication CST Override Allows all regions exceptRestricted Citrix Confidential - Do Not Distribute

  38. Client Selective Trust (CST) • Collects and analyzes ‘evidence’ from session launch details • Classifies ICA sessions into one of four regions: • oidTrustedRegion • oidIntranetRegion • oidInternetRegion • oidRestrictedRegion • Checks WI Site against Internet Explorer security zones • Blocks certain ICA Client actions (such as Pass-through) based on region settings (CTX124871) • Requires CST registry keys to be present (CTX128775) Citrix Confidential - Do Not Distribute

  39. ICA Log Analysis - CST Evidence ICA Client connection initialized AddEvidence InitialProgram=#KB-Win7-x32RTM Region All Regions AddEvidence ICAFileAddress=XenDesktop.get.services.citrite.net:1494 Region Trusted Region AddEvidence ServerAddress=XenDesktop.GET.SERVICES.CITRITE.NET Region Trusted Region AddEvidence CGPEnabled=True Region All Regions AddEvidence ServerIPAddress=10.54.67.220 Region All Regions EvidenceRequest Connection Authorisation (event: Open connection to Citrix Server) Granted • Collect • Inspect • Select • Authorize CTX124921 Citrix Confidential - Do Not Distribute

  40. Desktop Viewer CST Requirements • CST evaluates Initial Program value as evidence • Requires the desktop group name to be added to the CST whitelist if ‘Allow pass-through for all connections’ is not enabled • Used DebugView output to determine what evidence was being evaluated Citrix Confidential - Do Not Distribute

  41. CST Whitelist KB-Win7-x32RTM] Address=10.54.67.97:1494 AutologonAllowed=ON BrowserProtocol=HTTPonTCP ConnectionBar=1 InitialProgram=#WinXP 32-bit $P8 Launcher=WI LaunchReference=EE2998E87E058B78E1CAF7050FB40E SessionsharingKey=-R7YM1LL1qw5bcb7LTq21sC UseLocalUserAndPassword=On Wildcards don’t work here Citrix Confidential - Do Not Distribute

  42. Pass-through Authentication XenDesktop 5 Broker SQL ICA File XML Services Controller Web Interface WCF IIS HTTP(S) ICA Endpoint VDA Citrix Confidential - Do Not Distribute

  43. Root Cause Analysis • Broker • Required Set-BrokerSite -TrustRequestsSentToTheXmlServicePort $true • Virtual Desktop Agent • Enforce Auto Logon requires the ICA Client to automatically send credentials during ICA session launch • Endpoint • Client Selective Trust requires additional client policies to be used • Pass-through authentication is treated more securely than explicit authentication Citrix Confidential - Do Not Distribute

  44. Resolution • Provided a private binary that instead evaluates the ICA address, which supports wildcards • Client Selective Trust is being replaced by ICA File Signing • Recommending ICA File Signing as a replacement (eDoc) Citrix Confidential - Do Not Distribute

  45. Resources discussed

  46. For More Information • CTX127492 - How to enable Controller Service Logging in XenDesktop 5 • CTX128075 - XDDBDiag: XenDesktop 5 Database Diagnostics • CTX128909 - XenDesktop 5 Logon Process and Communication Flow • CTX127969 - Desktop Studio Logging Options • CTX127587 - XenDesktop 5 Reference Architecture • CTX128190 - How to Change Virtual Channel Priority in XenDesktop 5 • CTX127254 - XenDesktop 5 SDK PowerShell Cmdlet Help

  47. Questions and wrap up

More Related