Diagnostics

1. XenClient Enterprise 4.5 Diagnostics

2. Table of Contents

3. Engine and Synchronizer Problem Reporting The XenClient Enterprise Engine and Synchronizer both have built-in features for reporting problems directly to the Citrix support and product teams. Here is how it works: XCE Synchronizer XCE Engine User submits report from Engine or Synchronizer. Report is uploaded to a cloud-based report server over HTTPS. Reports are available to Citrix support and product engineers. XCE Report Server In order to submit a problem report, the Engine or Synchronizer must be connected to a network with Internet access. Any firewalls or HTTP proxies in the network must allow HTTPS uploads to the host “nxbr.virtualcomputer.com” (this hostname has historical significance).

4. Problem Reports and Technical Support Service Requests • Submitting a problem report from Engine or Synchronizer does not automatically create or update a Citrix Technical Support service request (SR). • If you require Technical Support assistance with a XenClient Enterprise issue, a service request should be opened before submitting the problem report. • The service request ID (or SR number) should be included in the subject of the problem report. • If it is not practical to open a SR before submitting the problem report, then you can submit the problem report first. When you open the SR, indicate that a problem report was submitted, and include the following information so it can be found: • Whether it was an Engine or Synchronizer problem report. • Approximate date and time the problem report was submitted. • Subject of the problem report. • Email address of the person submitting the problem report.

5. Problem Reporting FAQ

6. Engine Problem Reporting Applet Start the XCE Engine control panel. Select “Tools by Category”. Click the “Problem Reporting” applet.

7. Engine Problem Report Form • Subject • Provide a brief but descriptive subject. If the problem report is related to a support case or service request, include the case or service request number. • Description • The description should include: • Detailed description of the problem • Summary of events leading up to the problem • Error messages displayed in the Engine • What was done to recover from the problem? • Who requested the problem report? • Email • Enter your email address so we know who submitted the report. • Priority • The desired priority and should be based on how the problem is impacting your business operations. • Reproducibility • Indicate how reproducible the problem is. • Submit • Click Submitfor submitting the problem report.

8. Engine Problem Report Status Pending Status When a bug report is initially submitted, it will first go into “Pending” status. This means the Engine is processing the problem report and uploading it to the report server. Success Status The problem report will transition to “Success” status after the report has been generated and uploaded to the report server. Wait For Success Before Restarting If you are required to restart or shutdown the computer, wait for the problem report to reach “Success” status first. Otherwise the problem report would not get submitted, and interesting information about the state of the computer might get lost. Reports Stuck in Pending Status Problem reports can get stuck in “Pending” status if the computer does not have network access, or if a network firewall blocks uploads to the report server. The Engine might also have problems with some HTTP proxy servers.

9. Engine Diagnostics and Synchronizer • Engine diagnostics can be requested from the Synchronizer console. The content of the diagnostics is the same as an Engine problem report. This might be useful in the following circumstances. • The computer is connected to a network that does not have internet access, or that has a firewall or HTTP proxy that might block the report from being uploaded to the report server over the internet. • The engine policy is configured to block access to the launcher screen and control panel, so the problem reporting applet is unavailable. • The computer is in a remote location and cannot be easily accessed (the remote access feature might be helpful here).

10. Requesting Computer Diagnostics from Synchronizer Select the computer in the navigation panel. Click “Request Diagnostics”. Select the “Diagnostics” tab to see the status of the diagnostics request.

11. Collecting Computer Diagnostics from Synchronizer • After requesting the computer diagnostics, the request goes into a pending state, during which the following steps occur. • Synchronizer waits for Engine to check for updates. • Engine gets the message that it should generate diagnostics. • Engine generates a diagnostics package. • Engine uploads the diagnostics to Synchronizer. After the Engine uploads diagnostics to Synchronizer, a link appears which can be used to download them. Ensure that the request state is “No Request Pending” otherwise you might get an old set of diagnostics.

12. Engine Debug Levels • The debug level for Engine logging is controlled in the Problem Reporting section of the control panel. • The available settings are: • OFF: This is the default setting and is usually sufficient for most troubleshooting purposes. • VERBOSE: Occasionally requested, usually for wireless networking issues. • VERY VERBOSE: This debug level can result in unusably verbose logs and should only be set when specifically requested. • If verbose logs are requested, the debug level should be set before reproducing the problem. • The debug level is reset automatically after a period of time, or when the Engine restarts.

13. Engine TCP Dumps • A TCP dump in the Engine can be started from the problem reporting control panel. • The TCP data cannot be extracted directly from the Engine, but it is included in Engine problem reports or diagnostics. • To collect TCP dump data for a networking problem: • Turn on the TCP dump as shown. • Reproduce the problem. The TCP dump buffer file has a limited size so the problem should be reproduced quickly. • Submit a problem report or collect Engine diagnostics from Synchronizer. • The TCP dump is automatically reset after a period of time, or when the Engine restarts. • TCP dumps from the Engine are often requested in conjunction with TCP dumps taken in a Virtual Machine using a tool like WireShark (http://www.wireshark.org).

14. Synchronizer Problem Reports Synchronizer diagnostics can be generated for the primary Synchronizer server, or a remote Synchronizer server. First, select the server. Click “Submit Bug Report”. Select the “Submit a Bug Report” option if you want to generate Synchronizer diagnostics and upload them to the Citrix report server. Select “Generate Diagnostics Only” if you want to generate Synchronizer diagnostics without uploading them to the Citrix report server.

15. Synchronizer Problem Report Form • Subject • Provide a brief but descriptive subject. If the problem report is related to a support case or service request, include the case or service request number. • Description • The description should include: • Detailed description of the problem • Summary of events leading up to the problem • Error messages displayed in the console • What was done to recover from the problem • Who requested the problem report • Email • Enter your email address so we know who submitted the report. • Priority • The desired priority and should be based on how the problem is impacting your business operations. • Reproducibility • Indicate how reproducible the problem is. • Date and Time • Indicate the date and approximate time when the error occurred.

16. Synchronizer Problem Report Tasks • When a request for Synchronizer diagnostics is submitted, a task is created to handle the request. • The initial task state is “Queued” and it might take up to a minute to start running. • When the task is complete, the diagnostics are ready. • When the task is complete, the diagnostics have been uploaded to the Citrix report server. They can also be downloaded from the Synchronizer.

17. Generating Synchronizer Diagnostics • To generate Synchronizer diagnostics without uploading them to the Citrix report server: • Select the primary or remote server in the console navigation panel. • Click “Submit Bug Report”. • Choose the “Generate Diagnostics Only” option. • A background task is created to handle the request. When the task is complete, the diagnostics are available to download from the Synchronizer console.

18. Synchronizer Publish Diagnostics • There is a special set of diagnostics available from the Synchronizer for publish failures. When a publish operation fails, these diagnostics can be downloaded from the failed publish task. • Locate the failed publish task in the Synchronizer console. • Click the indicated icon to display the task details. The icon looks like this: • Click the link to download publish diagnostics. • The publish log might also be useful to review. • Publish diagnostics cannot be uploaded to the Citrix report server from the Synchronizer. If Citrix support asks for publish diagnostics, you must retrieve them manually then send them through email or upload to ShareFile or a FTP server.

19. Synchronizer Log Files • Whenever possible, Synchronizer diagnostics should be generated through the Synchronizer console using the methods outlined previously. • But sometimes it might not be possible to collect Synchronizer diagnostics, for example, if a problem with the Synchronizer prevents the console from starting properly. • In these cases the Synchronizer log files can be collected directly. The default folder for Synchronizer log files is:C:\Program Files (x86)\Apache Software Foundation\Tomcat 6.0\Logs • If you are asked to collect Synchronizer diagnostics, but cannot collect them through the Synchronizer console, zip up this folder instead and send it in. • This folder might be several megabytes in size so you might be required to use ShareFile or a FTP server to submit them.

20. Synchronizer Installation and Upgrade Logs When running the Synchronizer installation program to install or upgrade Synchronizer, the following log files are created which might be useful in troubleshooting install or upgrade problems. The Synchronizer install program will create a log file in the Temp folder for the user running the installer. Files beginning with “bitrock_installer” are Synchronizer install log files. The installer can also generate log files in the Synchronizer installation folder. But if the initial installation of Synchronizer fails, this folder is usually deleted as part of the post-install cleanup process.

21. Windows Crash Dumps • If a Windows Virtual Machine crashes or blue-screens, it should generate a crash dump file into one of the following locations: • L:\NxTop\KERNEL.DMP • C:\KERNEL.DMP • C:\Windows\MEMORY.DMP • Note that “L:\NxTop” is a hidden folder. The dump file might have a different name but it should end in a “.DMP” suffix. These files can be large so they should be uploaded to FTP or ShareFile, not sent as email attachments.

22. Manual Crash Dump Generation • Sometimes it might be useful to force Windows to crash and generate a dump file. This can be done by enabling the hotkey labeled “Platform: Crash VM for Debug” in the Engine control panel. • Open the Engine control panel and choose “Tools by Category view”. • Open the “Hot Keys” applet. • Enable the hotkey labeled “Platform: Crash VM for Debug”. • Bring the Virtual Machine you want to crash to the foreground. • Enter the hotkey, this should cause the Virtual Machine to blue-screen.

23. Log Files in Windows Virtual Machines • The following log files are available in Windows Virtual Machines running on XCE Engine. • If you are asked for “Windows VM log files”, collect all of these. • If you are only asked for the “NxTopMonitor.log” file, the others are optional. • L:\NxTop\Logs\NxTopMonitor.log • This file contains lots of useful information about how the Virtual Machine is interacting with the Engine and can be very helpful in troubleshooting issues with monitors, networking, optical drives, and many other problems within a Windows Virtual Machine.Note: • L:\NxTop\ is a hidden folder. • Some Virtual Machines might have the local drive mapped to a different drive letter. • C:\Program Files\NxTop PV Drivers\ • This folder is in “Program Files (x86)” on 64-bit Windows. Get these files: • setupapi.dev.log • install.log • DPINST.LOG • C:\Windows\inf\setupapi.dev.log • This file contains the latest record of device installs in Windows.

24. Windows Ring Buffer Tracing • The NxTop PV Drivers package includes a utility that can be used to dump the Windows kernel ring buffer.  This is often useful when troubleshooting issues in a deployed Virtual Machine.  In particular it can be useful when USB devices are not working as expected. • Start a command prompt with Administrator privileges. • Navigate to one of these folders: • 32-bit Windows: C:\Program Files\NxTop PV Drivers • 64-bit Windows: C:\Program Files (x86)\NxTop PV Drivers • Change directory into the “winxp”, “winlh”, or “win7x64” folder. • Change directory into the “vcilog” folder. • Run this command to start the trace: vcirb.exe > L:\ringbuffer.log • Reproduce the problem while the trace is active. • To stop tracing, press ENTER in the command prompt window. • Send L:\ringbuffer.log to whoever requested the tracing. • The trace output file “L:\ringbuffer.log” is an example. You can save it wherever you like.