1 / 30

Support Bootcamp - Storage

Support Bootcamp - Storage. Julian Chesterfield Storage Team Development Lead. Overview. Storage Components in XenServer PV Block drivers The Storage Manager API Backend Python Drivers XAPI storage objects Logging and Debugging. Storage Components.

klaus
Download Presentation

Support Bootcamp - Storage

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Support Bootcamp - Storage Julian Chesterfield Storage Team Development Lead

  2. Overview • Storage Components in XenServer • PV Block drivers • The Storage Manager API • Backend Python Drivers • XAPI storage objects • Logging and Debugging

  3. Storage Components • Kernel and Guest block device drivers • Blkback, blktap, blkfront, qemu-dm • Storage Manager API • SR plugins • LVM, File-based, Array specific plugins • XAPI storage objects • NICs for storage traffic

  4. Block Device Driver Architecture VM VM VM Dom 0 Dom 0 Hypervisor Hypervisor Storage Array

  5. Shared Storage Access • Required for VM migration • VDI.attach is called on each host • VDIs/LUNs must be uniquely addressable • NFS target + SR UUID • SCSIidudev path • Netapp Driver LUNid mapping

  6. Domain 0 Data Path Userspace Blktap Sysfs/Proc Controls Hard Mount/Soft Mount VHD on EXT3 VHD on NFS Kernel BlkBack LVM/Device Mapper SCSI ATA FC iSCSI SAS Local HD

  7. I/O Optimisations • I/O Request Batching • Blktapuserspace aggregation • Metadata Caching for VHD chains • Asynchronous I/O request handling • Zero copy blockfront/blkback data path • Multipath I/O aggregation

  8. VHD Sparse Allocation • Microsoft Virtual Hard Disk format • Bitmap Header stores ‘allocated’ block offsets • 2 Levels of locators: 2MB chunks, 512 Byte sub-blocks • Copy-on-Write image chains enable fast clone and snapshot • Provides Filesystem flexibility with close to raw block device performance

  9. VHD Cloning A) Standalone VHD, UUID X VHD Header Block Allocation Table Data Blocks ........... UUID X VDI.clone B) Parent VHD, UUID Y + 2 children UUID Y UUID X UUID Z CoW VHD Children, UUIDs X’ and Z HDR Parent Locator Block Allocation Overlay Table Data Blocks ...........

  10. Block device I/O Multipathing /dev/mapper/SCSIID /dev/sda /dev/sdb /dev/sdc ......... /dev/sdN Host iSCSI Fibre Channel LUN A Storage Array

  11. Device Mapper Multipath (DMP) • Generic SCSI device multipathing works above FC, iSCSI, SAS • Operational modes determined by the array type: active/active, active/passive • Behavior is customisable via config file: /etc/multipath-enabled.conf • Fully integrated with SMAPI, XAPI and XenCenter

  12. Device mapper Multipath (DMP) [2] Debugging tips: Multipath –ll <SCSIid> Multipath –v0 <SCSIid> [root@qlfc ~]# multipath -ll 360a980004334694b5034476f677a6836dm-0 NETAPP,LUN [size=20G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=8][active] \_ 0:0:0:0 sda 8:0 [active][ready] \_ 1:0:0:0 sdb 8:16 [active][ready] 360a980004334694b50344d457343355adm-6 NETAPP,LUN [size=2.0G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=8][active] \_ 1:0:0:2 sde 8:64 [active][ready] \_ 0:0:0:2 sdf 8:80 [active][ready] 360a980004334694b50344a555a317434dm-4 NETAPP,LUN [size=15M][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=8][active] \_ 1:0:0:1 sdc 8:32 [active][ready] \_ 0:0:0:1 sdd 8:48 [active][ready]

  13. iSCSI Software Adapter • Loads kernel modules to handle the data path and TCP connection; iscsi_tcp,libiscsi,scsi_transport_iscsi, • All control operations managed in userspace • Backend iscsilib.py provides wrapper for software adapter to backend plugins • /sbin/iscsiadm is the management tool • IP L3 routing based, moving towards NIC binding

  14. iSCSI Software Adapter [2] Examples: Iscsiadm –m session Iscsiadm –m node [root@khotan ~]# iscsiadm -m session tcp: [3] 10.80.224.21:3260,1 iqn.2006-01.com.openfiler:tsn.julian tcp: [4] 10.80.225.95:3260,1000 iqn.1992-08.com.netapp:sn.101186238 Iscsiadm –m node –p 10.80.225.95:3260,1000 –T iqn.1992-08.com.netapp:sn.101186238 -l

  15. SAN Debugging • Never trust the SAN config! • Always start at the hardware adapter, use the Qlogic or Emulex CLI tools to verify the LUNs known to the adapter • Use ‘xesr-probe type=lvmohba’ to trigger a bus refresh

  16. LVM/DM Debugging Tips pvs => List all Physical Volumes and the Volume Group that resides on them ls /dev/mapper => to list all active device mapper nodes vgs => to list status of all Volume Groups known to the system

  17. Storage Manager API • XenServer API • SR Objects • PBD Objects • VMs etc.. External • Storage Manager API : • SR.attach • SR.create • VDI.resize etc... Internal LVM Netapp NFS .......

  18. SMAPI: Benefits • Separate interface for managing storage operations mappings • Third party plugins • Flexible definition of storage operations (LVM volumes, Files, Raw LUNs etc...) • Storage access logged independently • Python code, readable, even editable (N.B. not supported...)

  19. SMAPI • SR API calls: • SR.create/delete • SR.attach/detach • SR.scan, SR.update • VDI API calls: • VDI.create/delete • VDI.attach/detach • VDI.clone, VDI.snapshot • VDI.resize

  20. SR Plugin Code [1] • Python modules located in /opt/xensource/sm/*SR • For each module under the SR directory, XAPI issues sr_get_driver_info() on startup • Each driver defines a DRIVER_INFO struct with a list of CAPABILITIES and Device Config attributes

  21. SR Plugin Code [2] • High level SR.py and VDI.py modules handle response framing and data validation • Individual modules can implement a subset of SMAPI functions • LVM over iSCSIand HBA inherits the base LVM SR module

  22. Example Plugin: NFS SR Device-config attributes: Server Serverpath useUDP(optional) • SR.create/delete – Create or delete a directory on the given server/path, identified by the SR UUID • SR.attach/detach – Softmount or unmount the NFS target to the local mountpoint (/var/run/sr-mount/<SR UUID>) • SR.scan – Synchronise the contents of the directory with the SR VDI records in XAPI

  23. Example Plugin: NFS SR [2] • VDI.create/delete – Create or delete a new VHD file (sparse provisioned) • VDI.attach/detach – Verify the file is accessible and return the FS path to the VHD file for use by the blktap I/O handler • VDI.clone, VDI.snapshot – Pause any active blktap instances accessing the VDI. Rename the VDI to a new UUID and create 2 child nodes that reference the parent. • VDI.resize (George release onwards) – Resize the BAT, increase the virtual size of the VHD

  24. Netapp SR Motivations [1]: LUN Creation Example Cloning is not supported through the web UI, you to script it or use the SnapDriveproduct Steps to create the LUN and assign it to the host

  25. Netapp SR motivations [2]: The benefits of the SMAPI

  26. XAPI to SMAPI Mappings • XAPI Operations map down on to a subset of Storage Manager API operations • XAPI handles end-to-end VM storage management from provisioning of storage through to activation of the data path: • Vm.create will call VDI.create multiple times (for each virtual disk) • For image configuration (e.g. Debian installation) it will attach the disks to Dom0 • VM.start will attach all the virtual disks and instantiate blktap/blkback • VM.shutdown will detach the disks once the VM has stopped

  27. Logging/Debugging • All backend drivers use the Smlog to record storage events • /var/log/Smlog • Logs are rotated, same as system message log, xensource.log files • In the python module util.py there is a helper log function -> util.SMlog(STRING) • On retail edition, all python backend files are editable for logging/debugging purposes

  28. Debugging tips • Use tail –f on any log file while actively debugging a system • Correlate logs between xensource.log, SMLog and messages • Verify IP settings, firewall config for any IP based storage connections • Check status of dependent objects, such as PIFs, VIFs, PBDs etc...

More Related