1 / 52

INF-SEC2031

Been There Done That: YHMAN’s Private Cloud Implementation and Lessons Learnt Kevin Barrass YHMAN Network Development and Support Jonathan Gohstand VMware, Inc. Ed Carter YHMAN Business Manager. INF-SEC2031. #vmworldinf. Disclaimer.

lottie
Download Presentation

INF-SEC2031

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Been There Done That: YHMAN’s Private Cloud Implementation and Lessons Learnt Kevin BarrassYHMAN Network Developmentand Support Jonathan GohstandVMware, Inc. Ed CarterYHMAN Business Manager INF-SEC2031 #vmworldinf

  2. Disclaimer • This session may contain product features that are currently under development. • This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product. • Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Technical feasibility and market demand will affect final delivery. • Pricing and packaging for any new technologies or features discussed or presented have not been determined.

  3. YHMAN Presentation to VMWorld Europe, 10th October 2012 • BEEN THERE, DONE THAT: • YHMAN Private Cloud Implementation & Lessons Learnt • Ed Carter - YHMAN Business Manager • Kevin Barrass - YHMAN Network Support & Development Officer Leadership in the Public Sector The Practical Cloud: YHMAN Best Practice: YHMAN YHMAN Ltd ®

  4. YHMAN Presentation to VMWorld Europe, 10th October 2012 • Presentation Content • YHMAN Shared Virtual Data Centre (SVDC) • a Private Community Cloud • Stretched Cluster Data Centre Topology • with Secure Tenancy & Network Access • Lessons Learnt & The Way Forward • ‘Right Here, Right Now’ • - vCloud Networking and Security, vCNS 5.1 • (including Live Demo vCNS Edge Generic Firewall/NAT) • - vCloud Director with vCNI or VXLAN • - VXLAN • Q & A YHMAN Ltd ®

  5. Background • YHMAN is a joint venture company of 8 universities in Yorkshire UK • est. 1998 • The business drivers - ‘do more for less, better’ • Funding changes require UK universities to • deliver even more within tightening budgets • Institutions must meet carbon reduction commitments • Opportunities to exploit economies of scale & balance asset utilisation across shared service partners • Increasing pressure to deliver measurable cost efficiencies • To enable growth and enhanced service standards • Stringent security requirements to adhere to YHMAN Ltd ®

  6. Unique resilient ‘stretched’ 80Km Data Centre Network (DCN), currently based on 3 nodes, provides performance, business continuity & disaster recovery JANET/YHMAN Core Network Connection Points (Points-of-Presence, PoP) University of York University of Leeds University of Leeds DC1 University of Bradford Leeds Met University Leeds Met University DC1 University of Huddersfield University of Hull JANET5 & Internet • Scalable Optical Network Infrastructure: • Support for 4Gbps, 10, 40 & 100Gbps wavelengths over wide area distances using C/DWDM • Support for Ethernet & Fibre Channel Protocols 80km University of Sheffield Sheffield Hallam Univ University of Sheffield DC1 • Overlay Virtual Data Centre Network: • Low Latency allowing synchronous 2- or 3-way data storage mirroring • Providing the Data Centre (DC) interconnects, • currently 3 DCs but more can be provisioned • as demand grows, optimised for access performance YHMAN Ltd ®

  7. Stretched SVDC Network deploying Spanning Tree 802.1s • Multiple Spanning Tree, 802.1s • 802.1q and 802.3ad DC Interconnects • VRRP/HSRP YHMAN Ltd ®

  8. Highly Resilient Multi-Site Storage Cluster Network RAID Level Set at Per Volume basis YHMAN Ltd ®

  9. SVDC Tenant VMs protected by Stretched HA Cluster across 3 Sites, 80Km apart YHMAN Ltd ®

  10. In the event of a site failing: all VMs on failed site will be started on an alternate site with DRS Affinity “Preferential” Rules used to control vShield Edge placement YHMAN Ltd ®

  11. vNetwork Distributed Switch • SVDC currently uses vDS version 4.1.0 • Simplifies Management • Maintain Portgroup consistency across all hosts in cluster • Ingress traffic shaping as well as Egress • Shape traffic going in/out of vShield Edge external interface to control tenancy access to internet. YHMAN Ltd ®

  12. SVDC Tenancy Setup with Dedicated VLAN backed Portgroups, Firewall, Resource Pool and 1TB Data Store per Tenant Full Tenant Isolation from the Internet and Other Tenants YHMAN Ltd ®

  13. vShield Edge 5 1 2 3 4 YHMAN Ltd ®

  14. Client Manager’s see only their Virtual Data Centre tenancy • SVDC Tenants VM managers provided with login access to SVDC vCenter Server to manage VMs assigned to tenant. • Permissions provided to tenant to perform: • Create VMs • Power on/off VMs • Configure VMs • Console to VM • Install VMware Tools • Upload/Download from Datastore • Create Snapshots/Templates/Clones on VMs • VM Deployment options: • Tenant creates VMs/vAPP’s • Tenant deploys VM/vAPP’s • from Templates. YHMAN Ltd ®

  15. SVDC Lessons Learnt to date • VMware vSphere and vShield is providing a stable and scalable solution • Relying on vCenter client to provide a cloud-like interface for our customers is not ideal • Complex vCenter permissions, easy to make mistakes • Opening large number of infrastructure IP’s to clients • Using VLAN’s adds additional admin overhead, change complexity and makes solution less flexible • VLAN’s need to be created by Network Team • Systems Team add VLANs to Blade Chassis uplinks • (in our case, HP chassis Flex-10 cards) • Systems Team create VLAN backed Portgroup • Clients cannot self-provision networks on demand YHMAN Ltd ®

  16. SVDC Moving Forward from lessons learnt MOVING FORWARD • vCloud Network & Security deployment • Involved with VMware Beta Testing of vCNS 5.1 primarily Edge • Improve on existing vShield service we offer • Advanced Encryption Standard (AES-NI) support for secure VPN • HTTPS and TCP support on load balancer • vCloud Director • Proof of concept with vCloud Director planned • Offer true Cloud portal for SVDC clients with • Software Defined Networking based on either: • vCloud Director Networking Infrastructure (vCDNI) • MAC-in-MAC encapsulation or • Virtual eXtensible Local Area Network (VXLAN) • MAC-in-IP encapsulation • VXLAN • Ongoing discussions with VMware • Internal testing along with “pie in the sky” thoughts YHMAN Ltd ®

  17. VMware vCNS 5.1 vCNS BETA TESTING • Features Tested • Edge • Firewall • NAT/Routed • IPSec VPN • SSL VPN • Edge HA • Load Balancer • Basic Testing of App • Two Beta Builds tested • with all results regularly • fed back to • VMware YHMAN Ltd ®

  18. VMware vCNS 5.1 vCNS BETA RESULTS • Edge HA provides fast stateful failover • SSL VPN provides greater agility for our users • Ability to connect into tenancy from anywhere securely not just from site with IPSec VPN • Improved Edge Command Line Interface (CLI) • View Flow table Information • View Firewall rules with matching flow info • View statistics for a firewall rule using VSM User interface • More flexible firewall rule format • Object based • Rule Direction • Pre-NAT/Post-NAT inspection • Rules based on source and destination interface • Enhanced NAT rules with ability to add comments • Multi Interface support • AES-NI for improved VPN performance YHMAN Ltd ®

  19. SVDC vCNS 5.1 vCNS MOVING FORWARD • Completed vCNS Beta testing with VMware 3Q2012 • Re-ran beta tests on GA release of vCNS 5.1 3Q2012 • Starting internal testing of upgrade from vShield 5 to vCNS 5.1 4Q12 • Plan to deploy vCNS 5.1 4Q2012/1Q2013 • Utilise Edge HA for all tenants • Make use of new SSL VPN for VM management • Make use of new Load Balancer features • HTTPS support • TCP support for applications such as SMTP • Deploy App firewall 2Q2013 YHMAN Ltd ®

  20. SVDC vCloud Director vCD MOVING FORWARD • Completed small virtual lab of vCloud Director 1.5.1 using vCloud Director Network Isolation (vCDNI) 3Q2012 • Progress the Proof of Concept (POC) on vCloud Director 1.5.1 & 5.1 using real hardware 4Q2012 • If POC is on both vCloud Director 1.5.1 & 5.1 compare • vCDNI (MAC-in-MAC encapsulation) & VXLAN (MAC-in-IP) 4Q2012/1Q2013 YHMAN Ltd ®

  21. SVDC VXLAN VXLAN MOVING FORWARD • Initially use VXLANs with external VLANs spanning our 3 DC’s • External VLAN(s) handle North/South traffic from any of 3 DC’s DC Network Access Network YHMAN Ltd ®

  22. SVDC VXLAN VXLAN MOVING FORWARD • VXLAN and ‘Pie in the sky’ thoughts • Can VXLAN be used to eliminate the need for any VLAN’s spanning • our DC Interconnects between physical DC’s and support • Equal Cost Multipath [ECMP]? • This is something YHMAN want to achieve and are keeping a close • eye on VMware and VXLAN. Access Network YHMAN Ltd ®

  23. vCNS Edge LIVE DEMO VMware vSphere + vCNS • Using similar virtual lab as used for beta testing • Create firewall & DNAT rule to publish SSH service • Access SSH service and show vShield Manager User Interface and Edge CLI tools to trace traffic through the Edge virtual appliance • Failover vCNS Edge showing HA YHMAN vCNS Edge Demo Tenant A Legend : Tenant A VM running SSH Service- 192.168.0.2 (Inside_VM01) 192.168.142.100 192.168.0.1 Active SSH TCP22 192.168.0.2 vCNS Edge VMs inside 192.168.0.1/24 outside 192.168.142.100/24 Standby Inside Outside External Access Portgroup Outside Inside Portgroup Inside YHMAN Ltd ®

  24. YHMAN Shared Virtual Data Centre - ‘A Community Cloud’ • BEEN THERE, DONE THAT: • YHMAN Private Cloud Implementation & Lessons Learnt • Thank you - Q&A • Ed Carter - YHMAN Business Manager • Kevin Barrass - YHMAN Network Support & Development Officer http://www.yhman.net.uk/projects/index.htm YHMAN Ltd ®

  25. FILL OUTA SURVEY AT WWW.VMWORLD.COM/MOBILE COMPLETE THE SURVEY WITHIN ONE HOUR AFTER EACH SESSION AND YOU WILL BE ENTERED INTO A DRAW FOR A GIFT FROM THE VMWARE COMPANY STORE

  26. Been There Done That: YHMAN’s Private Cloud Implementation and Lessons Learnt Kevin BarrassYHMAN Network Developmentand Support Jonathan GohstandVMware, Inc. Ed CarterYHMAN Business Manager INF-SEC2031 #vmworldinf

  27. YHMAN Shared Virtual Data Centre - ‘A Community Cloud’ • BEEN THERE, DONE THAT: • YHMAN Private Cloud Implementation & Lessons Learnt • Screen Dumps – Generic Firewall / NAT http://www.yhman.net.uk/projects/index.htm YHMAN Ltd ®

  28. vCNS Edge LIVE DEMO Screen Dumps Select Edge to manage under the Datacenter>Network Virtualization>Edges Create firewall rule to allow SSH from Laptop to Inside_VM01 based on Objects YHMAN Ltd ®

  29. vCNS Edge LIVE DEMO Screen Dumps Add DNAT and apply to outside interface YHMAN Ltd ®

  30. vCNS Edge LIVE DEMO • SSH to Inside VM01 and analyse traffic flow and perform • stateful failover • View any current flow statistics for firewall rule using VSM interface • Check User created DNAT rule for hits using Edge CLI • View flow statistics for specified flow spec and flow matching • firewall rule using Edge CLI • Verify flow table is being replicated to Standby Edge • Debug traffic flow on “outside” interface • Debug traffic flow on “inside” interface • Show which Edge is active, standby • Power off Active Edge • Show SSH session is still active, also run ping to show • any lost packets YHMAN Ltd ®

  31. vCNS Edge LIVE DEMO Screen Dumps View any current flow statistics for firewall rule using VSM interface Check User created DNAT rule for hits using Edge CLI YHMAN Ltd ®

  32. vCNS Edge LIVE DEMO Screen Dumps View flow statistics for specified flow spec TCP with Destination port of 22 View flow matching firewall rule using Edge CLI YHMAN Ltd ®

  33. vCNS Edge LIVE DEMO Screen Dumps Debug traffic flow on “outside” interface Debug traffic flow on “inside” interface YHMAN Ltd ®

  34. vCNS Edge LIVE DEMO Screen Dumps Show which Edge is active, standby YHMAN Ltd ®

  35. vCNS Edge LIVE DEMO Screen Dumps • Failover Edge with SSH and ICMP session through active Edge • New DNAT and Firewall rule created to allow ICMP Ping through Edge to Inside_VM01 • Show flows on active Edge. Flow = “192.168.142.1:1168--192.168.142.100:22” • Show flows on standby edge. Flow = “192.168.142.1:1168--192.168.142.100:22” • Power off Active Edge • Show dropped pings • Show active flows on now active Edge YHMAN Ltd ®

  36. vCNS Edge LIVE DEMO Screen Dumps • Failover Edge with SSH and ICMP session through active Edge • Standby Edge takes over, Failed Edge would be restarted by HA and become Standby Edge • Active Edge has same active Flow = “192.168.142.1:1168--192.168.142.100:22” • SSH session still active due to stateful failover • Only dropped 4 pings YHMAN Ltd ®

  37. YHMAN Shared Virtual Data Centre - ‘A Community Cloud’ • BEEN THERE, DONE THAT: • YHMAN Private Cloud Implementation & Lessons Learnt • Screen Dumps – SSL VPN http://www.yhman.net.uk/projects/index.htm YHMAN Ltd ®

  38. vCNS Edge LIVE DEMO • Edge SSL VPN • Configure Edge SSL VPN • Ping and SSH over Edge SSL VPN with TCP Optimization Enabled • Run Edge CLI commands to debug Edge SSL VPN • SSH over Edge SSL VPN with TCP Optimization Disabled • Show different flow characteristics and firewall requirements • when TCP Optimization is disabled. YHMAN Ltd ®

  39. vCNS Edge LIVE DEMO Screen Dumps • Configure Server Settings • Specify interface for Edge SSL VPN to bind to (192.168.142.100) • Configure listening port (443) • Configure Cipher (AES256-SHA) • Select Server Certificate or use default Certificate (default) YHMAN Ltd ®

  40. vCNS Edge LIVE DEMO Screen Dumps • Add IP Pool • Configure IP Range and Gateway • Add description YHMAN Ltd ®

  41. vCNS Edge LIVE DEMO Screen Dumps • Add Private Network • Configure Private Network Subnet • Add description • Enable TCP Optimization to prevent TCP over TCP meltdown YHMAN Ltd ®

  42. vCNS Edge LIVE DEMO Screen Dumps • Add Authentication Server • Configure Local Authentication YHMAN Ltd ®

  43. vCNS Edge LIVE DEMO Screen Dumps • Add PHAT Installation package • Add Windows “default” installation package • Configure Edge Gateway for SSL VPN YHMAN Ltd ®

  44. vCNS Edge LIVE DEMO Screen Dumps • Add Users to Local Authentication • Add single test user YHMAN Ltd ®

  45. vCNS Edge LIVE DEMO Screen Dumps • Enable Edge SSL VPN Service • Go to Dashboard and click Enable button YHMAN Ltd ®

  46. vCNS Edge LIVE DEMO Screen Dumps • Download and Install Full SSL VPN client PHAT • Browse to SSL Service IP • Log into Edge secure webpage • Download and install full access client (PHAT Client) YHMAN Ltd ®

  47. vCNS Edge LIVE DEMO Screen Dumps • Log into SSL VPN • Run SSL VPN Client “VMwareTray Icon” • Click “Login” then enter username and password YHMAN Ltd ®

  48. vCNS Edge LIVE DEMO Screen Dumps • Debug Edge SSL VPN • Create firewall rule to allow SSH and Ping to Inside_VM01 • SSH into Inside_VM01 and run constant ping to Inside_VM0 • Show Flow for SSH and Ping Sessions with TCP Optimization enabled • Show Flow for SSH and Ping Sessions with TCP Optimization disabled Rule-id 133127 is user created rule Rule-id 131074 is Internal generated rule with Edge as source YHMAN Ltd ®

  49. vCNS Edge LIVE DEMO Screen Dumps • Debug Edge SSL VPN • Show Flow for SSH and Ping Sessions with TCP Optimization disabled (not default) Rule-id 133127 is user created rule Rule-id 131074 is Internal generated rule with Edge as source YHMAN Ltd ®

  50. vCNS Edge LIVE DEMO Screen Dumps • View Edge SSL VPN Statistics • View Edge SSL VPN stats from VSM User interface YHMAN Ltd ®

More Related