1 / 11

“Virtually Unstoppable” July 25, 2007 Ken Fell CIO New York Independent System Operator

“Virtually Unstoppable” July 25, 2007 Ken Fell CIO New York Independent System Operator. Introduction.

halla-baker
Download Presentation

“Virtually Unstoppable” July 25, 2007 Ken Fell CIO New York Independent System Operator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. “Virtually Unstoppable”July 25, 2007Ken FellCIONew York Independent System Operator

  2. Introduction By mid-2006, the NYISO was averaging < 2% utilization across nearly 500 UNIX servers and suffering from growing power, cooling and space requirements. The financial situation was equally as bleak. Enter virtualization. By mid-2008, the NYISO will realize a 500% increase in server utilization, a 25% reduction in server footprint and a savings of $6M+.

  3. About the NYISO • The NYISO manages the electricity transmission grid and wholesale energy markets for all of New York State • 19.2 million customers • $50B in transactions since 1999 • Not-for-profit organization • 400 employees, 100 in IT • Considered a technology company by board of directors

  4. Goals • The overarching business goals included: • Reducing TCO • Increasing IT agility • The goals of virtualization included: • Increasing server utilization • Reducing data center power, cooling and footprint • Reducing cycle time for new environments • Standardization of platforms and technologies

  5. Technologies • In the beginning… • 120 - Sun B100 blades servers • 38 - Sun V480 mid-range servers • 158 total physical servers • Running Solaris 8 or Solaris 9 supporting TIBCO and BEA application servers • Production and Development/Test servers segregated by subnet and host • Note: Test environment segregated for performance, integration, unit and user acceptance testing • The transition… • 9 - Sun V890 mid-range servers • 1 V890 – Dev/Unit Testing (35 Containers) • 1 V890 – Integration Testing (48 Containers) • 1 V890 – Shared for Production DR/Performance DR (27 Containers) • 3 V890s – Performance and Availability Testing (49 Containers) • 3 V890s – Production and DR (39 Containers) • Total of 198 Virtual Machines on 9 Physical Servers • Hitachi Enterprise Storage • Enterprise Storage was selected to allow for portability of the Solaris Containers. The design made provisions to allow for a Performance Test server to be placed quickly on the production subnet and attached to the production Hitachi Disk in the event of a catastrophic server failure. This allowed us to select more cost effective server hardware with fewer swappable parts. • Veritas Storage Foundation Suite • Veritas File Systems were selected to assist with the portability of the Solaris containers using the advanced import and export capability of Veritas.

  6. Technologies • Other factors… • This effort presented an opportunity to combine application upgrades with the server migration. During these upgrades, we were able to provide additional environments for our standard release management processes. • Current Status… • We have migrated all of our TIBCO infrastructure and 50% of our BEA infrastructure to Solaris containers. • We are seeing peak loading of less than 30% of system utilization • Plans are in place to virtualize our Tru64 environment and we are looking to using HP-UX on Itanium • Phase 2 and 3 of virtualization is under way • Includes our Windows platforms using VMWare and Oracle platforms using IBM LPARs • Estimated reducing 180 Windows servers to less than 30 servers • Oracle and Tru64 platform reduction from 160 Tru64 servers to less than 10 HP-UX servers

  7. Key Decisions • Selecting the right platform was important • The NYISO could have selected either Solaris or AIX to begin its virtualization efforts • Solaris was selected due to perceived ease of administration and greater expertise with the OS • Selecting the right hardware was important • The NYISO could have selected high or low-end hardware for implementation • Low-end hardware was selected due to reduced costs and virtualization clustering advantages

  8. Challenges • Solaris containers presented unexpected networking challenges • Multiple default routes not supported well across zones • ISV licensing models were immature • Software licenses could not be limited to a zone • Educating business units when “they want their own server” • Project teams needed to migrate to availability, performance and capacity requirements versus hardware requirements • Migrating to new hardware and implementing virtualization simultaneously presented a “expand to contract” scenario

  9. Critical Success Factors • Engaging vendors early helped avoid costly design mistakes • Engaged Sun during initial architecture of the project; held at least 3 design meetings with Sun for requirements gathering and to propose several implementation scenarios • Limiting the scope of the initial migration greatly improved the project’s chances of success • Virtualization touches the entire IT organization • We limited scope to one tier of our n-tier environment • By limiting the scope we could concentrate on migration, development and testing efforts w/o stressing a resource constrained organization and minimizing visibility to our business • Creating a business case helped demonstrate value and generate support from other business units • The business case was instrumental in providing positive visibility to this effort and gain the support of our Board of Directors and CEO • This visibility took our strategy from proof of concept to one of the top 5 initiatives at the NYISO • The CIO was and is an important champion and facilitator of the effort • By supporting this effort, we were able keep priorities focused and eliminate delays

  10. Results • By mid-2008, over $6M in hardware and software savings will be realized • By 2010, the total savings will be nearly $11M • By mid-2008, total server count will be reduced from 450+ to 300 • Cycle time for new environments has gone from 2 weeks to 45 minutes • Unfortunately, administrative overhead has not been reduced

  11. Lessons Learned • Virtualization requires proper planning, and it’s not a panacea • The complexity of a virtualized environment can eliminate any cost savings you expect from reduced administration • The flexibility of the Solaris containers caused us to develop several iterations of deployments; which in turn caused significant support and training issues • Standardize when possible • Be sure to involve vendors • We did not keep Sun engaged during the entire project, causing delays in getting critical patches to solve a technical patching flaw • Be sure to get commitment from the business

More Related