240 likes | 305 Views
2012 HPC Summit for Wall Street Storage Architecture for High Performance. Next-Generation Distributed Storage – Panel Discussion.
E N D
2012 HPC Summit for Wall StreetStorage Architecture for High Performance
Next-Generation Distributed Storage – Panel Discussion The ability of servers to process high volumes of data coupled with advances in the bandwidth of storage media has begun to reorder traditional storage architectures. In large server clusters, having compute resources and storage resources on separate ends of a uniform network is no longer practical. This panel session will dive into the challenges of compute frameworks with distributed storage and explain how diverse applications, from grid computing to databases, are moving to this architecture to accelerate application performance. Panelist Michael Versace, Research Director, IDC Financial Insights (Moderator) Bob Quinn, Vice President and Chief Technologist, LSI Corporation Gary Smerdon, SVP & GM, Accelerated Solutions Division, LSI Corporation Dave Weber, Program Director, Global Financial Markets & Wall Street Center of Excellence, IBM Systems and Technology Group VinodKutty, Associate Director, Head of Distributed Computing/Open Systems R&D, CMEOwen Monroe, Fellow and R&D Specialist, Oktay Technology
Who is IDC? • Since 1964, IDC has been providing: • technology intelligence • industry analysis • market data • strategic guidance • tactical guidance • to users, providers, and builders of information technology. A division of IDG, the world's leading technology media, research, and event company.
IDC’s Global Research Network • 1200+ analysts providing a global information community • Coverage of 50+ countries around the world • Almost 50 years experience analyzing IT and Communications markets
Michael Versace GlobalDirector,Risk Analytics mversace@idc.com p: 508-988-7812 social @versace57 www.idc-fi.com IDC Financial Insights Community: www.idc-insights-community.com/financial
High Performance Storage ArchitecturesNext Generation Discussion Premise • Erasing the Divide between Storage and Servers • Performance mandates driving flash-based storage, database performance, IO performance, application acceleration • Flash based storage is be architected “in” at one of two locations, on servers as either internal or DAS or as network-based storage arrays. • Predominate Use Cases on Wall Street in Trading Risk & Bank Book Analytics • Multi-class Asset Modeling, Valuation, Simulations • Liquidity, High Frequency Trading • Metrics are Different for IO intensive applications • Cost per IOPS vs. cost per capacity • Infrastructure Considerations • Flash cache options – positioning flash cache in servers, arrays • Openflow architecture considerations • How transparent to the application and CIO • Ownership, management operations risk 6
It’s Not All About PerformanceOther Drivers Forcing Industry Transformation Cloud Mobility Social Business Big Data/ Analytics Cloud, Mobile, Analytics & Social Distributed Storage Strategies
The Challenge: Closing the Performance Gap Memory / Storage Gap (Enterprise) Increase storage performance to close CPU / Storage gap Processor L1 Cache 1 ns L2 Cache 10 ns Use best of Flash and Hard Drivesfor optimal price / performance Main Memory 100 ns Latency Penalty for leaving memory hierarchy 100,000x Transparent placement of data T1 Storage 10,000,000ns (10 ms) T2 Storage 20,000,000ns (20 ms) Near Line >20,000,000ns (>20 ms) Don’t require application rewrites, or expensive admin time
Evolution of Caching CPU CPU CPU L1 Cache L1 Cache CPU L2 Cache L2 Cache Cache L3-Cache Northbridge Northbridge Northbridge Northbridge Memory Memory Memory Memory IOC IOC IOC Flash Storage Cache Disk Disk Disk
Growing Gap: System Needs ↔ Flash Capabilities • Increasing Writes • Higher Reliability • More Performance System Requirements Growing Dependence on Flash Storage Processingbased solutions • All flash is different and changes every 18 months! • Minus 30% CAGR per bit cost • Endurance, Reliability, Performance, Interface 24-bit • Less Writes • Lower Reliability • 30% Cost Reduction Y-Y Flash Capabilities 15-bit 8-bit 4-bit Time
The Coming Evolution of Enterprise Flash Solutions Flash Segment 2015 TAM LSI has products and technologies to address all segments • Initial Segment – Early Adopters • Costly, but high value non-transparent storage $500M High capacity, PCIe flash for primary storage • Using flash to transparently accelerate high value storage • Balance of cost and value Medium capacity flash with seamless caching / acceleration software $1.4B • Bringing value of “flash to the masses” • Smartest use of flash providing best value Additional Growth Opportunity Low capacity flash with seamless caching / acceleration software Source: IDC 2012
Distributed Storage and Architectures • Storage Tiering and Distributed Storage • T0, but at what cost? • “Forget about storage tiers and architect from the application back” • Flash Cache for big structured data businesses • Gauge with industry performance benchmarks • Openflow architecture • Flash cache servers, arrays, data in server flash • Operational knowhow - data protection, replication, security, performance info • Storage “tiering” was a driving force behind the adoption of new storage architectures in 2011. • High-density, lower-performance drives continued to be used to store data, while faster media such as SSD, RAM were increasingly adopted for higher-performance services.
Distributed Storage and Data Protection • Data Protection – how does it change • Cache effect on end-to-end RPO and RTO • Achieving zero data loss applications • Virtualization-aware replication, snapshots, versioning, and replication management integrated together will be the new mechanism for protecting applications and data. • The market will continue to gain comfort with new RAID-alternative forms of data dispersion to provide for data recovery from failures. Increased drive capacity in concert with RAID rebuild times will cause firms to seek better alternatives.
Distributed Storage and Mobility Storage Management via Mobile Mobile Content Delivery Mobile Performance • In 2012, mobile devices will become a media of choice for managing storage. Storage administrators, architects, and engineers will expect storage management tools designed for mobile devices and will want to monitor and manage storage remotely from these access points. • 2012 will bring more storage offerings aimed at tablet and smart phone users, rather than exclusively PC users, networks
Distributed Storage and Performance CTOs view of storage performance, how far the architecture should extend. Cost vs. Performance, IO improvements or database or application acceleration Who owns, who manages? Changes every 18 months? Where to invest • Performance requirements are forcing a reshaping of enterprise and external IT as a service storage architectures as CIOs are faced with meeting improved user experience and application performance trends
Outlook Risk analytics is a target application market for flash cache marketplace. Supporting trading and bank book of business. Trading Valuations, Simulations, Performance Testing Wealth, etc. Storage performance is a differentiator. Options exist, flash Cache co-exists with current tier storage, must be integrated for protection, security, recovery Keeping data close to the user. HPC application design influenced even more by storage architectures, T0 possibilities. The industry is moving this way, but will take some time. Storage engineering and data science go hand in hand Risk analytics exist in tactical, operational, and strategic environments Analytic models are moving in-memory with the data. Data protection and security
Michael VersaceResearch Director,IDC Financial Insights Michael Versace is focused on the Global Risk end-user and technology marketplace. The global risk advisory service covers retail banking, cash and treasury management, payment services, insurance, core corporate banking technologies, and capital markets. This service helps financial institutions make mission-critical decisions on their investments in a variety of risk technologies and advises providers of these technologies on strategies for success in the market. Michael’s 25 year career in financial services has included extensive interactions with central bankers, financial services agencies, corporate and retail banks, investment firms, and the risk technology communities at both an enterprise and industry level. As a risk management practitioner, consultant, industry representative, and analyst, Mr. Versace has the background and experience to bridge a wide range of business and technology challenges facing today’s global risk managers and technology executives. Michael’s holds a B.S. in Accounting and Finance from Bentley University, Waltham, Massachusetts, and is a certified information security professional (CISSP).
Bob QuinnVice President & Chief Technologist, Corporate Strategy Office, LSI Corporation Bob Quinn is responsible for charting the technical and business strategy for the company's products. In his role he works with LSI's business units as well as partner companies to develop innovative solutions to challenges facing datacenters and mobile networks. Bob has extensive experience in the datacenter server and networking businesses, with over 15 years' experience in establishing relationships and connections with key CTOs and engineering executives in the IT industry. Bob is the founder of 3Leaf Systems, a pioneer in creating the Dynamic Data Center with ASIC and Software virtualization technologies; iModl, a developer of intelligent models of silicon components; and Network Virtual Systems, a developer of caching bridge chips for Intel processors. He also was director of Advanced Development at Unisys. Bob is a graduate of University College Cork in Ireland.
Gary SmerdonSenior Vice President and General Manager, Accelerated Solutions Division, LSI Corporation Gary Smerdon oversees all marketing, engineering and manufacturing of the company’s server-based application acceleration solutions combining PCIe flash technology with intelligent caching and management software. The Accelerated Solutions Division’s products accelerate application performance, reduce total cost of ownership, and deliver robust, interoperability to the enterprise server and storage markets. Prior to joining LSI in 2007, Gary was chief marketing officer at Tarari, which was acquired by LSI. He also was president and CEO of Greenfield Networks, where he was responsible for raising $35M in funding and leading the company from a pre-product, pre-customer start-up to an early-stage revenue company. Prior to that, Smerdon was vice president of Marketing at Marvell, responsible for the communications product group as well as all corporate marketing activities. Gary received a degree in Electrical Engineering from Duke University.
Dave WeberProgram Director, Global Financial Markets & Wall Street Center of Excellence,IBM Systems and Technology Group Dave Weber is focused on server and low latency technologies. He created and has been leading the IBM Wall Street Center of Excellence in New York City since 2008 and in 2011 has begun to expand the initiative globally. The Center has a particular focus on Financial Markets clients to help them solve their most challenging business objectives relating to ultra low latency computing, with technologies spanning solid state, hardware accelerators and acceleration techniques, network adapters and switches, as well as collaborating with partners and clients to drive new standards such as precision time. Dave also leads the IBM relationship with STAC®, the Securities Technology Analysis Center, which works with clients and vendors to create industry standard benchmarks for high-performance capital markets technologies.
VinodKuttyAssociate Director, Head of Distributed Computing/Open Systems R&D, CME Group VinodKutty plays a lead role in the design, implementation and support of the server infrastructure that handles electronic trading for CME Group. Performance, scalability, reliability, openness and cost effectiveness are constant challenges in this environment. Vinod’sresponsibilities include leading the Distributed Computing/Open Systems Research and Development activities as well as internal Linux Support. He works closely with vendors on new technology evaluations, and helps voice performance and functionality requirements with Linux distro vendors and hardware OEMs. Vinodis active in the Linux community, helping to drive features users need via the Linux Foundation. He is also an active member of Webmonsters.
Owen MonroeFellow and R&D Specialist,Oktay Technology Owen serves as an R&D specialist for clients, introducing buyer clients to emerging technology and helping supplier clients develop and deliver cutting-edge technology to meet the specific needs of the buyer clients. Prior to Oktay Technology, Owen served as the Global Storage Architect at Credit Suisse and expert in high performance computing. He was formerly a technical architect at Alliance Bernstein Global Enterprise Technical Services Group, where he helped establish the Storage, Enterprise Systems Management and Data Governance, and Legal e-Discovery strategies. Owen has also been a senior storage and systems engineer and/or architect for various financial services companies, including TD Waterhouse, Instinet, Merrill Lynch, and Morgan Stanley. At Morgan Stanley, he was one of the first technologists on Wall Street to recognize and harness the power of open systems enterprise storage arrays and MPP-based super computers. Owen was a member of the IBM Power Parallel team that developed the IBM SP supercomputer. He holds two patents in high-performance communications protocols. Owen has a Bachelor of Science degree in computer engineering from Syracuse University.