1 / 29

Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds

Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds. Rhys Davies, IT Director University of Leeds & Chairman YHMAN Ltd Friday 8 th November 2013 Location: Aldwark Manor Golf & Spa Hotel, Alne Nr York Meeting: 22nd Annual NYHDIF Conference.

juana
Download Presentation

Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Big Data & CollaborationThe Theory, Practice & Opportunity, a view from the University of Leeds Rhys Davies, IT Director University of Leeds & Chairman YHMAN Ltd Friday 8th November 2013 Location: Aldwark Manor Golf & Spa Hotel, Alne Nr York Meeting: 22nd Annual NYHDIF Conference

  2. Agenda • Introduction & background • The Theory • Approach, Vision & Plan • The Practice • Preparation, Timing, Execution & Outcomes • The Opportunity • What this might mean for Health Sciences • Where we are now • What next ? • Questions ?

  3. The Theory … My Role Approach Vision

  4. The Theory … My Role Approach Vision “To create a Collaborative Centre of Excellence which can be used for Research Computing for the benefit of Academia, Health, Commerce and the greater good; to be built on a mutually beneficial ‘model’ with the belief that by sharing assets (equipment, intellect, funding) we can deliver more, better, cheaper for all concerned”

  5. The Practice … Preparation What were the required ingredients ? People: Shared Vision; Skills; Energy; Appetite for Risk; Trust Process: Collaboration; Consistency; Secure; Sustainability Technology: Network; Compute; Storage; Data; Service

  6. The Practice … Preparation, Timing & Execution What were the required ingredients ? People: Shared Vision; Skills; Energy; Appetite for Risk; Trust Process: Collaboration; Consistency; Secure; Sustainability Technology: Network; Compute; Storage; Data; Service

  7. The Practice … Execution, Networks #1 The National and International picture Secure, Free for research Beyond the UK JANET Services offer onward connect to: UK Internet Peering Europe (GEANT Network) US Internet, Abilene & ESnet2 Japan (NI) & China (CERNET)

  8. The Practice … Execution, Networks #2 • The Local picture • Secure, • Resilient, • ‘Limitless’ bandwidth, • Free at point of consumption

  9. The Practice … Execution, Compute #1 • The UK’s first triangulated datacentre • One of the worlds’ largest spanning datacentres • Has the unique capability to span to more than 3 hubs

  10. The Practice … Execution, Compute #2 • Innovative • Award winning • Secure • Resilient • Virtual • Highly available • Linked to HPC

  11. The Practice … Execution, Compute #2 • Innovative • Award winning • Secure • Resilient • Virtual • Highly available • Linked to HPC • This also covers Security; Storage; Services

  12. The Practice … Execution, next step… Fundamental Building Blocks are in place… • The physical network • The shared virtual data centre = ‘safe haven’ • The Supercomputer • The skills necessary to exploit

  13. The Practice … Execution, Super Compute #1

  14. The Practice … Execution, Super Compute #2

  15. The Practice … Execution, Compute #3 N8 HPC ‘Approach & Way of Working’ Research-led (Vertical) Themes Network Industrial Partnerships Research Cross-cutting (horizontal) themes in methods and techniques Centre of Excellence Research Impact & Industrial Growth • Institutional, specialist research computing support • Specialist Facility Support • N8 Industry Innovation Forum • Business Engagement Teams • Research Computing Training • Doctoral Training (CDT) Infrastructure

  16. The Practice … Execution, Super Compute #4 N8 HPC • EPSRC funded in March 2012 • Capital • First year set-up and running costs • Aims • Establish a Tier 2 HPC facility • Develop a computational science research network • Share support and training expertise • Develop collaborative links with Tier 1 partners • One stop shop for business – key themes for engagement • Future running costs underwritten by partners

  17. The Practice … Execution, Super Compute #5 5312 2.6Ghz Intel Sandy Bridge cores 2:1 blocking QDR infiniband 4GB/core (256 cores @16GB/core) 174TB Lustre parallel filesystem CentOS/Redhat 6.3 based. SGE scheduler, Intel/GNU Compilers, OpenMPI/IntelMPI/MVAPICH2 Locally- and centrally-provided software. Co-located with 4500-core Leeds HPC Purchased through Esteem framework agreement: SGI hardware, Alces integration #291 in June 2012 Top500

  18. The Practice … Outcomes #1, Proctor & Gamble

  19. The Practice … Outcomes #2, Proctor & Gamble

  20. The Practice … Outcomes #3, BBC Opportunity & Challenge • Relocation of BBC to Salford • Exploring opportunities to deepen relationship with University of Manchester • “Making Musical Moods Metadata” • 128,000 audio files • 53 transformations (classifying mood as f(time)) • On current tech: Over a year of processing time Outcome • Over a year of processing down to 12 hours. • “The entire dataset was processed in only 12 hours, creating the world's largest time-varying musical feature database. Their combination of cutting-edge facilities and outstanding support was of huge benefit in getting the project completed and we look forward to working with them again.” – Chris Baume

  21. The Practice … Outcomes #4, A VRE

  22. The Practice … Outcomes #5, Secure Storage ITC • HE Community Storage • Secure • UK Located • Cost effective • Functionally richer than anything else in the current market • Institutional scale • Single point of access to all data sources • Authenticated to your systems • “UNIVAULT”

  23. Research & Commercial Users The Opportunity Research Assets MRS Electro-microscope Instrumentation & ‘the Data Deluge’

  24. The Opportunity, What might this mean for Health #1 Leeds Teaching Hospitals Trust University of Leeds Shared InfrastructureInvestment Research Infrastructure Investment Clinical Infrastructure Investment Consent system Data extraction N8 / HPC PPM

  25. The Opportunity, What might this mean for Health #1 Phenobanking Biobanking & Analysis Leeds Teaching Hospitals Trust University of Leeds Shared InfrastructureInvestment Research Infrastructure Investment Clinical Infrastructure Investment Consent system Data extraction N8 / HPC PPM

  26. The Opportunity, What might this mean for Health #2

  27. The Opportunity, What might this mean for Health #3 Figure 3

  28. What Next ? Shared Virtual Data Centre – we have our first customer and are investigating options with other HE N8 High Performance Datacentre - we are looking to develop the next iteration of this Secure Storage in the cloud - we are starting to market this and looking for BETA testers Big Data Collaboration with LTHT - MRC decision expected this month

  29. Questions… Thank you ???

More Related