1 / 33

NEON

NEON. TNC2010, May 31, Vilnius Maarten Koopmans for UNINETT Sigma maarten@vrijheid.net. Who. ING Group -2002 SURFnet 2002-6 ICTU (govt) 2006-8 vrijheid.net 2008- qtask.com ibeamsystems.com Uninett Sigma. Different mindsets. NEON Goals. state-of-the art of cloud computing;

marla
Download Presentation

NEON

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NEON • TNC2010, May 31, Vilnius • Maarten Koopmans for UNINETT Sigma • maarten@vrijheid.net

  2. Who • ING Group -2002 • SURFnet 2002-6 • ICTU (govt) 2006-8 • vrijheid.net 2008- • qtask.com • ibeamsystems.com • Uninett Sigma

  3. Different mindsets

  4. NEON Goals • state-of-the art of cloud computing; • cost of moving and running non-HPC jobs on a cloud computing environment; • how to do this in practice; • a list of identified risks/benefits on a short/long perspective.

  5. Areas

  6. Shortlists

  7. #8: Why deliver?

  8. #7 Design to fail

  9. #6 USP: “root” - really?

  10. #5 Can you trust it? “Cloud computing is about gracefully losing control while maintaining accountabilityeven if the operational responsibility falls upon one or more third parties.”

  11. #4.5 “Core infra” Computing Queues STORAGE

  12. #4 Public clouds lead • Spot instances • Elastic load balancing • Virtual Private Cloud • Elastic map-reduce • Cloudfront • ..... • SQS • SimpleDB • Cloudwatch • Autoscaling • RDS

  13. #3: Management

  14. #2 Keep an eye on Apache ZooKeeper

  15. #1 The USERS are key

  16. <Questions so far?/> or maarten@vrijheid.net

  17. Why again?

  18. Let’s zoom in on storage

  19. Storage: requirements • No client to access the data. • Transparent versioning. • Transparent encryption - both transport and storage • AAI integration • Allow sharing of resources

  20. Current Cloud storage API based - complex for end users

  21. No AAI integration at all

  22. So...

  23. Cloudbacked storage initial request access granted token returned (rotating?) authenticate user user authenticated AAI AAI: enrollment

  24. Resource naming version = 1 Map resource to hash code Versioning Continue Storage cloud Webdav daemon Metering Encryption Stream data metered through encryption ...........into the cloud “Just WebDAV”

  25. Resource naming Map resource to hash code Get reference to most recent version Versioning Acquire lock Webdav daemon Locking Return lock ...Locking...

  26. Resource naming Get most recent version = 1 Versioning Continue Storage cloud Webdav daemon Decryption Metering Stream data metered via decryption ...........from the cloud ...downloading...

  27. Stand on the shoulders of giants • WebDAV widely deployed, lots of 3rd party clients. • Service on top of Java VM • Scala (integration language) • Cloud access libraries (often Java based) • Apache ZooKeeper (configuration management, locking) • Apache Cassandra or HBase (metering) • AAI integration components • ...

  28. Lessons learnt so far: • WebDAV is a nice start for client-less access to file based resources. • CPU intensive due to the encryption per "stream". A language (model) with concurrency support is a big plus. This breaks the trend of asynchronous I/O based network services. • Stand on the shoulders of giants: Apache Zookeeper, Bookkeeper, the JVM, Scala language, libraries for cloud access • OS X requires DAV level 2 and does a lot of locking. But: from 10.5.x onwards it also does HTTP 1.1 chunked encoding; that broke a lot of servers. • Windows works best with digest authentication. • Linux seems to be most forgiving and least demanding. • all clients support SSL. • WebDAV's XML is relatively simple but the usage may differ per client type.

  29. Questions so far? or maarten@vrijheid.net

  30. Cloud management

  31. Torque/PBS on AWS via RightScale • OSGi on Eucalyptus • MPI • R • Challenge: Matlab, BLAST etc. - how to deal with licensing? Computing

  32. Questions - the final or maarten@vrijheid.net

More Related