1 / 24

Stephen Dart LaRDS Service Manager Monash e-Research Centre

Stephen Dart LaRDS Service Manager Monash e-Research Centre. LaRDS Staging Post Enhancing Workgroup Productivity. Managing User Expectation. In a perfect world. In reality, inconsistency. Dedicated wire 1Gb/s 125 MB per second 7.5 GB per minute 450 GB per hour 10 TB per day.

alaqua
Download Presentation

Stephen Dart LaRDS Service Manager Monash e-Research Centre

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stephen Dart LaRDS Service ManagerMonash e-Research Centre LaRDS Staging Post Enhancing Workgroup Productivity

  2. Managing User Expectation

  3. In a perfect world In reality, inconsistency • Dedicated wire • 1Gb/s • 125 MB per second • 7.5 GB per minute • 450 GB per hour • 10 TB per day • Slow Speed • 3~30 MB per second • Workstation, Server or LaRDS? • Share Hangs or Disconnects • Please Explain!

  4. Network at the Edge

  5. Complications at the Core

  6. Current LaRDS Samba service • LaRDS Samba service for workgroup sharing files • End user experience is speed limitations • Not suited for workstation backup • Not suited for bulk upload • Oversubscribed disk is pushed to tape • Something faster please

  7. Many factors to make things work slow • Current situation • LaRDS Samba based on virtual server • Workstations at the edge of the network • Network bandwidth contention getting to LaRDS

  8. Current ARMI workstation service • Single Network Port per workstation • 1Gb/s bit rate on port • Effective throughput peak below 10% • Common network switch for whole floor • Can handle many point to point within floor • Must share floor bandwidth to building switch • Common network switch for building • Must share building bandwidth to precinct switch

  9. What can be done now • Provide a local data service for workstations • Install Staging Post on same switch as users • bypass VeRA for uploads and backup • Increase bandwidth between floor switch and the precinct router • Extra floor and building uplinks • Faster links between switches

  10. What can be done now • Offload the big data as quickly as possible • To a local cache that can be used as a working share • Sync the data on a daily basis with LaRDS

  11. Something still not right • NAS on same switch and subnet as workstation • One session ok, but second session kills first! • Network engineers insist NAS too slow and dropping packets • Serious detective work starts

  12. Network Engineers in Denial • Network bandwidth to NDT server • http://ndt.its.monash.edu.au/toolkit/ • Network bandwidth to Speedtest.net • http://www.speedtest.net/ • Network Weather Map all clear • http://cacti.its.monash.edu.au/cacti/weathermap/weathermap.html • Low utilization and no errors

  13. QoS Policy set at default for VOIP

  14. Research networks generate data at theedge for upload to the core Traditional Corporate Intranet Research and Instrumentation Intranet

  15. Tackle System Integration • Rethink QoS • Trial with QoS off (unmanaged) • Open call with CISCO • TCP/IP behaviour • Get Network Engineers trained in QoS • Make sure NAS connected to AD • VeRA Samba was not AD connected

  16. What can be done now • Offload the big data as quickly as possible • To a local cache that can be used as a working share • Sync the data on a daily basis with LaRDS

  17. Updated QoS rolled out to all switches

  18. Five Size Options for Staging Post

  19. Re arrange existing disk usage • Provide two file systems match usage • Working data sets (fast, local disk) • Online now, used often, interim results • Archive data sets (deep, NFS to DMF) • Step or phase completion • Reference for future work • Storage object as a group of files • Publication and citation

  20. Integrate with Grid Access • Grid Users using DMF for home folders • Grid processes flooding DMF shares • Many small files gone by the time they hit the front of the migration queue • DMF recalls stall Grid jobs • Provide non-DMF Grid Scratch • Don’t back it up

  21. Outstanding Issues • Speeding up other VMs without hardware scale out • Presenting Samba users with indication of Offline status • User Indoctrination

  22. Questions

More Related