1 / 18

LambdaStation

LambdaStation. Phil DeMar Don Petravick NeSC Oct. 7, 2004. Overview. The Problem Space Functional Description of LambdaStation Components of LambdaStation Technical Approach Case Study Test Environment Project Outline Current Status Summary. The Problem.

solana
Download Presentation

LambdaStation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LambdaStation Phil DeMar Don Petravick NeSC Oct. 7, 2004

  2. Overview • The Problem Space • Functional Description of LambdaStation • Components of LambdaStation • Technical Approach • Case Study • Test Environment • Project Outline • Current Status • Summary Demar, Petravick: Lambda Station

  3. The Problem • Very high (1-10 Gb/s) bandwidth research networks becoming available • Typically not intended for general network traffic • Alternate paths for high impact data movement • Vision of the future: optical networks & lots of λ’s • Not integrated with facility production LANs • Production systems & storage facilities not directly accessible • Customized local access usually required… • WAN paths difficult to effectively use • Used mostly with prototypes or in demonstrations Demar, Petravick: Lambda Station

  4. The Problem (II) • Proposed implementations don’t scale • Multi-homed end system model breaks down with multiple alternate paths • Graceful failover capability missing or very complex • Virtual interface model difficult to administer & adds complexity on host systems • Applications may need customization for each new technology or high-performance path • A last mile problem between advanced WAN & facility production LANs… Demar, Petravick: Lambda Station

  5. Proposed Solution: LambdaStation • LambdaStation: an alternate path selection service • Agent to coordinate use of alternate WAN paths: • Dynamic configuration adjuster of local network devices • Goal is per-flow granularity for alternate path selection • LambdaStation is a research project • A 2-3 yr US Dept of Energy, Office of Science • Fermilab and Caltech • In very early stages of development • Still in design phase • Production network use is a ways off Demar, Petravick: Lambda Station

  6. How LambdaStation Would Help • Advanced research networks • Offer alternate, high bandwidth WAN paths • May be a scheduled resource • Typically demands customized local access • LambdaStation intended to: • Facilitates application request for high bandwidth WAN path • Coordinates alternate WAN path establishment • Dynamically reconfigures LAN forwarding of application’s data Demar, Petravick: Lambda Station

  7. Components of LambdaStation • A network path controller with modules interfacing to: • Advanced WANs • Local network infrastructure • Peer LambdaStations • Local applications & middleware • AA infrastructure • Software on host systems to coordinate with LambdaStation Demar, Petravick: Lambda Station

  8. Host Interactions • Path request primitives: • Flow identification: • Source/dest address/port • DSCP code point • Possibly IP version • Transfer characteristics • Bytes, peak rate, duration • Schedule (range) • Path status primitives: • Allocated bandwidth • Path setup status • Path termination status • Normal or early abort… Demar, Petravick: Lambda Station

  9. LAN Interactions • LAN device forwarding reconfigurations • Graceful alternate path setup & teardown • ACL modifications at site egress point • Customizable interface to fit site specifics • Non-objectives: • QoS • Traffic policing Demar, Petravick: Lambda Station

  10. WAN Interactions • Capability for multiple alternate WANs • Customizable interface to fit each WAN • Provide abstract view for applications • Path reservation • Resource availability • Scheduling • Path setup/teardown • Graceful… • Non-objectives: • QoS • Link efficiency Demar, Petravick: Lambda Station

  11. Remote Site Interactions • Establish path symmetry • Exchange traffic selectors • Coordinate WAN path setup/teardown • Ack LAN path reconfiguration • Verify path continuity • Implies concurrent connectivity via reserved & commodity network paths • Inform of: • Path termination • Traffic termination Demar, Petravick: Lambda Station

  12. Authorization & Authentication • Initial AA capability: • LambdaStation internal • Site-controlled • But some trust-level between sites • Longer term goal: • Utilize VO AA infrastructure • Raises site-level control issues Demar, Petravick: Lambda Station

  13. Technical Approach • Flow identification: • Source/destination IP and DSCP code point tag • Source/destination IP & Source/destination port # • Local path forwarding reconfigurations: • Based on policy routing ACLs • Advanced WAN interface model: • Site egress/ingress router to alternate path(s) • Layer 2 site-to-site connections (vlan) or MPLS paths • Transparent failover back to production path: • Bandwidth limits or QBSS on fall-back traffic Demar, Petravick: Lambda Station

  14. Use Case • FNAL SRM gets request to copy 3.5TB data set to CMS tier-2 site - Transfer involves 3500 1GB files stored on 50 dCache systems • SRM initiates data transfer optimized for production network path • SRM makes alternate path request to local LambdaStation • LambdaStation: - Checks authentication & authorization - Schedules & coordinates establishment WAN path - Coordinates with remote site LambdaStation for symmetrical path - Verifies alternate WAN path continuity - Reconfigures local LAN for selective flow forwarding - Notifies SRM that alternate path is established • SRM makes parallelism changes to re-optimize data transfer • SS Potentially shapes traffic to avoid congestion at pipe entrance. • Path reservation expires; LambdaStation gracefully tears path down - Remaining data movement traffic reverts to production network path - LambdaStation notifies SRM of alternate path teardown • SRM re-optimizes data transfer parallelism to complete the transfer Demar, Petravick: Lambda Station

  15. Test Environment • LambdaStation Project Sites: • FNAL (US-CMS Tier-1 Center) • Cal Tech (US-CMS Tier-2 Site) • Working to include UCSD as well (US-CMS Tier-2 Site) • CERN CMS Edge Computing also planned • LambdaStation Test WANs: • US DOE UltraScience Net • Two 10 GE (OC192) lambda’s • MPLS tunnels over DOE’s ESnet • Sub-10GE chunk out of ESnet production network Demar, Petravick: Lambda Station

  16. Project Outline Demar, Petravick: Lambda Station

  17. Current Status • A “Pre-LambdaStation” prototype exists: • Dynamically reconfigures LAN devices to forward select traffic over alternate paths • End system-applied DSCP tags for special-treatment flows • Prototype work-arounds: • Manual input of reservation parameters • Static alternate WAN path – no setup required • Asymmetric – no reverse path setup • Results: • Alternate traffic path established & torn down cleanly • Path reroute & failback unnoticed by application Demar, Petravick: Lambda Station

  18. Summary • LambdaStation’s role in data-intensive science: • Dynamically connect production end-systems to advanced high-performance wide-area networks • Bring production-use systems to advanced networks • Bring advanced networks to production-use applications • LambdaStation’s status: • Prototyping has demonstrated feasibility of dynamically reconfiguring local network paths for specific data flows • Much work remains to be done for complete proof-of-concept • It’s a research project… Demar, Petravick: Lambda Station

More Related