1 / 17

CCB The Condor Connection Broker

CCB The Condor Connection Broker. Condor Connections. Central Manager. Execute Node. Job Submit Point. advertise. advertise. negotiate. you’ve been matched. run this job. transfer files. Execute Node Unreachable. Central Manager. Execute node is behind a firewall or is NATed.

rene
Download Presentation

CCB The Condor Connection Broker

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CCBThe Condor Connection Broker

  2. Condor Connections Central Manager Execute Node Job Submit Point advertise advertise negotiate you’ve been matched run this job transfer files

  3. Execute Node Unreachable Central Manager Execute node is behind a firewall or is NATed. Execute Node Job Submit Point advertise advertise negotiate you’ve been matched no go! run this job transfer files

  4. Submit Node Unreachable Central Manager Submit node is behind a firewall or is NATed. Execute Node Job Submit Point advertise advertise negotiate you’ve been matched no go! run this job transfer files

  5. Common Scenarios • Why cross private network boundaries? • Flocking • Multi-site Condor pool • Glidein

  6. CCB: Condor Connection Broker • Condor wants two-way connectivity • With CCB, one-way is good enough Execute Node Job Submit Point run this job I want to connect to the submit node transfer files reversed connection CCB_ADDRESS=ccb.host.name

  7. CCB: Condor Connection Broker • Works in the mirror case too Execute Node Job Submit Point I want to connect to the execute node run this job reversed connection transfer files CCB_ADDRESS=ccb.host.name

  8. Execute Node Job Submit Point no go! CCB_ADDRESS=ccb2.host CCB_ADDRESS=ccb1.host Limitations of CCB • Doesn’t help with standard universe • Requires one-way connectivity GCB or VPN can help

  9. Connecting to CCB CCB Server CCB server must be reachable by both sides. Execute Node Job Submit Point CCB listen CCB connect READ authorization level DAEMON authorization level CCB_ADDRESS=ccb.host

  10. CCB Server Behind Firewall CCB Server Must have an open port to connect to CCB Execute Node Job Submit Point CCB listen CCB connect open port here (default 9618) CCB_ADDRESS=ccb.host

  11. Security on Reversed Connection CCB Server Client and server security policies are enforced in logical direction Execute Node Job Submit Point CCB listen CCB connect run this job reversed connection daemon-side client-side CCB_ADDRESS=ccb.host

  12. GCB: Generic Connection Broker • GCB: Condor 6.9.13 • Clever: mostly invisible to Condor code • However, this makes some things difficult! • CCB: Condor 7.3.0 • Inspired by GCB • More tightly integrated into Condor • Not a complete replacement

  13. Why CCB? • Secure • supports full Condor security set • Robust • supports reconnect, failover • Portable • supports all Condor platforms, not just Linux

  14. Why CCB? • Dynamic • CCB clients and servers configurable without restart • Informative log messages • Connection errors are propagated • Names and local IP addresses reported(GCB replaces local IP with broker IP) • Easy to configure • automatically switches UDP to TCP in Condor protocols • CCB server only needs one open port

  15. Configuring CCB • The Server: • The collector is a CCB server • UNIX: MAX_FILE_DESCRIPTORS=10000 • The Client: • CCB_ADDRESS = $(COLLECTOR_HOST) • PRIVATE_NETWORK_NAME = your.domain (optimization: hosts with same network name don’t use CCB to connect to each other)

  16. Tests of CCB • Igor Sfiligoi’s Cross-Atlantic Mega Condor Glidein Test Pool for CMS • one machine with 70 CCB collectors • execute nodes in private networks • GSI authentication • 100,000 registered Condor daemons • 200,000 jobs/day with one schedd

  17. Summary • CCB makes Condor work if • You have one-way connectivity Fine Print: • And using Condor 7.3+ • And the private side sets CCB_ADDRESS • And the private side is authorized at the DAEMON authorization level by CCB • And the public side can connect to CCB • And the public side is authorized at the READ authorization level by CCB • And not using “standard universe”

More Related