Adaptive Matchmaking in Distributed Computing

1. System Model TCP/IP Producer Consumer TCP/IP TCP/IP Matchmaker mputer Engineering C Adaptive Matchmaking in Distributed Computing Koen Bertels, Kamana Sigdel, Behnaz Pour Ebrahimi, Stamatis Vassiliadis,Email:{koen,kamana,behnaz, stamatis}@ce.tudelft.nl Associated Problems • Centralized matchmaking: • Low scalability: efficient only in a certain range of • population size • Low robustness: • - when matchmaker leaves the network/ goes down • due to connection failure or overloaded with • information • Peer to peer: • Low throughput: when there is no interaction between • nodes’ neighborhoods Motivation Designing an adaptive and robust mechanism that enables the network to change its internal matchmaking approach from peer to peer to a more centralized form or vice versa whenever it is required. Adaptive Approach • For instance, • when population size of the system grows beyond the • capacity of single matchmaker, then adapt to more • localized model i.e. partition the system into • segment each segment having its own matchmaker • when population size of the system shrinks, then • adapt to more centralized model i.e. combine the • segments to reduce the number of matchmakers Meta View of the System 1 2 N 3 N-1 … Adaptation Model Node Attributes • System Level: • more matchmakers – when system grows • less matchmakers – when system shrinks • Node Level: • producer/consumer => matchmaker • matchmaker => producer/consumer Consumer attributes =>{ resource request, job manager, job queue }Producer attributes =>{ resource report , resource manager, resource context } Matchmaker attributes => { resource list, job list, matchmaking function machmaker list, load balancer } Adaptation Issues • how, when and who decides to reconfigure the • system? • how an individual node becomes a matchmaker? • how to partition the system into segments to • balance between jobs and resources? Legends Matchmaker Job Leaving/joining system Resource Computer Engineering Mekelweg 4 (15th floor) 2628 CD Delft The Netherlands Phone: (+31) 15 2783644 Fax : (+31) 15 2784898 http://ce.et.tudelft.nl Faculty of Electrical Engineering, Mathematics and Computer Science Delft University of Technology