1 / 19

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes. Outline. ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes.  Motivation.  Motivation.  Established Techniques.  ADaPT.

cooper
Download Presentation

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes

  2. Outline ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes  Motivation  Motivation  Established Techniques  ADaPT  Performance Comparison  Conclusions

  3. Motivation What is the Laboratory for Neural Dynamics? • A computational-science section of the Center for Neural Engineering • Part of a National Science Foundation engineering research center dedicated to biomimetic microelectronic systems • Combines computational electrophysiology, engineering, pharmacology, and other disciplines • Integrates empirically-measured, realistic, and biologically-inspired synaptic models for the purposeof temporal signals processing

  4. Motivation The Dynamic Synapse • Biologically-inspired rather than realistic • Computationally-complex and non-linear • Signals processing application was originally a proof of concept • Now a synergistic field for the Center Postsynapse Neuron Presynapse Na+ Figure 1: Electro-chemical synaptic transmission Ca2+ Ca2+ Feedback threshold threshold Action Potential input Glutamate release Synaptic Potential Summation Action Potential output

  5. A AP AP AP EPSP EPSP EPSP EPSP EPSP EPSP EPSP EPSP EPSP EPSP EPSP EPSP Motivation Dynamic Synapse Neural Networks LAYER 1 • Classical NN structure • Increased synaptic functionality • Parameter trainingvia genetic algorithms Array of K input neurons LAYER 2 3xK Pre-synaptic Matrix 3xK Post-synaptic Matrix K Output Neurons 3 2 1 AP AP A AP AP Feedback Modulation Captured Sound K length filter bank Microphone Array Figure 2: 3xK 2-Layer DSNN Single Word Classifier.

  6. Outline ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes  Motivation  Motivation Established Techniques  ADaPT  Performance Comparison  Conclusions

  7. Established Techniques Master Scatter-Gather Computation Time • Naïve Approach • Multiple SequentialScatter-Gathers • With uniform computation time, exhibits decent parallelism • With variable computation times,significant idle time Worker n -2 Worker n -1 Worker 1 Worker 2 Worker 3 Worker 4 Worker 5 Worker 6 Worker n Master Worker n -2 Worker n -1 Worker 1 Worker 2 Worker 3 Worker 4 Worker 5 Worker 6 Worker n Master Worker n -2 Worker n -1 Worker 1 Worker 2 Worker 3 Worker 4 Worker 5 Worker 6 Worker n Master Figure 3: Multi-phase evaluation of 3n genomes by n workers using naïve scatter-gathering.

  8. Established Techniques A More Efficient Mapping Master Computation Time • Asynchronous scattering • Reduced idle time for workers • Closer to optimal time to solution • Dynamic allocation of resources • More difficult Worker n -2 Worker n -1 Worker 1 Worker 2 Worker 3 Worker 4 Worker 5 Worker 6 Worker n Master Figure 4: Multi-phase evaluation of 3n genomes by n workers using a more efficient mapping.

  9. Outline ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes  Motivation  Motivation  Established Techniques  ADaPT  Performance Comparison  Conclusions

  10. ADaPT Adaptive Data-parallel Publish/Subscribe Transport Protocol • Publish/Subscribe • Worker-centric, i.e. processes subscribe to the master • Data is transported (published) to workers as events • Unsubscription is possible • Two-phase adaptive protocol • Learning phase: request-reply, monitoring of time between requests • Aggressive phase: events are pushed to workers at regular intervals

  11. Outline ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes  Motivation  Motivation  Established Techniques  ADaPT  Performance Comparison  Conclusions

  12. Performance Comparison Message-passing costs for MPI scatters • Two protocols • Aggressive & Conservative • Scatter/Gathers in most implementations use conservative protocols • Analysis due to Gropp, et. al. C(MPI Scatter) = (# pop.)[3s + r(n+3e)] Where n = event payload e = envelope r = network bandwidth s = latency Equation 1: Computation time cost in of scatters In MPI.

  13. Performance Comparison Computational Costs for Multiple scatters in MPI • Our assumption is a normally distributed population of compute times • An ideal ordering of computations would be sortedby compute time • How much idle time is present? C(Computation) = (# pop.)(avg. compute time) + (# workers)(avg. compute time) Equation 2: Computation time of a normally-distributed population using scatters in MPI. Figure 5: Graph of sorted compute times of anormal distribution illustrating idle time.

  14. Performance Comparison Message-Passing costs for ADaPT • Three different costs of event-passing in ADaPT: • Subscription • Learning Phase • Aggressive Phase C(subscription) = (# workers) x [s + re] C(learning) = (# samples) x [2s + r(n+2e)] C(aggressive) = (# pop - # samples) x [s + r(n+e)] Note: we assume control events to be of size e Equation 3: Event-passing costs of ADaPT.

  15. Performance Comparison Unsubscribe Costs for ADaPT • An unsubscribe occurs when a worker’s event buffer is in danger of overflowing • With ADaPT, an overflowoccurs when a worker receivesm-1 events triggering computetimes greater than the estimatedaverage (assuming a worker buffers m events) • Conservatively, we have decidedthat workers should clear theirbuffers before resubscribing - We used a Monte Carlo simulation (details in paper) to determine E,the % pop with compute times > than the estimated mean given error as a function of % pop. sampled P(unsubscribe) = E*Pop C m-1 Pop C m-1 C(unsubscribe) = P(unsubscribe) x [2(s+re) + (m-1)(avg. compute+ Δ)] Equation 4: Costs of worker unsubscription in ADaPT.

  16. Performance Comparison Analysis (# pop - # samples)2re + (# samples)(avg. compute time) >(# pop / m) x P(unsubscribe) x [2re + (m-1)(avg. compute time)] • Which protocol is more appropriate? • For simplicity of comparisonwe will drop the latency termand assume the number of samples to be equal to thenumber of workers • i.e. each worker’s firstcomputation is monitored Equation 5: Cost comparison of MPI vs. ADaPT. Figure 6: Graph of inequality in Equation 5.

  17. Outline ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes  Motivation  Motivation  Established Techniques  ADaPT  Performance Comparison  Conclusions

  18. Conclusions What have we shown? • ADaPT is useful when multiple scattering of data must occur due to natural aggregation • An example is the training of the DSNN using genetic algorithms • Worker-centric approach for reduced processor idle time • Unsubscription is expensive but can be avoided withgreater event-buffering capabilities • ADaPT exploits an event pattern which emerges fromthe application of a well-known architectural pattern

  19. Thank You

More Related