170 likes | 429 Views
An Alternate Multiplicity-2 Task Assignment Scheme for Distributed Computations. D. Szajda, J. Owen , B. Lawson, A. Charlesworth University of Richmond Richmond, VA USA. Outline. Volunteer Distributed Computing Simple Redundancy Alternatives - Vertical Partitioning - Clustering
E N D
An Alternate Multiplicity-2 Task Assignment Scheme for Distributed Computations D. Szajda, J. Owen, B. Lawson, A. Charlesworth University of Richmond Richmond, VA USA
Outline • Volunteer Distributed Computing • Simple Redundancy • Alternatives - Vertical Partitioning - Clustering • Analysis • Conclusions
I. Volunteer Distributed Computing • Organization of the distributed computation: - tasks - participants (anonymous) - supervisor • Examples: - SETI@Home, Folding@Home - DNA sequence alignment - commercial applications (exhaustive regression, graphics rendering, etc.)
When many PCs are involved… POWER! • Key idea: only significant results are returned to the supervisor • Issue addressed here: TRUST - Code executing in unknown environments - Significant results may be withheld - Cheating: credit for work not performed
II. Simple Redundancy • In an N-task computation, create identical copies and assign to 2N participants • Doubles the work required • Supervisor’s MO: if the two returned copies do not match, this signals a problem (check manually) if the two returned copies do match, it is usually assumed the work is correct
Biggest Weakness: Collusion • Many participants (and thus tasks) could be under control of a single individual - Results may be corrupted (either intentionally or unintentionally) - Significant results may be withheld
Example: Exhaustive Regression • One dependent/response variable (Y), five independent/predictor variables (X1, …, X5) Goal: find the “best” linear regression equation using any/all of the predictor variables Y = 0 + iXi + jXj + … 25 – 1 = 31 possible regression equations
Participant Assignments With 2N = 8 participants, we can divide the computation into N = 4 tasks: A, B, C, D Tasks collusion potential??
III. Alternative #1: Vertical Partitioning • Idea: spread the job of verifying the work of one participant to all other participants. Tasks Participant Assignments Subtasks
Advantages: Verifying a single task’s worth of work checks all N participants Exploits the finer task granularity and has improved control of result verification Identification of colluding parties / adversaries Drawbacks: Ability to subtask required Adversary with two tasks will always control a subtask Task assignment database management (prohibitive for large N)
Practical Vertical Partitioning: Clustering • Idea: break the computation into several clusters and apply the vertical partitioning strategy within each cluster Example: 4 tasks with 3 subtasks each, C = 2 clusters Participant Assignments
Advantages to the Clustering method: • An adversary who controls multiple participants is not guaranteed matching subtasks • Decreased task tracking overhead • Flexibility: designs can range from the two extreme cases of simple redundancy to vertical partitioning
IV. Analysis Suppose an adversary controls a certain proportion p of the 2N participants. For the three procedures, consider: • expected number of tasks/subtasks under her control? • variance? • probability of detecting the adversary if only a single task is verified?
Punch lines • Expected # of subtasks compromised: pN(2pN – 1) (for all schemes) • Variance of subtasks compromised: SR: VP: 0 • Probabilities are difficult but can be expressed
60 tasks, with C = 1 (VP), 3, 4, 6, 60 (SR) VP + C = 3 C = 4 C = 6 o SR
Conclusions • Presented a novel, tunable approach for applying redundancy in distributed computations • No additional computational burden • Improved detection of adversaries and collusion • Things get even better when ringers are employed
Questions? An Alternate Multiplicity-2 Task Assignment Scheme for Distributed Computations Jason Owen wowen@richmond.edu