Arnold B. Urken Professor Emeritus Stevens Institute of Technology Castle Point on the Hudson

1. Error-Resilient Decision System SupportHow to Build IT Tools to Overcome Network Limitations and Enhance Distributed Work Arnold B. Urken Professor Emeritus Stevens Institute of Technology Castle Point on the Hudson Hoboken, New Jersey 07030 aurken@stevens.edu March 28, 2008

2. Outline • What is Error-Resilient Decision System Support? • Examples of Error-Resilient Control Mechanisms • Research Opportunities

3. Software systems that predict the conditions under which collective inferences based on incomplete and imperfect information will be the same as collective decision derived from complete and/or perfect information conditions. Network-centric intelligence IT tools for processing information in Client-server networks Peer-to-peer environments Not decision support systems => formation of preferences What is Error-Resilient Decision System Support?

4. COLLECTIVE DECISION RATINGS VOTING SYSTEM Decision System Support Mechanisms Examples

5. Central supervisor relies on 10 managers to reach consensus about knowledge to be transmitted to next shift in a 24-hour knowledge factory Binary decision task: choose A or B? Each manager casts a single vote for “A” or “B” and sends their vote to the central supervisor 6 votes, a majority, are required to reach a consensus to transmit knowledge The supervisor has received 6 votes in favor of “A” An Intuitive ExampleClient-Server Model

6. Therefore the supervisor can transmit knowledge immediately because The majority requirement has been satisfied If the remaining 4 votes went to B, the score would be changed (6 for A, 4 for B) BUT the collective inference, A, would not change So “A,” is an error-resilient collective decision because it cannot be changed by lack of information caused by network breakdowns and/or manager decision making errors. An Intuitive ExampleClient-Server Model[continued]

7. Managerial Advice Time spent explaining something or trying to reach a consensus, especially when the discussion takes place over a great distance, is time taken away from the task. For this reason, employees must be able to do their work with a minimum of interaction among sites. - Amar Gupta The Journal Business Report September 15, 2007

8. How long before the end of a shift should managers Define a consensus for transmittal? Coordinate operations in different locations when time zones overlap? How should managers engineer the collection of information in Client-server networks? Peer-to-peer networks? Possible Applications

9. Selected Error-Resilient Control Variables • The number of voters • The number of choices • The number of dimensions on which the choices are rated • Voter preference distribution (including rating scale) • Voter competence (reliability) distribution including • False negatives • False positives • Competence (reliability) weighting rules • Voting system • Method for expressing preferences (e.g., One Voter, One Vote (OVOV) ) • Aggregation rule (e.g., plurality) • Tie-breaking: none, random or optimized

10. What Error-Resilient Decision System Support Can Do for Distributed Decision Making 1. Optimize the number of decision makers to produce an inference that has a probability of greater than 0.9 of being correct, 2. Exceed requirements for detecting threats and taking action (in data fusion centers) by controlling false positive and false negatives, 3. Deliver guidance about the benefits of collecting more data or waiting longer before making a risky choice

11. Each voter casts one vote for their top-ranked choice 1000 Voters 100 Voters Monte Carlo methods: 40,000 random samples 10 Voters Control 1: Increase the number of voters Decision Task 5 choices Increasing the number of voters can boost error-resilient efficiency in binary choices.

12. The Condorcet “Jury Theorem”

13. HOW MUCH INFORMATION IS NEEDED? Same conditions with heterogeneous (diverse) preferences => Copeland voting is optimal. Control 2: Shape the distribution of preferences MONTE CARLO SIMULATION RESULTS FOR 100 VOTERS Homogeneous (similar) preferences (below—see Input Details) => OPOV is optimal Example: In Homogeneous Voter Cultures, with OPOV, only 22% of the voters need to report before a decision (inference) can be made with 90% confidence. In Heterogeneous Voter Cultures, with Copeland Voting, 70% of the sensors must report to achieve the same level of confidence.

14. Same conditions with heterogeneous (diverse) preferences => Copelandvoting is optimal. Homogeneous (similar) preferences => OPOV is optimal Control 3: Design multiple factors for a mixed voter culture 100 voters: 75 Voters with homogeneous or heterogeneous preferences and .9 mean competence, 25 voters with homogeneous or heterogeneous preferences and .48 mean competence, False Positive Rate = 0.01, False Negative Rate = 0.01, Non-linear weighting. Ties randomly broken. HOW MUCH TIME IS NEEDED (Rayleigh time)? Example: In Homogeneous Voter Cultures, using OPOV, 160 seconds must elapse before a 90% decision can be made. In Heterogeneous Voter Cultures, using Copeland Voting, 410 seconds must elapse before achieving the same 90% confidence level.

15. Without Decision System Support • Waiting exacerbates uncertainty • Is it message delay? • Is there a network communications failure? • Did the decision makers make a choice? • Uncertainty and complexity • Make it difficult to identify risks that seem extreme, but aren’t • Mask the dangers of intuitively plausible, but disastrous risks

16. Receiver Operating Characteristics (ROC) Curves • Parameterizing of ROC curve: confidence amount for 50% TPF and 50%FPF and TPF = FPF as decision helper. • Proving the effect of error-resilience on changes of ROC parameters • Further demonstration of integration of multiple platforms with better ROC behavior

17. Research Opportunities Complex Decision Tasks • Collaborative work and assessment in semi-structured evaluation of design • Ad hoc polling • Client-server • Peer-to-Peer • Improve data quality • Multidimensional assessments • Risky conditions

18. Research Opportunities[continued] Data Fusion Centers • Overcome delay and decision making error • Optimize • Design and deployment of decision makers • Data quality (e.g., biometric fusion) • Minimize false positives, false negatives • Response time in emergencies

19. Research Opportunities[continued] Dynamic Software Updating Timing for multi-threaded program updating - Iulian Neamtiu Michael Hicks Jeffrey S. Foster Polyvios Pratikakis “Contextual Effects for Version-Consistent Dynamic Software Updating and Safe Concurrent Programming,” Principles of Programming Languages,’08, January 7–12, 2007, San Francisco, California.

20. Research Opportunities[continued] Theoretical Issues • Monte Carlo studies of decision scenarios • ROC analyses • Find analytic solutions • Experimental analyses • Test predictions about the direction and magnitude of error-resilient patterns • Identify individual and organizational factors that explain deviations from theoretical predictions