P1622 Audit Use Case

1. P1622 Audit Use Case Neal McBurnett IEEE P1622 meeting 2011-02-08

2. Audit:Compare system's reported results with the evidenceTake sample of detailed results by audit unitand compare to hand counts of those audit units

3. Report of Vote Counts by Audit Unit

4. Recent progress in Auditing Growing momentum behind good audits Law in Colorado: risk limiting audits by 2014 New law in CA to pilot risk-limiting audits NM: state-wide audit starting 2010 EAC audit grant funding opportunity 2010-2012; notes risk-limiting small batch audits

5. Principles and Best Practices forPost-Election Audits (2008)http://electionaudits.org/principlesLeague of Women VotersElection Audit Report (2009)American Statistical Association 2010‏ statement on Risk-Limiting Audits and small-batch audit reporting

6. Typically, amount of work required to perform a risk-limiting vote-tabulation audit is roughly proportional to the batch sizes, with no reduction in statistical efficacy.

7. State-wide coordination of audits Auditable tabulation reports from counties to state Audit unit selections from state back to counties Audit discrepancy reports back to state Potential for additional selections Final audit report aggregates discrepancies

8. Audit results also need to be published in a standard formatInclude hand counts and discrepancies for all units audited, whether they were targeted or randomly selected, information needed to validate the selections (e.g. algorithms used, random seeds used as input) , results of the audit

9. Audit unit result Audit unit id For each candidate or “under” or “over” System tally Hand count (multiple?)‏ Discrepancies (Hand count - system)‏ Resolution code Resolution text description

10. Proof of Concept:ElectionAudits Audit Management SystemOpen source, MIT licenseSupports the Best Practices and risk-limiting auditsImports standard election report file, e.g. Hart, Sequoia; EML 510 supportAutomates many steps of the auditEnter the data, publish with statisticsRoll 15 dice, publish all the selections

11. Biggest challenge: getting small-batch data out of election systems Want Interoperable systems, out-of-the-boxNot just Integratable via consultantsSee David Flater's CDF presentation

12. Summary Proper audits are critical Risk-limiting audits in CO, CA law Small-batch reports help enormously Coordinated state-wide audits: lots of data flow Standard, machine-readable format is needed EML looks appropriate