Pre-Election Testing and Post-Election Audit of Optical Scan Voting Terminal Memory Cards

1. Pre-Election Testingand Post-Election Audit of Optical Scan Voting Terminal Memory Cards Voting Technology Research (VoTeR) Center Department of Computer Science and Engineering University of Connecticut http://voter.engr.uconn.edu Seda Davtyan, Sotiris Kentros, Aggelos Kiayias, Laurent Michel, Nicolas Nicolaou, Alexander Russell, Narasimha Shashidhar, Andrew See, Alexander A. Shvartsman Work funded by the Connecticut Secretary of the State Office

2. Outline • Motivation • Introduction • Goals of the Memory Card Audit • AccuVote OS • AV-OS Software Components • Auditing Process • Results and Observations • Conclusion 2

3. Motivation • In a recent primary in an unnamed state there was a mix of hand-counted and machine-counted precincts • It was observed that in hand-counted precinct Candidate A was favored by the voters, while in optical-scan tabulated precincts Candidate B was favored • There were sensible demographic reasons for this • Nevertheless, a valid question was asked: Were the voting machines programmed correctly? • The state officials did not have an answer 3

4. Motivation • The machine in question is Premier’s Accu-Vote Optical Scan tabulator • Provides inherent VVPB/VVPAT • Not the “bleeding edge” machine – relatively few attack vectors • But: • [Hursti’05] Memory cards are easy to tamper with if removed from the tabulator • [EVT’07] Memory cards are easy to tamper with if sealed in the tabulator • Reports by other workers and CA, CT, FL, AL,… • Tests/audits of equipment/technology are necessary 4

5. AccuVote OS (AV-OS) • AV-OS Firmware version 1.96.6 • Memory cards programmed on GEMS 5

6. Process in Connecticut Ballot information for a district Memory cards programmed using GEMS (at LHS Associates) Cards shipped Cards used in the election at the district Cards inserted and tested at the district

7. Goals of the Memory Card Audit • Pre-election Memory Card Audit • Perform an integrity check of the contents of the memory cards • Post-election Memory Card Audit • Integrity check of contents • State of cards consistent with election use 7

8. Outline • Motivation • Introduction • Goals of the Memory Card Audit • AccuVote OS • AV-OS Software Components • Auditing Process • Results and Observations • Conclusion 8

9. AV-OS Software Components • The behavior of AV-OS is determined by two components: • AV-OS Firmware • Data and program on Memory Card • Memory Card includes: • Status Information • Audit Log • Ballot Description • Counters • Bytecode 9

10. Outline • Motivation • Introduction • Goals of the Memory Card Audit • AccuVote OS • AV-OS Software Components • Auditing Process • Results and Observations • Conclusion 10

11. Auditing Process • Preparation for audit • Analysis of the AV-OS firmware, development of custom firmware, a data collection and comparison tool, and analysis of the bytecode • The auditing process • Data collection from memory cards • Analysis of the data 11

12. Contractual Issues • Contract between Premier and State of CT • Prohibits “reverse engineering”, “de-compilation”, “re-assembly”, etc. • One exception: Contract permits modification/alteration of software/firmware to “display” data “related to election results” • We used this exception to perform engineering to understand the format of memory cards and to extract this data using special purpose firmware we designed 12

13. Custom Firmware • Custom firmware was developed to resolve major issues in using the built-in dumping procedure of AV-OS: • Relying on the undocumented built-in procedure is questionable • Avoid altering card contents (audit log) • Ensure faithful reading of contents • Speeding up memory card dumping 13

14. Custom Firmware Development • Four main point were considered during the production of new firmware: • Memory Card Access • Serial Port Access • Delivery of the Memory Card data • Avoid any logging on the memory card (Technical details in the full paper) 14

15. Format of the Memory Card • Epson 128K card • Our analysis revealed the following formatting of the memory cards 15

16. Data Collection Tool • The Data Collection/Comparison tool serves two purposes: • Collecting the memory card dump sent using run length encoding • Auditing the collected data by comparing baseline and audit data and analyzing the differences 16

17. Testing Methodology • Testing for potential data inconsistencies and integrity problems of the memory cards requires collection of three types of data: • Baseline Data • Pre-Election Data • Post-Election Data 17

18. State of the Memory Card • Memory card examination focused on: • Card Format (data and byte code) • Card Status (set for election, etc.) • Counter Status (zero / non-zero) • Election Count (usage) • Audit Log 18

19. State Diagram • State transitions for a memory card 19

20. Outline • Motivation • Introduction • Goals of the Memory Card Audit • AccuVote OS • AV-OS Software Components • Auditing Process • Results and Observations • Conclusion 20

21. Results and Observations • Pre-election audit performed on 522 memory cards • Covers 75% of all districts • 378 out of 522 memory cards were received prior to the election, the rest later • Post-election audit was performed on 100 cards • Partial audit en route to future broader audits • 36 out of 100 memory cards were used during the election • Represents > 5% of the cards used in election 21

22. Pre-Election Sampling Issues • A few differences between the procedures followed by the poll workers and the procedures defined by SOTS were noticed: • The cards were not chosen uniformly at random for the audit • Instead of choosing random memory cards for each district random districts were chosen • Some cards were labeled “backup” 22

23. Pre-Election Memory Card Audit Results 23

24. Post-Election Memory Card Audit Results 24

25. Conclusions • The following were identified during the memory card audit • Examination of memory cards revealed no incorrect ballot data or bytecode • Poll workers did not follow the exact testing procedures • Surprising number of cards with “junk data”: 3.5% in pre-election audit and 8% in post-election audit 25

26. References • Black Box Voting http://blackboxvoting.org • Jonathan Bannet, David W. Price, Algis Rudys, Justin Singer, Dan S. Wallach: Hack-a-Vote: Security Issues with Electronic Voting Systems. IEEE Security & Privacy 2(1): 32-37 (2004) • Help America Vote Act (HAVA), http://www.fec. gov/hava/law_ext.txt • Harri Hursti, Critical Security Issues with Diebold Optical Scan Design, Black Box Voting Project, July 4, 2005 http://www.blackboxvoting.org/BBVreport.pdf • A. Kiayias, L. Mchel, A. Russell, A.A. Shvartsman, M. Korman, A. See, N. Shashidhar and D. Walluck, Security Assessment of the Diebold Optical Scan Voting Terminal, http://voter.engr.uconn.edu/ voter/Report-OS.html • A. Kiayias, L. Michel, A. Russell, N. Sashidar, A. See, and A. Shvartsman, An Authentication and Ballot Layout Attack Against an Optical Scan Voting Terminal. 2007 USENIX/ACCURATE Electronic Voting Technology Workshop (EVT 07), Augist, 2007, Boston, MA. • A. Kiayias, L. Michel, A. Russel, N. Sashidar, A. See, A. Shvartsman, S. Davtyan. Tampering with Special Purpose Trusted Computing Devices: A Case Study in Optical Optical Scan E-Voting. Twenty-Third Annual Computer Security Applications Conference (ACSAC), December, 2007, Miami Beach, Fl. 26

27. About the UConnVoTeR Center • Participation in Connecticut Voting Technology Standards Board 2005-2006 • Relationship with the CT SOTS Office • Advising on voting technology issues • Evaluation of proposed voting equipment • Development of safe use procedures • Technology audits and security analysis • Faculty: A. Shvartsman, A. Kiayias, L. Michel, A. Russell • Research Assistants: S. Davtyan, S. Kentros, N. Nicolaou, N. Sashidhar, A. See 27