220 likes | 360 Views
Keystroke Biometric & Stylometry Systems. Teams 2 & 4 The Michael L. Gargano 9th Annual Research Day Presentation Presenters Edyta Zych & Vinnie Monaco. May 6, 2011 Seidenberg School of Computer Science and Information Systems Pace University, Graduate Center White Plains, New York.
E N D
Keystroke Biometric & Stylometry Systems Teams 2 & 4The Michael L. Gargano 9th AnnualResearch Day PresentationPresentersEdyta Zych & Vinnie Monaco May 6, 2011 Seidenberg School of Computer Science and Information Systems Pace University, Graduate Center White Plains, New York
Agenda • Team and Project Leader Introductions • KBS & Stylometry Projects Overview • Project Specifications & Deliverables • System Components & Enhancements • Results & Conclusions • Future Work
Project Stakeholders Keystroke Biometric Stylometry Team Members • Vinnie Monaco • Tyrone Allman • Mino Lamrabat • Mandar Manohar Customers / SMEs • Dr. Tappert • John Stewart • Robert Zack Team Members • Edyta Zych • Omar Canales • Vinnie Monaco • Thomas Murphy Customers / SMEs • Dr. Tappert • John Stewart
Two Projects Act As One, Two Team Leads • Person Manager • Facilitate Weekly Meeting Schedule • Task Assignments • Driving Everyday Activities • Tech Training & Documentation • Technical Manager • Subject Matter Expert (SME) • Technical Scope • Design & Implementation of all System Enhancements • Programming Tasks
Overview: Keystroke Biometric System • Pace University has conducted over 8 years of research on Keystroke Biometrics • The Keystroke Biometric System (KBS) can be used for both identifying and authenticating users from their typing rhythms • Keystroke dynamics are the patterns of rhythm and timing created when a person types, including: • Overall speed • Variations of speed moving between specific keys • Common errors • The length of time that keys are depressed (duration) • This semester’s work focuses solely on the KBS as it relaters to an online test taking environment
Overview: Stylometry • Stylometry is the study of the unique linguistic styles and writing behaviors of individuals in order to determine authorship • Stylometry uses statistical pattern recognition, and artificial intelligence techniques • Stylometry features typically used to analyze text include word frequencies and identifying patterns in common parts of speech • This semester’s work focuses on text input being used in conjunction with the keystroke analysis to improve authentication results including • Determining authorship in documents (Beneficial academically to assist with on-line test taking) • Protecting against plagiarism through a third party
Project Specifications • Work closely with our project customer to define the most appropriate Keystroke & Stylometry Features and add additional features to assist in validating/authenticating the identity of students taking an online exam • Extract the selected Feature Set for Keystroke Biometric and Stylometry Analysis and run experiments to measure program performance utilizing the enhanced systems: Input System, Feature Extractor and Classifier • Run experiments and tests on the data collected to support the identification of subject and online test-taker authorship
KBS Project Deliverables Stylometry Input System Feature Extractor Classifier Input System Feature Extractor
Overview of System Components • Input System • Captures keystroke and stylometry data in an online test format • Feature Extractor • Measures raw data to obtain a feature vector for each sample • Classifier • Uses feature vectors to test authentication
Input System Enhancements • Upgraded from a Java Applet to a standalone java program. • Implemented a user management system to simulate an online test taking environment • Change to test taking format, instead of free text or copying tasks • Moved to a more general XML data format, to handle both keystroke and stylometry data • More restrictions in place on how users interact with the system • Disable cut/copy/paste ability • Users must complete the test in full • Capture and log keystrokes from every successful login attempt
Feature Extraction Enhancements • Feature extraction implemented in the functional language Clojure • Easy integration with Java front end • Better data handling, filtering, and mapping capabilities • New Normalization method tested • Old formula • New formula • Improved outlier removal • Integrated stylometry and keystroke features
Benchmark Results: 18 subjects, 180 samples Before After
Normalization Results on Benchmark Data Good Bad Still OK
Analysis / Results • 40 students, 10 samples each from 1 test • Weak training • Keystroke and Stylometry biometrics
Analysis / Results • 38 students, 20 samples from 2 tests • Strong training • Stylometry biometrics FAR (%) FRR (%)
Keystroke Combined Data • 38 students, 20 samples each from 2 tests • Weak training • ~11% equal error rate • 38 students, 20 samples each from 2 tests • 2 samples combined yielding 10 samples each • Weak training • ~5% equal error rate 20 20 FAR (%) FAR (%) 0 0 100 100 FRR (%) FRR (%)
Keystroke vs. Stylometry ROC Curve • 38 students, 10 samples from 2 tests • Weak training • No equal error rate for stylometry
Stylometry Combined Data • 40 students, 10 samples each from 1 test • No equal error rate • 30 students, 30 samples each from 3 tests • 6 samples combined yielding 5 samples each • ~30% equal error rate 40 60 FAR (%) FAR (%) 0 0 100 100 FRR (%) FRR (%)
Stylometry Combined Data 24 students, 10 samples CombinedWeak Training • Authenticating students • ~32% equal error rate • Authenticating test • ~35% equal error rate 100 100 FAR (%) FAR (%) 0 0 100 100 FRR (%) FRR (%)
Future Work Keystroke and Stylometry Biometrics • Improve stylometry authentication results by identifying important features • Combined more samples to obtain stylometry features on longer text input • Determine if samples may be authenticated to a test, as opposed to the individual Data Collection • Modify the input system to eliminate some problems with giving an online test • Authenticate with first/last name only • Ability to traverse the questions in the test • Integrate keystroke authentication with users logging into the system
THANK YOU! Teams 2 & 4Keystroke Biometric& Stylometry Systems • Tyrone Allman, Omar Canales • Mino Lamrabat, Mandar Manohar • Vinnie Monaco, Thomas Murphy • John Stewart, Dr. Charles Tappert • Robert Zack, Edyta Zych