370 likes | 519 Views
Mobile Dictation With Automatic Speech Recognition for Healthcare Purposes. Tuuli Keskinen, Aleksi Melto, Jaakko Hakulinen , Markku Turunen, Santeri Saarinen, Tamás Pallos TAUCHI research center, School of Information Sciences, University of Tampere, Finland
E N D
Mobile DictationWith AutomaticSpeechRecognition for Healthcare Purposes Tuuli Keskinen, Aleksi Melto, Jaakko Hakulinen, Markku Turunen, Santeri Saarinen, TamásPallos TAUCHI research center, School of Information Sciences, University of Tampere, Finland Riitta Danielsson-Ojala, Sanna Salanterä Department of Nursing Sciences, Faculty of Medicine, University of Turku, Finland Kites symposium 2013
Content • Background & Motivation • Dictationapplication • User evaluation • Results • Discussion and conclusion • Endingwords
Content • Background & Motivation • Dictationapplication • User evaluation • Results • Discussion and conclusion • Endingwords
Background • Firstspeechrecognitionsystems for medicalreportingweredevelopedover20 yearsago[1] • Doctors’ dictationsarestillcommonlytypedmanually, bututilization of speechrecognition is increasingespecially in radiologyand pathology • Nurses’ use of speechrecognition is rare and oftenlimited to filling the templates [X] Numbersrefer to the actualreferences in the paper.
Background • Utilizingspeechrecognition in Finnishhealthcarestudied, e.g., in [2] whereradiologistswerefollowedchangingfromcassette-basedrecording to speechrecognitionbaseddictating • Severalstudies in the area of speechrecognition in healthcaredone, e.g., [1, 3, 4, 5] • Previousstudiesfocusmainly on objectivequalities, such as dictationdurations and recognitionerrorrates
Motivation ”Voi kun meilläolisimahdollisuus saneluun!” - Anonyymi YTHS:nsairaanhoitaja
Motivation for ourstudy • Paucity of utilizingspeechrecognition in Finnishhealhcare, especially in nursing • Obvious and unnecessarydelays in gettingpatientinformation to the nexttreatmentsteps • Lack of researchfocusing on the userexpectations and experiences of dictationapplicationsutilizingspeechrecognition in healthcare
Content • Background & Motivation • Dictationapplication • User evaluation • Results • Discussion and conclusion • Endingwords
Dictation application • Based on ”MobiDic” by Turunen et al. [6] • The mobile client (Androidapplicationon a tablet) includesfunctionality for recording and editingdictations, and modifying the dictationtexts • The server side manages the dictations (audio and text) and communicates with speechrecognitionengines and M-Filesdocument management system
Dictation application • Notonlyspeechrecognition is utilized, but a variety of othertools is included to improveresults: • State of the artnaturalprocessingtools (e.g., spelling and grammarchecking) • Statisticsbased on useractions • Optimizedmultimodaltouch-screen U • Distributed applicationmodelmakes a variety of usecasespossible: • Real-timedistributedassisteddictation • Workflow management • Plug-and-play component management (e.g., speechrecognizer, NLP tools, document management) • UI canbeadapted for differentusagecases and devices
Content • Background & Motivation • Dictationapplication • User evaluation • Results • Discussion and conclusion • Endingwords
User evaluation • Real-worldcontext, realusersand realdictations • Twowoundcarenurses in one of the UniversityHospitals in Finland • Lastedthreemonths in total, covering 30 and 67 dictations for the participants • Wizard-of-Ozapproach • The medicallanguagemodelavailablewasbased on medical and nursingdocumentation, and thus, itwasnotsufficient to recognize the languageusedby the woundcarenurses
Methodology • Backgroundinterview • Main focus on participants’ normalpractices on makingand/ordictatingpatiententries • Subjective data gathered with questionnaires • User expectations and experiences (SUXES [8]) • Usability-relatedexperiences (SUS [9]) • Open questions • Log data • Allapplication and servereventslogged
SUXES method • Enablescomparisonbetweenuserexpectationsbefore the usage and userexperiencesafter the usage on a set of statements • Expectations reported by giving two values • acceptable level: the lowest acceptable quality level for even using the system (or property) • desired level: the uppermost level that can even be expected of the system (or property) • Experiencesreported by giving a single value on the samestatements • Expectationsform a gapwhere the experiencedlevel is usuallyexpected to be • Ifbelow something is wrong; Ifabove success
SUXES method • Expectations • Experiences • Comparison Low High x x Using the phone is fast. x Using the phone is fast. Using the phone is fast.
SUXES method • Expectations • Experiences • Comparison Low High x x Using the phone is fast. x Using the phone is fast. Using the phone is fast.
SUXES method • Expectations • Experiences • Comparison Low High x x Using the phone is fast. x Using the phone is fast. Using the phone is fast.
Expectations and experiences • Weused the nineoriginalstatements of SUXES • speed, pleasantness, clearness, error free use, error free function, learning curve, naturalness, usefulness, and future use • …and fiveadditionalstatementscomparing the dictationapplication to the normallyusedentrypractice • faster, more pleasant, more clear, easier, and prefer in the future
Content • Background & Motivation • Dictationapplication • User evaluation • Results • Discussion and conclusion • Endingwords
User expectations on the application Median responses of acceptable – desiredlevels(greyareas), n=2.
User experiences on the application Median responses of acceptable – desiredlevels (greyareas) and experiences (blackcircles), n=2. P1 and P2 refer to participant 1 and 2.
User expectationscompared to normalentrypractice Median responses of acceptable – desiredlevels(greyareas), n=2.
User experiencescompared to normalentrypractice Median responses of acceptable – desiredlevels (greyareas) and experiences (blackcircles), n=2.
Content • Background & Motivation • Dictationapplication • User evaluation • Results • Discussion and conclusion • Endingwords
Discussion • The desired level was 6 or 7 on all statements • The experiencedlevelwas at least 6 on allbutonestatements • The usefulness of the dictationapplicationcanclearlybeseen in the results • Moreimportantly, the participantswouldpreferusing the application in the future, i.e., theywouldbeready to droptheirfamiliar and saferoutines
Conclusion • Due to nothaving an accurateenoughlanguagemodel for nurses’ purposes, weused a Wizard-of-Ozscenario to finalize the speechrecognitionresults • The userexperienceresults show a truepotential for ourdictationapplication – notonly to smoothendictationprocess, but as a relevant option for writing the nursingentries
Futurework • Finalizing a languagemodel for nurses and utilizingit in Finnishhealthcare to enabletotallyautomaticdictation-to-textprocess is crucial • Wearenotdeveloping the languagemodels by ourselves, butwillbe in closecollaboration with ourpartnersin the development and evaluation • Wearealsodevelopingourapplicationfurther to provideevenmorepleasurableuserexperience and seamlessprocess
FutureWork • In order to makethisreality, weneed a properprocess for iterativedeployment: not a stand-aloneproductwhichcanbesold to hospitals, for example • Wehavedevelopedallnecessarycomponents: client and backend software, connections to 3rd party components, tools to supportdeployment, and a completedeploymentprocess • Ready for commercialization – looking for partners!
Acknowledgements • Project ”Mobile and UbiquitousDictation and Communication Application for MedicalPurposes” (”MOBSTER”) • Fundedby the FinnishAgency for Technology and Innovation (TEKES) • Lingsoft and M-Files, and otherprojectpartners