1 / 22

Live Captioning with Dragon

Live Captioning with Dragon. Live Captioning with Dragon. SETBC project, to provide students with speech to text input from their teacher, using Dragon Naturally Speaking. Key Project Contacts: Karen Bell, Support Teacher Jill McCullum, District Hearing Resource Teacher.

leah-joyner
Download Presentation

Live Captioning with Dragon

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Live Captioning with Dragon

  2. Live Captioning with Dragon SETBC project, to provide students with speech to text input from their teacher, using Dragon Naturally Speaking Key Project Contacts: Karen Bell, Support Teacher Jill McCullum, District Hearing Resource Teacher

  3. Meeting of the Minds Karen Bell,Support Teacher at SMS Jill McCullum,District Hearing Resource Teacher SD67 Jason Corday, Principal of SMS Dan Paccioretti, Audiologist, Phonak Canada Anita Toneatto, Distict Technology Helping Teacher Carla Norheim, Hearing Resource Teacher SD23 Holly Colquhoun, Teacher at SMS Sandra Cureatz, OCF/CEA Erica McDowell, OCF/CEA • During a meeting, September 13th 2012, the idea of providing closed captioning on a laptop for our hard of hearing students, evolved from discussion re: using a Smartboard or T.V. screen. The grade 7 classroom was chosen because: • Teacher was tech savvy • Deaf and Hard of Hearing CI student is very bright • SET-BC student was already in this classroom • Experienced OCF’s available (Our project has evolved several times since the initial meeting)

  4. Proposal The project will be successful if: • By the end of the school year the students are able to use the technology independently – take notes and organize the material from lectures. To make the classroom teacher’s auditory message a visual message, thereby assisting students with: • memory issues • motor issues • written output issues • organization issues • compromised audition • processing speed • limited English speaking capabilities

  5. Proposal Summary con’t When the teacher is delivering instruction, he will use: • Dragon, the school’s infrared sound field system and Phonak FM technology to create captions on the student’s laptops.  • This will allow the student to refer back to the teacher’s oral instructions/directions for:  assignments, clarification, review and vocabulary development.  • The lecture can then be converted into Kurzweil and highlighted with the assistance of the classroom CEA.  • The notes can then be extracted and saved to a file under the course.  Later the student can have the notes read back to him with Kurzweil if he/she decides for studying.

  6. Equipment Existing Equipment: Utilized existing classroom infrared sound field system (Frontrow to Go) , we also tried with a Simeon 500 WU. The following pieces of hardware and software have been requested, approved and readied for the project: • 3 laptops – 2 are older Toshiba Satellites and 1 is a Toshiba Tecra • 3 Phonak MyLink receivers • Inspiro Premium Transmitter – teacher • Easy Boom microphone – teacher • Microsoft Office • Antivirus software • Digi-Master X was also setup but not essential to the project • Dragon Naturally Speaking and Kurzweil were already loaded on all 3 laptops

  7. FM System Cost

  8. Technology Setup The MyLink receivers were connected to all three student laptops. The teacher was fitted with the boom microphone and the Inspiro Premium Transmitter, clipped to his belt.

  9. Technology Setup con’t • After ensuring that all devices were connected and updated, Dragon was opened on one of the laptops to “train” the program.  Then after the training was complete the teacher’s and OCF’s profiles were copied and transferred to the other student laptops.  • As the teacher spoke, the text appeared on all three laptops with very few errors.  It was discovered that the MyLink receivers work best if hung around the student’s necks rather than on the desk beside the laptops. • When the teacher was finished, the documents were saved as .rtf files so they could be used in Kurzweil, Office etc. • As a bonus, the DigiMaster X is transmitting the teacher’s voice through the classroom’s Front Row speaker system which allows other students in the room to benefit from the clear consistent sound of the teacher’s voice. Kathy Ryan (SETBC) provides guidance for Dragon Speak voice profiling to the team OCF prepares voice profile for Dragon and Tech. department sets up the laptops.

  10. Responsibilities Teacher Responsibilities • Prepare Dragon profile – initially this takes about 15 minutes.  A key to doing this is to create the profile with the microphone that you will be using. • Become familiar with Dragons features • Become familiar with editing terminology for Dragon for continual training • Be flexible in having different people visiting their classroom to view the project in place • Student Responsibilities • Each morning have computer out and safely positioned on desk top • Have Dragonpad running, ready for teacher’s dictation • Ensure computers and MyLink equipment is recharged daily • Training on Kurzweil • Save documents as .rtf files • Be able to transfer documents to Kurzweil program • OCF’s Responsibilities • Prepare Dragon profile to use if teacher is away • Supervise the student’s day to day routines – ensuring computers get plugged in etc. • Label/personalize all equipment • Charge and safely store away the Inspiro Transmitter • Train using Kurzweil to assist students

  11. Use of Dragon NaturallySpeaking for Real-Time Transcription for the Deaf or Hard of Hearing TEACHER MODEL: • Teacher has a microphone that is broadcasting to a computer which has Dragon NaturallySpeaking running which is doing the real-time transcription. • the throughput accuracy will vary greatly depending on the speaker, the environment, and the subject being discussed. * (This was the first method that we tried in the classroom, which later evolved into our next method)

  12. The Beginning… • Things we are doing right: •  The teacher that we chose is comfortable with technology, his voice is clear/articulate and he often teaches lecture style. •  There is a full time CEA in the classroom who is an Oral Communication Facilitator and is used to working with sound equipment. •  The three students are sitting in different areas of the classroom. It doesn’t seem to affect the accuracy but there is a slight lag time. •  We have an Audiologist, District Hearing Resource Teacher, District Technology team, partnered with SETBC and with Phonak Canada. • One of the students involved in the project asked if he could a write a blog. The blog has been created and is linked to this page.

  13. Frustrations • When we did the initial training with Dragon, it was 98% accurate. As the teacher used the program he could not see the errors and therefore did not error correct. As the weeks went by, the accuracy dropped to 70% and then down to 10%. • We learned that Dragon only works as well as we have trained it. Without monitoring for errors and correcting, it is not accurate.  • When the teacher is putting the boom microphone on, it is critical to have the EasyBoom Microphone placed with the correct orientation to the mouth. • The teacher was using Dragon to teach Math while using the Smart Board. It was suggested to us that this would be problematic because he wouldn’t be using full sentences. We are learning that some subjects are more suited than others.  • Our student with the cochlear implant was not looking at the teacher and trying to get all of his information from the captioning. He was working twice as hard to get the information because the screen was not accurate. We told him that it was best to follow what the teacher was saying and refer to the captioning for missed information. • The laptops are powered on for the better part of the day. The older laptops battery life is not as long lasting. Each day the three laptops, Inspiro and MyLinks have to be recharged. The OCF takes care of this but we have realized that it takes a lot of counter space. We also need to keep the equipment secure.

  14. Frustrationscon’t • No Structure, words jumbled together • Students begin to ignore the screen out of frustration • The program wasn’t meeting their needs • Dragon losing accuracy Troubleshooting • Believed that our problems were cause by the software, Dragon Naturally Speaking • Teacher & OCF spent hours updating their profiles • Troubleshooting the Dragon • Testing all connections • Testing Boom mic in various settings (noise) • Testing other wireless mic’s with Dragon and Windows 7 Speech Text software Turning Point • Realized that the MyLink receiver was experiencing a lot of interference (loud static) and picking up voices from Digimaster down the hall. • Now OCF must check the receiver frequently

  15. Perspectives… Teacher Perspective: “The project has been both exciting and frustrating. As all projects with technology usually are. It was frustrating when the project was stalled by technology glitches which couldn't be resolved. It has also been surprising to find out how much of instruction is conversational based.” OCF Perspective: “I completely agree with the classroom teacher, that this project has had it’s ups and downs. It’s a lot to learn, in regards to the technology and the software. Ideally, the project should run smoothly but has encountered its share of problems. It’s also evolved several times to meet the needs of our students and the classroom but I’m still excited to move forward with it!”

  16. Perspectives… • Student Perspectives: • “When I first heard of this project, I got excited, but lead to some little disappointing things. The accuracy was slow and way off. The teacher says one thing, the laptop says another. Its really confusing, and misleading. This is a little disappointing with all the stuff I have to watch and do to keep up with the class, the laptop makes it worse.” -A.W • “I may find it nice, but its distracting in the class. I find the laptop useful, but not the dragon. It makes way to much mistakes.” -G.M • “Right now, its making a lot of mistakes, but I know it will get better. I have used dragon before.” -E.S

  17. New Vision…. It occurred to us that when we were using the Teacher’s voice this placed too many demands on the student participants. They were stressed at the amount of visual data and Dragon inaccuracies. As the teacher instructed, he often included irrelevant information, and didn’t always speak in sentences. Our students are still learning to access real time closed captioning. They do not have the capacity to extract the important information from the lecture. They are in grade 7. We have recognized that we set the bar too high.

  18. Use of Dragon NaturallySpeaking for Real-Time Transcription for the Deaf or Hard of Hearing OCF MODEL: • There is a OCF (redictator) is listening to the teacher/speaker and is dictating their spoken words which broadcasts to the computer and into Dragon NaturallySpeaking. • the OCF (redictator) can be skilled at simultaneous dictation (extracting the main points of a lesson) which can result in much higher accuracy. * (Current model)

  19. New Vision con’t…. Remote Re-dictator • In this model the OCF is skilled in simultaneous re-dictation which can be captured and transcribed onto the students’ PC… (Note, “should have requested an additional computer for the OI.”so that the OI can check the dictation for accuracy) This digresses from our previous focus to have the Classroom Teacher be the sole conveyer of the message. The OCF will extract the significant points from the lesson and use Dragon to deliver the main points to the students’ laptops. • There are many features that recommend this model, especially for the Deaf and Hard of Hearing. However, it demands access to a classroom support person with competency in Dragon software. • We will try to dovetail our project with technology being used at the post secondary level (University Campus). It’s important to know if we are accurately mirroring what might be anticipated at university. Educating our students to utilize a closed captioning visual stimulus is part of our goal.   • ReDictator to be capturing and transcribing classroom directions, discussion and question and answer sessions (keeping it brief and simple in the beginning, then building their content & commands as the OCF builds their skills) • By placing the ReDictator in the remote location there's no "distraction" for other members of the classroom. • There are quite a number of wireless technologies that can be used to broadcast the instructors dictation or capture a classroom discussion e.g. Dragon, Windows 7 Speech Recognition.

  20. Human Element: Journey Jill McCullum Student(s) Personal bias/self image/equipment Age/skill limitations (extracting notes, visual fatigue, set up on desk) Bigger picture vs. tunnel vision Patience or lack there of Anxiety…type ‘A’ personality, perfection/Parent-Guardian involvement. Team Time constraints…approx # hours to date: 100+ (training, equipment monitoring, tech meeting attendance, personal involvement on own time) Enthusiasm (team effort, great support from many people with vested interest e.g. • Technical staff SD#67/SET-BC • Admin • Teacher(s) • Support Staff-OI team (kudos) • Companies Phonak/FrontRow/Points West/SET-BC • Student participation (mixed emotions) At various points along the way we have all hit the wall Committed…absolutely Vision (future application) desire to save others money and time Conclusion: Not Yet….but Soon !!!!

  21. FurtherDiscussion…

  22. Thanks to… Team and Project Managers: Karen Bell and Jill McCullum School Team: Jon Broadbent Sandra Cureats Jason Corday Thora Delaquis Holly Colquhoun Erica McDowell Tech.Team: Danny Francisco Anita Toneatto Igor Pavlina SETBC Team: Kathy Ryan Flo Wong Company Contacts: Dan Paccioretti, Audiologist, Phonak Tracey Cochrane, Frontrow

More Related