Voice Controlled Surgical Assistant ECE 7995 Dupindar ghotra, Muhammad Syed, Sam li, Sophia lei

1. Voice Controlled Surgical Assistant ECE 7995 Dupindar ghotra, Muhammad Syed, Sam li, Sophia lei

2. Outline Significance Quick recap Overview of system In-depth analysis Testing and Validation Limitations Competitive technology Future modifications and goals Timeline and budget

3. Significance • Why is there a need for this kind of robot? Machine does the repetitive tasks Nurse for patient care Efficiency Cost • Any similar work? Penelope - Columbia University Tumor in forearm

4. Significance • Leads to answer another question Would this be accepted by medical people? • Successful field study has already been performed • What is the need for carrying on something that is already done?

5. Recap – Objectives Objective1: Develop a robot as an independent assistant Objective2: Voice integrated system; delivers and retrieves tools Objective3: Safe, efficient, accurate & cost-effective

6. Overview of the system

7. Working Software W1: Voice recognition W2: Evaluation through pc W3: Respective transmission to microprocessor W4: Evaluation by microprocessor W5: Final command to the robot Hardware

8. Working and Challenges • W1: Voice Recognition How is it being done? First thought – Build a voice recognition circuit Instead - Microsoft speech SDK Cost and effectiveness Libraries and tools to develop programs in Windows

9. Working and Challenges • Problems related to recognizing voice - Male / Female - Pronunciation Can be reduced/eliminated by training the system more - Loudness microphones designed specifically for voice recognition - Surrounding noise

10. Working and Challanges Voce input taken. What do we do now? • W2: Evaluation through pc Initial Approach –Have words typed in notepad Text file as input Difficulty – Unable to find a method to have text file as input in real-time Visual Basic • W3: Respective transmission to microprocessor Validation of data being sent at this step

11. Working and Challanges • W4&5: Evaluation of code received by microprocessor Validation: storing the received data in a cleared memory space Knowing what the expected Hex value to be received is Comparing the stored values to expected numbers Final Command sent to the robot’s arm

12. Microcontroller & Programming • Freescale microcontroller chip (HCS12 family) • serial communication interfaces (SCIs) channels (RS232) • Forth programming language flexibility and efficiency • SwiftX is a interactive cross-compiler for embedded microprocessors and microcontrollers.

13. Working and Challanges SCI1.Available 1 >= if \ has data in buffer 0 tr.sci1 $65 = if (S1-Key) drop (S1-Key) CASE 53 OF po.1 ENDOF 54 OF po.s ENDOF 55 OF po.c ENDOF ENDCASE ASCII Voice Chr Dec Up 1 49 Down 2 50 scissor a 97 knife b 98 : po.s 3500 PWMDTY2 ! 1700 PWMDTY0 ! 2400 PWMDTY4 ! WAIT 1000 MS 4450 PWMDTY2 ! M.on Pulse-Width Modulator (PWM) Servo Direction & Operation Angle

14. Robotic Arm • 3 Degree of freedom • Standard size servo motor • Rotate up to180 degrees • Offer a 417oz./in. (5v) of torque Aluminum Tubing

15. Electromagnet as a Gripper Using a LED to indicate the Electromagnet ON/OFF Holding Value: 2 lb Volts: DC 12 Max. Watts: 1.4 Amps: 0.12 Net Weight: 2.1 oz.

16. Using a MOSFET as a Switch

17. Testing and Validation • Rather simple method for testing and validating the system • Different tools were asked for • Validated by observing the right action being performed - right tool taken / put back

18. Timeline

19. Budget

20. Current Limitations • Motors Torque Range of motion • Electromagnet Weight limitation • Voice recognition surrounding noise • Further testing and analyzing of system various tools used in surgeries

21. Limitations and Questions Considered • Open-loop system!! • No feedback taken from the motors • What if the tool drops because of some reason on its way? • What If someone hits the tray carrying tools and they move from their position?

22. Limitations and Questions considered • What if the tool used in a surgery is non-magnetic !#? • How does the system know what tool is being placed back? Can it recognize it?

23. Future Goals - Improvements • Objective1 – A thorough research on what different types of tools (size, shape, weight) are used in surgeries • Objective2 – Have continuous feedback to the system

24. Future Goals - Improvements • Objective3 – A better gripper • Objective4 – Integrate with image guided system • Objective5 – A track of tools what it started with & what it is ending with

25. System Analysis after modifications • System is getting a feedback letting it know whether tool is still in gripper • Be able to grab non-magnetic tools • Would know what tools are available to it, where each one is and what tool is being used at a certain time

26. New Timeline May – July 2006 • Objective1: Research on different types of tools • Objective2: continuous feedback to the system • Objective3: better gripper August – November 2006 • Objective4: Image guidance • Objective5: Track of tools

27. Questions??