Computer-Assisted Surgery Medical Robotics Medical Image Processing

1. Computer-Assisted Surgery Medical Robotics Medical Image Processing LECTURE 1 What‘s in a surgery Technical tools in CS CAS systems

2. PAST: Cut, then see

3. PRESENT: See, then cut Preoperative Imaging Intraoperative Execution

4. FUTURE: Combine, see, minimally cut Image guidance Augmented reality

5. How do surgeries proceed? • Diagnosis • based on physical exams, images, lab tests • Preoperative planning • determine the surgical approach • elaborate intraoperative plan (path, tools, implants) • Surgery • prepare patient and assess condition • acquire intraoperative images, adapt and execute plan • Postoperative follow-up • exams, lab tests, images to be corroborated

6. Treatment procedures • Invasive • neurosurgery: tumor removal • hear surgery: clogged arteries, transplants • orthopaedic surgery: spine, hip replacement, knee, fractures • gall bladder removal, prostate, various cancers • Non-invasive • radiation therapy • kidney stone pulverization

7. Medical imaging modalities • Preoperative • Film X-rays, Digital X-rays, Ultrasound, Angiography, Doppler, …. • Computed Tomography (CT), Magnetic Resonance (MR), Nuclear Medicine (PET, SPECT, …) • Intraoperative • X-ray fluoroscopy, ultrasound • video images (laparoscopy, arthorscopy) • Open MR

8. Medical imaging modalities: X-rays X-ray Fluoroscopy Film or Digital X-ray

9. Medical imaging modalities: continuous X-ray angiography

10. Medical imaging modalities: Ultrasound

11. Medical imaging modalities: CT Series of parallel slices 2mm apart Single slice

12. Medical imaging modalities: MRI Good imaging of soft tissue

13. Medical imaging modalities: Nuclear medicine (PET, SPECT, NMR) Functional imaging: colors indicate electrical activity

14. Medical imaging modalities: video TV quality image from small camera (laparoscope or endoscope)

15. Surgical approaches • Open surgery • area of interest directly exposed by cutting • direct sight and touch of anatomy by surgeon • direct access but causes additional damage • Closed surgerynot always feasible • indirect access to anatomical area of interest • no direct visual sight or tactile feel • catheterization, biopsies • intraoperative imaging is often required • require more skills: lengthier, more difficult • Diagnostic surgery

16. Minimally invasive surgery • Provides treatment through small incisions • Uses imaging equipment for seeing and instruments for touching • Advantages: less damage, faster recovery • Disadvantages: hand/eye coordination, time • Examples: • brain tumor removal, laparoscopic surgery

17. Laparoscopic surgery

18. Brain surgery

19. Total Hip replacement -- principle

20. Total hip replacement procedure Procedure Tools Fluoroscopic images

21. What is required to perform surgery? • Knowledge intensive task • anatomy, procedures, cases • experience, skills, customization and generalization • Manual and cognitive skills • dexterity, precision, strength, tool manipulation • spatial orientation and navigation • Determination • information integration • judgement, decision, execution

22. Medical and surgical trends • Imaging improved dramatically diagnosis • started with X-rays last century • 30% of all cases use images • Move towards minimally invasive procedures • introduced in the mid ‘70s, slow acceptance (laparoscopy) • the method of choice now • More precise and delicate procedures • Development of sophisticated surgical hardware • High degree of craftsmanship and skills

23. Socio-economical medical trends • Increase of aging population and associated problems: tumors, osteoporosis, Alzheimers • Larger population volumes • Universal, first rate, highly specialized care • Health care costs reduction (managed care) • Higher patient requirements • Legal and regulatory aspects

24. Surgical Needs Augment the surgeon’s capabilities with better quantitative planning, execution, and integration • Support for image-guided surgery • Passive and active devices for accurate spatial positioning, tracking, and execution • Modeling, planning, viewing, diagnosis systems • Systems integration: from diagnosis to post-op • Improve current practice and enable new procedures • Simulation and training systems Parts of the technology already available elsewhere!

25. Current clinical status • Imaging • vast databases of medical images • digitized atlases • mostly uncorrelated unimodal qualitative interpretation • Devices • mostly passive and non-invasive (supports) • laparoscopic camera, • some real-time tracking • Planning, modeling, visualization • 3D reconstruction, some registration

26. Part 2: Computers and Robots Technology and algorithms available today

27. How can computers help?(or are already helping…) • Image processing • single image: enhancement, noise reduction, segmentation, quantitative measurements • image stacks: 3D reconstruction, segmentation • image sets: registration, comparison, data fusion • Planning and simulation • integrate medical images and CAD models • planning and simulation programs • Computer vision and graphics • camera modeling, image registration, rendering

28. Image processing Measurements 2D segmentation 3D segmentation Surface modeling Image fusion Atlas

29. Planning and simulation Nail selection Walking simulation

30. Virtual man project -- digital model

31. How can robots and sensors help?(or are already helping…) • Robotic devices • passive, semi-active, active devices • instrument and anatomy positioning and holding • cutting and machining • Real-time tracking • optical, video, electromagnetic devices • navigation tools

32. Robotic devices Passive Semi-active Active

33. Real-time tracking devices camera instrument Passive markers Instrument has infrared LEDs attached to it Active markers

34. Computer-Assisted Surgery (CAS) A computer-integrated system to enhance the dexterity, visual feedback, and information integration of the surgeon • Key points: • The goal is NOT to replace the surgeon • A new paradigm for surgical tools • Address a real clinical need • Prove efficacy and cost-effectiveness

35. Elements of CAS systems

36. Elements of CAS systems • Preoperative planning • image acquisition, modeling, analysis, simulation • plan elaboration, tool and prosthesis selection • Output: preop images, 3D models, prosthesis type and position, navigation and cutting plan • Intraoperative execution • passive, semi-active, active robot • real time tracking • intraoperative imaging (fluoroscopy, ultrasound) All integrated by computer!

37. State of the Art (1) • Main clinical procedures • neurosurgery: biopsies, tumor removal • orthopaedics: hip and knee replacement, spine, pelvis and femur fractures • maxillofacial and cranofacial • laparoscopy: laparoscope holders • new fields: dentistry, ophtalmology, prostate • Mostly rigid structures: bones!!

38. State of the Art (2) • Commercial navigation systems • main uses: neurosurgery and spine surgery • Commercial robotic systems • ROBODOC for total hip replacement • laparoscope arm holders • Research • very active, very interdisciplinary • a few dozen systems tested in-vitro

39. State of the Art (3) • Major players • INRIA Sophia Antipolis, Grenoble, Johns Hopkins, Brigham Women’s H./MIT, Shadyside H./CMU, Imperial College, many places in Germany and Japan • Interdisciplinary conferences and journals • started in 1994: MRCAS’94; Orthopaedic CAS meetings, visualization, etc, • Journals: Computer-Aided Surgery, Medical Image Analysis

40. Examples of CAS systems in use • Image-guided navigation systems • ROBODOC: Total hip replacement surgery • LARS: Laparoscopic assistant • Radiosurgery Brief overview follows; will be covered in detail later

41. Image-guide navigation • Purpose • accurate placement of instruments with respect to imaged anatomy for several procedures • Problem addressed • provide 3D vision of unseen structures replace static 2D fluoroscopy or larger openings • improve precision of biopsies, screw placements • Scope • non-invasive • creates surface model from preop images • registration of images to anatomy by direct contact

42. Image-guided navigation

43. Image-guided navigation (2)pedicle screw insertion

44. Status • In clinical use • Over 7,000 neurosurgeries performed with commercial systems • Gaining popularity in pedicle screw insertion • Decreased the misplacement rate from 10-40% to 5-18% (clinical study of 700 cases) • More clinical applications under development

45. ROBODOC: Total hip replacement • Purpose • precise machining of cementless hip implant canal • Problem addressed • complications in canal preparation and implant fixation • improve positioning accuracy and surface finish • Scope • invasive, numerically controled machining • plan from preop CT, registered via pins • adapted commercial robot • custom bone fixator and bone motion detection

46. Artificial hip joint

47. Total hip replacement procedure Procedure Tools Fluoroscopic images

48. ROBODOC: Total Hip Replacement F S e e m c u t r i o n Manual broaching Robotic broaching

49. ROBODOC system diagram

50. ORTHODOC Planning