360 likes | 496 Views
Randy Ellis, Ph.D. User Interfaces and Human-Computer Interaction. Control Loop: Classical View. Command (control input). Decision (difference between command and sensor). Sensor (state of WORLD ). Action (effect on WORLD ). WORLD : an unknown dynamical system.
E N D
Randy Ellis, Ph.D User Interfaces and Human-Computer Interaction
Control Loop: Classical View Command (control input) Decision (difference between command and sensor) Sensor (state of WORLD) Action (effect on WORLD) WORLD: an unknown dynamical system
Control Loop: Surgeon’s View Command (control input) Computer: Integrates plans, sensors, etc. Sensor (screen update) Action (updates internals) INTERNALS: an unknown system (to the user)
Driven by Requirements A requirement is a property that, if not present, means that the software is useless Functional requirements include: • Features, speed, order of actions, etc. Nonfunctional requirements include: • Appearance, documentation, etc. Technical constraints include: • Hardware, operating system, etc.
Application-Oriented Architecture Base (common) software Trans-Gastric app. Neurosurgery app. MRI Biopsy app.
Storyboards for Applications Engineer and clinicians work together: • Define the basic approach to the interventional procedure • Prototype screen appearances (paper cartoons or PowerPoint are useful) • Specify screen-to-screen actions • Implement “happy-day” scenario • Elaborate alternative workflows (e.g., error handling, new image acquisitions) Result: fast development of well tested software
Technical Overview • Requirements drive the process • Application-specific: • Rapid specification of ideal workflow • Catch design problems early • Elaborate necessary extensions only, not all possible variants • Test-oriented: • The specification produces tests before code is written • Repeat all tests whenever changes are made Result: robust, trustable systems
Clinical Problem: Biopsy in CT Scanner Idea: • Perform brain biopsy using 2D slices Technique: • Attach fiducials to head holder • Scan holder and patient • Select slices showing structures at risk • Superimpose needle trajectory on slices • Verify biopsy with new CT acquisition(s)
CT Biopsy Phase 1: Procedure Workflow • Prepare patient (including fiducials) • Transfer patient to CT scanner • Acquire the CT scan (?inject?, image) • Load scan into Navigator • Software calculates registration • Software displays needle trajectory in CT • Perform the biopsy • Close the patient
CT Biopsy Phase 2: Software Actions • Load instrument descriptions • Load CT scan • Detect fiducials • Calculate registration • Select key CT slices • Track the needle guide • Overlay needle trajectory on slices
CT Biopsy Phase 3: Procedure Steps • Screen to select scan from disk • Screen to give registration progress • Screen to select CT slices • Screen to display navigation Additional Requirements: • Forward/backward buttons • Exit button • Tool status display (e.g., visibility)
CT scan found & loaded Fiducials detected & computed Slices selected by clinician CT Biopsy Phase 4: State Transitions File Screen Registration Screen Selection Screen Navigation Screen
Tool Status Display EXIT LOGO GOES HERE CT Biopsy Storyboard: File Selection All we need here is a simple dialog box
Tool Status Display EXIT LOGO GOES HERE CT Biopsy Storyboard : Registration Need a progress bar during registration, then report RMS error after registration
Tool Status Display EXIT LOGO GOES HERE CT Biopsy Storyboard: Selection Idea #1 SELECT SLICE 1/2/3/4 Show original slices in here Scroll bar
Tool Status Display EXIT LOGO GOES HERE CT Biopsy Storyboard: Selection Idea #2 Show original slices in here Drag/Drop into here Scroll bar
Tool Status Display EXIT LOGO GOES HERE CT Biopsy Storyboard: 2D Navigation Show the needle trajectory on top of each slice Slice #1 Slice #2 Slice #3 Slice #4
CT Biopsy Phase 5: Implement and Test • Create scans with/without fiducials (to test registration algorithms) • Build phantoms to measure navigation accuracy • Power cycle computer to test system robustness • Unplug/plug tracking system, etc.
CT Biopsy Phase 6: Next Iteration Next round, we may need to: • Calibrate the biopsy needle intraoperatively • Navigate the biopsy needle • Use needle-oriented reslicing • Migrate system to use MRI scans (new application: software re-use) • Control MRI scanner for in-scanner biopsy (new application: software re-use)
Clinical Problem: Deep Brain Biopsy Idea: • Use flexible scope to biopsy from within a brain ventricle Technique: • Same setup as for CT biopsy navigation • Show 3D location of scope inside volumetric image • Needs calibration of optical image to tracking device
Deep-Biopsy Phase 1: Procedure Workflow • Prepare patient (including fiducials) • Transfer patient to CT or MRI scanner • Acquire the 3D scan (inject & image) • Load scan into Navigator • Software calculates registration • Calibrate ventriculoscope • Software displays scope in volumetric scan • Perform the biopsy • Close the patient
Deep-Biopsy Phase 2: Software Actions • Load instrument descriptions • Load volumetric (MRI/PET/CET) scan • Detect fiducials • Calculate registration • Calibrate ventriculoscope to tracker device • Confirm calibration using a sharp probe • Track the ventriculoscope • Show scope in volumetric (MRI/PET/CET) scan
Deep-Biopsy Phase 3: Procedure Steps • Screen to select scan from disk • Screen to give registration progess • Screen to calibrate ventriculoscope • Screen to verify calibration • Screen to display navigation Additional Requirements: • Forward/backward/exit buttons • “Tabs” to other tasks label “forward”
MRI/PET/CT scan found & loaded Fiducials detected & computed Calibration accepted by surgeon Calibration verified by surgeon Deep-Biopsy Phase 4: State Transitions File Screen Registration Screen Calibration Screen Verification Screen Navigation Screen
LOAD CALIBRATE VERIFY NAVIGATE REGISTER EXIT LOGO GOES HERE Deep-Biopsy Storyboard: File Selection All we need here is a simple dialog box ACCEPT
LOAD CALIBRATE VERIFY NAVIGATE REGISTER EXIT LOGO GOES HERE Deep-Biopsy Storyboard: Registration Need a progress bar during registration, then report RMS error after registration ACCEPT
LOAD CALIBRATE VERIFY NAVIGATE REGISTER EXIT LOGO GOES HERE Deep-Biopsy Storyboard: Calibrate Vscope Show live video feed in here CALIBRATE
LOAD CALIBRATE VERIFY NAVIGATE REGISTER EXIT LOGO GOES HERE Deep-Biopsy Storyboard: Verification 3D probe model on top of live video feed here ACCEPT
LOAD CALIBRATE VERIFY NAVIGATE REGISTER EXIT LOGO GOES HERE Deep-Biopsy Storyboard: 3D Navigation Scope field-of-view cone inside volume Live video feed from scope Forward rendering from scope
Deep-Biopsy Phase 5: Implement and Test • Create scans with/without fiducials (to test registration algorithms) • Calibrate optics to tracker device • Verify calibration with sharp probe • Build phantoms to measure navigation accuracy • Power cycle computer to test system robustness • Unplug/plug tracking system, etc.
Deep-Biopsy Phase 6: Next Iteration Next round, we may need to: • Calibrate the biopsy needle • Navigate the biopsy needle • Use needle-oriented reslicing • Acquire/integrate intraoperative MRI (new application: software re-use) • Migrate to laparoscopic ultrasound (new application: software re-use)