140 likes | 315 Views
Grid Enabled Neurosurgical Imaging Using Simulation. http://wiki.realitygrid.org/wiki/GENIUS. Introduction The GENIUS project aims to model large scale patient specific cerebral blood flow in clinically relevant time frames Objectives :
E N D
Grid Enabled Neurosurgical Imaging Using Simulation http://wiki.realitygrid.org/wiki/GENIUS
Introduction The GENIUS project aims to model large scale patient specific cerebral blood flow in clinically relevant time frames Objectives: • To study cerebral blood flow using patient-specific image-based models. • To provide insights into the cerebral blood flow & anomalies. • To develop tools and policies by means of which users can better exploit the ability to reserve and co-reserve HPC resources. • To develop interfaces which permit users to easily deploy and monitor simulations across multiple computational resources. • To visualize and steer the results of distributed simulations in real time
HemeLB Efficient fluid solver for modelling brain bloodflow called HemeLB: • Uses the lattice-Boltzmann method • Efficient algorithms for sparse geometries • Topology-aware graph growing partitioning technique • Optimized inter- and intra-machine communications • Full checkpoint capabilities.
Getting the Patient Specific Data Data is generated by MRA scanners at the National Hospital for Neurosurgery and Neurology trilinear interpolation 5122 pixels x 100 slices, res: 0.468752 mm x 0.8 mm Our graphical-editing tool 20482 x 682 cubic voxels, res: 0.469 mm
Cross-site Runs with MPI-g GENIUS has been designed to run across multiple machines using MPI-g Some problems won’t fit on a single machine, and require the RAM/processors of multiple machines on the grid. MPI-g allows for jobs to be turned around faster by using small numbers of processors on several machines - essential for clinician HemeLB performs well on cross site runs, and makes use of overlapping communication in MPI-g
HemeLB/MPI-g Requires Co-Allocation • We can reserve multiple resources for specified time periods • Co-allocation is useful for meta-computing jobs like HemeLB, viz and for workflow applications. • We use HARC - Highly Available Robust Co-scheduler (developed by Jon Maclaren at LSU). Slide courtesy Jon Maclaren
HARC • HARC provides a secure co-allocation service • Multiple Acceptors are used • Works well provided a majority of Acceptors stay alive • Paxos Commit keeps everything in sync • Gives the (distributed) service high availability • Deployment of 7 acceptors --> Mean Time To Failure ~ years • Transport-level security using X.509 certificates • HARC is a good platform on which to build portals/other services • XML over HTTPS - simplerthan SOAP services • Easy to interoperate with • Very easy to use with the Java Client API
Real Time Visualisation and Steering • A way to let HemeLB know the parameters to be steered --> we use the RealityGrid steering system to steer the input data on the fly. • One aim is to do all this for distributed (cross-site) simulations • For medical applications, need may be urgent
Application Hosting Environment • Need to utilize resources from globally distributed grids • Administratively distinct • Running different middleware stacks • Wrestling with middleware can't be a limiting step for scientists • Need tools to hide complexity of underlying grids
PDA/Cellphone Visualisation and Steering • RealityGrid: a tool for modelling and simulating very large or complex condensed matter structures… • Human Factors issues key • Large-scale applications, scheduling when resources become available, ‘around the clock’ computation… • AHE application launching built in to client • User interaction/workflow: not necessarily 9:00am – 5:00pm, from the ‘desk’ environment
AHE - HARC integration • Users can co-reserve resources from the AHE GUI client using HARC • When submitting a job, users can either run their jobs in normal queues, or use one of their reservations • AHE passes reservation through to the Globus GRAM • AHE client uses HARC Java client API to manage reservations
AHE ReG & WS GRAM Integration • For steerable applications, AHE server starts up a ReG Steering Web Service (SWS) for the application, & sets the end point reference in the job’s environment • AHE can submit jobs to GridSAM (described in JSDL) or WS GRAM (described in JDD) by applying a XSL Transform to the job specific WS-ResourceProperties document • AHE --> WS GRAM can launch single or multisite MPIg jobs
Conclusions • We have developed computationally efficient tools to manipulate, simulate and visualize patient specific vascular systems • Goal is to better comprehend blood flow behavior of normal and anomalous cerebral systems and provide patient specific models for clinical guidance in surgical operations • Application Hosting Environment extended to launch MPI-g cross site runs, integrate ReG steering and visualisation, as well as making HARC cross site reservations