190 likes | 275 Views
A Plan for Visualization and Mapping Neurons. Ying Zhu Department of Computer Science Georgia State University. Project Goal. Create a intuitive web-based mechanism for recording and visualizing on a 3-D brain model the locations of: identified neurons, observed neurons,
E N D
A Plan for Visualization and Mapping Neurons Ying Zhu Department of Computer Science Georgia State University
Project Goal Create a intuitive web-based mechanism for recording and visualizing on a 3-D brain model the locations of: • identified neurons, • observed neurons, • neuron arborization.
User Interface • Create a 3D brain atlas • Allow users to manually deform the 3D atlas to match the target brain • Can overlay brain images to guide the deformation • Users can mark the location of the neurons on the deformed 3D model • Enter additional neuron attribute information.
Creating the Model • We create the 3D model by breaking down the brain shape into 4 regions. • Connecting the lobes will happen later. • Nerves will also be added later 2 3 4 1
Deform the Brain Model • Physics based deformations are difficult to control. • We use non-physics based deformation algorithms • More efficient • Users have interactive control over the deformation process
Select the Location • Using a mouse or virtual laser pointer, the user selects a point that they believe represents the location of the neuron.
How to represent neuron locations? • We want to conduct comparative studies of homologous neurons • But brains come in different sizes and shapes • To compare their locations we need a common coordinate system that adapts to shape/size variations
Existing Methods • Create a 2D or 3D brain atlas from brain cross section images • Define a coordinate system for the atlas based on certain landmarks • Deform the atlas to fit the target data (images), or vice versa • Eventually assign a coordinate to the features on the target images
Characteristics of Tritonia Brain • Most neurons are on the surface or just below the surface of the brain • Cross section images are not necessary as in the study of other types of brains. • We want to mark the neuron locations directly on a 3D brain model • The shape of Tritonia brain is relatively simple • But we still need to deal with shape and size variations
Our Solution • Use “texture mapping” technique to establish a direct mapping between the 3D model and a 2D image • Each point on the 3D brain model is mapped to a point on the 2D image
What is Texture Mapping? • Texture mapping is the method of taking a flat 2D image of what an object's surface looks like, and then applying that flat image to a 3D computer generated object. • Makes a surface look textured even though geometrically it isn’t.
Texture Mapping • Using the texture mapping technique we can map a (x,y,z) value a (s, t) value, or vice versa. • Forwardx = X(s,t)y = Y(s,t)z = Z(s,t • Reverses = S(x,y,z)t = T(x,y,z) • And… we can deform the object, but the (s,t) texture for that vertex does not change. (x,y,z) t s
Texture Mapping Example • Each pixel on the 2D image is assigned to a vertex on the 3D object.
Neuron Mapping • The 3D neuron location is mapped to the 2D “texture” image coordinate • The 2D coordinate will be stored in database • Can easily project this 2D coordinate back to the 3D brain atlas • The size and shape of the brain model may change but the mapping between 3D model and 2D image is stable • The 2D “texture” image thus provides a common coordinate space
What is stored? • Final data stored for Neuron location is: • Which region (lobe) the point was marked. • The 2D texture coordinates (s,t) of the point that was marked. Coord: 2 3 3 4 1 , 150,380
Benefits • Does not rely on brain cross section images • User-guided, semi-automatic model deformation for better control • Allow user to mark neuron locations directly on 3D models • Address brain shape/size variations by mapping neuron locations to a common 2D coordinate space • Can be easily adapted to different species • Use different 3D atlas, “texture” image, and/or mapping equations
Limitations • Only works for brain models where neurons are on or close to the surface • It will take some practice to learn how to deform the atlas model. • Texture Mapping is not a one-to-one mapping, so some inaccuracies may result. • Can be minimized by matching the resolution of the 2D image with that of the 3D atlas • How accurate do we need to be?
Acknowledgements • People • Paul Katz (Biology) • Raj Sunderraman (CS) • Jason Pamplin (CS) • Robert Calin-Jageman (Biology) • Jim Newcomb (Biology) • Hao Tian (CS, Brains & Behavior Fellow) • Christopher Gardner (CS, Brains & Behavior Assistantship) • Lei Li (CIS, Brains & Behavior Fellow) • Brains & Behavior Program
Questions to Neuroscientists • Do you foresee any problem with this approach (particularly the neuron localization)? • Will this brain mapping approach be useful for brains of other invertebrate species? • How do you save neuron location information in your database? • Can there be a general way to specify neuron locations for invertebrates?