Responsive Dynamic Three-Dimensional Tactile Display Using Hydrogel

1. Responsive Dynamic Three-Dimensional Tactile Display Using Hydrogel Concept: A responsive flexible 3D interface that assists visual impaired people to communicate with the modern IT (Information Technology) world, and with the health care services. With this 3D tool they can more effectively understand image-assisted content, and interact with electronic devices that will open new doors for them to learn and communicate more efficiently. Functions:The surface of interface provides a real-time 3D topological shape change according to input information, such as image brightness, color-maps, and so on. For example, a 3D display may have 256 x 256 pixels, with 32 level of height variation for each pixel. The higher brightness of the optical image pixel, the higher the height of each 3D tactile display pixel. Blind people can FEEL the image brightness distribution by the sensing the 3D tactile height distribution. Technology and Team:The system uses fabrication with hydrogel material with scalable micro-manufacturing technology that employs MEMS devices. The collaboration between visional impaired people, Material Engineering, Mechanical Engineering, Bioengineering, Electrical Engineering, the Sciences, and Education experts are essential for the success of the movable 3D tactile system development.

2. System Concept View A responsive tactile display skin is a flexible interface to produce a dynamic 3D tactile surface, representing the optical light intensity distribution from an underlying iPhone, iPad, or computer screen. A stand-alone responsive tactile display will produce a dynamic 3D tactile surface representing the optical light intensity distribution from electric input files.

3. System Schematics with Building Modules Stand-alone unit 3D tactile topology shape changing unit based on hydrogel data input, converting and process unit Active Smart skin unit 3D tactile topology shape changing unit based on hydrogel data converting and process unit Light information acquisition unit to extract the information from attached optical display Self responsive smart skin unit 3D tactile topology shape changing unit based on light-sensitive hydrogel. When attaching it to an optical display, the light from optical display will work as stimulus to drive the topology change of the hydrogel

4. Building Material - Hydrogel Hydrogel = solvent (usually water) + (a network of polymers): Responsive to Environmental Stimuli. reversible collapsed swollen Stimuli, which cause the hydrogel shrinking or swelling • temperature • pH • ions • enzymes • electric field • light

5. Example: Thermal Responsive Hydrogel Galaxy optical image Hydrogel swelling: transition temperature, slope, and dimension change can be tuned through chemical composition/solvent. Modeling of 3D tactile display based on temperature sensitive hydrogel (based on right hand swelling ratios)

6. Pixel Design Based on Hydrogels Hydrogel block for each pixel Package to enclose the hydrogel and solvent Solvent Display pixel Stimulation component for each pixel Different types of stimuli can be employed, such as heat, cooling, light, electrical field, pH and so on, depending on the type of hydrogel used. The pixel separator is designed to isolate individual pixel to avoid crosstalk and improve stimulation efficiency. For example, when we use heat to drive a temperature sensitive hydrogel, well-constructed thermal insulation cavities need to stand between pixels. Or when we use light to drive photosensitive hydrogel, a light blocker is needed between adjacent pixels. Stimulation component Hydrogel Pixel separator solvent

7. Technology Development - Hydrogel Design,Synthesis, and Optimization • Two General Approaches: • Thermoresponsivehydrogels (Upper critical solution temperature (UCST) ones are preferred, for example, poly(acrylamide)/poly(acrylic acid) interpenetrating network. ) • Light sensitive hydrogels, for example, chlorophyllin based hydrogels which are sensitive to visible light. • The transition temperature, dimension change, and response rate will be optimized by co-monomer, solvent, and additives. Preliminary work on Poly (N-isopropylacrylamide) (PNIPAAm) (More on next slide)

8. Technology Development - Mechanics of Integration of Soft and Hard Materials Si elements PDMS (adhesive) PNIPAAm (temperature sensitive gel) • Interactions between multi-disciplinary physics fields --- two focused investigations: • Use a material model to understand how hydrogel behaves under various stimuli • Use numerical platform to model concurrent deformation of soft gels and hard packaging materials Heat up: shrink 350X Cool down: swell Tunable curvilinear gels integrated with hard materials t =0 @ RT t= 2 min @ 45oC

9. Technology Development Detail --- MEMS –based Pixel Component Development Photodiode Optical intensity acquisition from attached optical display with be performed with (1) a photodiode built on flexible substrate for active smart tactile skin, and (2) a optical lens-array on flexible substrate. Top dynamic surface will be covered by an elastomer polymer, for example, polydimethylsiloxane (PDMS), which will conformally change the shape with swelling of hydrogel. PDMS will protect hydrogel from contacting environment and hand touching. Solvent encapsulation and pixel separation will be realized with micro- manufacturable polymer – Parylene-C and air cavities Since involving a solvent (e.g. water), fabrication should be at low temperature and wafer compatible. This will bring a great challenge in microfabrication and packaging. Various processes will be explored. solvent

10. Responsive  Smart 3D tactile display Module Data analysis and process Module Tactile Sensor Module The responsive 3D dynamic tactile display may become smart by integrating a sensor module and central intelligent electric unit. Then it becomes fully functional two-way tactile interface. • Other potential applications: • Gaming control interface; • An emergency human-machine interface in addition to the usual optical and sound interfaces, e.g., in case a space crew or an airplane pilot are blinded or loose their eye-sight temporarily.

11. Development Plan • Short Term (year 1): Feasibility Verification • Design and synthesize UCST hydrogel with broader transition temperature window • Fabricate integrated hydrogel and hard packing/substrate materials • Realize 6×6 pixels with each pixel about 5-10 mm in size and have 4 levels of height difference • Main challenges: material synthesis, integration, and pixel separator • Funding request: $120,000 • Long Term (years 2-5): Optimization and Full Functionality • Synthesize hydrogels with different environmental sensitivities • Realize 256×256 pixels with each pixel about <1 mm in size and have 32 levels of height difference • Main challenges: more robust material with large swelling ratio, MEMS fabrication to realize much smaller pixel size • Funding estimation : ~ 2M

12. Gain on Fundamental Research and Education • Education • An evolutionary interface to assist in the STEM education of visually impaired or blind people. • Understand the methodology to employ this new 3D tactile interface in education in general. • Applications • A new dimension for human-machine interface. • Potentially wide consumer-electronics applications (gaming, flight, or space-flight). • Science and Engineering • Design, synthesize, and optimize responsive hydrogels • Understand the mechanics of integrating soft and hard materials. • Develop a scalable micro-manufacture technology using a combination of soft and hard materials, polymers, and solvents.

13. Direct Benefit for NASA and Space Technology • Dynamic 3D terrain by hydrogels to train the miniaturized rovers on some missions, for example, making 3D map with mm- (or cm-) sized features to reproduce the surface profile of other planets (e.g., Mars) to train robots (e.g., Curiosity rover). • The hydrogelcan be designed to be responsive to certain wavelength or wavelength range thus can provide 3-D “visualization” of particular spectra of NASA’s interest, for example, ultraviolet (upper atmosphere and beyond) vs. visible and infrared (atmosphere and earth). • The hydrogel can respond to certain fingerprint light signals as sensors. • Hydogel-based dynamic and interactive (using the proposed “smart” or feedback mechanism) strategy planning board (instead of static one using models) Mars surface before and after landing of Curiosity: it can be mimiced by hybrodgel-based 3D terrain Dynamic board to replace the static one

14. Research Team Scientific Lead: Dr. Rogier Windhorst Visional Impaired people and other customers Responsive 3D tactile display Device Development Education Development Chemical Engineering: Dr. Lenore Dai Mechanical Engineering: Dr. Hanqing Jiang Electrical Engineering: Dr. Hongyu Yu Computer Engineering: Dr. Baoxin Li Biologist: Dr. Debra Baluch Astronomer: Dr. Rogier Windhorst Education Expert: Dr. Terri Hedgpeth (blind) Blind Students: Ashleigh Gonzales, …