1 / 5

Design Tactile Interfaces with Enhanced Depth Images With Patterns and Textures for Visually Impaired People

This paper presents the design and development of Tactile Interfaces with Enhanced Images with Patterns and Textures for Visually Impaired People. In particular, haptic interfaces uses a number of predefine tactile texture and patterns to enhance the experience of the tactile feeling of images for visually impaired people. The open source OpenCv application is used to automatically change group of colours from depth images to tactile texture and patterns. The depth images are captured with the he p of RGB Depth cameras. The tactile images are then explored by visually impaired people with the use of haptic interfaces. The predefined textures and patterns are more easily distinguished than uniformly colors in depth images. George Kokkonis | Kostas Psannis | Christodoulos Asiminidis | Sotirios Kontogiannis "Design Tactile Interfaces with Enhanced Depth Images With Patterns and Textures for Visually Impaired People" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-1 , December 2018, URL: https://www.ijtsrd.com/papers/ijtsrd20210.pdf Paper URL: http://www.ijtsrd.com/engineering/computer-engineering/20210/design-tactile-interfaces-with-enhanced-depth-images-with-patterns-and-textures-for-visually-impaired-people/george-kokkonis<br>

teamijtsrd1123
Download Presentation

Design Tactile Interfaces with Enhanced Depth Images With Patterns and Textures for Visually Impaired People

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. International Journal of Trend in International Open Access Journal International Open Access Journal | www.ijtsrd.com International Journal of Trend in Scientific Research and Development (IJTSRD) Research and Development (IJTSRD) www.ijtsrd.com ISSN No: 2456 ISSN No: 2456 - 6470 | Volume - 3 | Issue – 1 | Nov 1 | Nov – Dec 2018 Design Tactile Interfaces Patterns and Textures tile Interfaces with Enhanced Depth Images With nd Textures for Visually Impaired People Kostas Psannis1, Christodoulos Asiminidis2, Sotirios Kontogiannis Applied Informatics, University of Macedonia, Thessaloniki ith Enhanced Depth Images With or Visually Impaired People George Kokkonis1, Kostas Psannis 1Department of Applied Inf 2Laboratory Team of Distributed Microcomputer Distributed Microcomputer Systems, Department of Mathematics niversity of Ioannina, Ioannina, Greece Sotirios Kontogiannis2 Thessaloniki, Greece Mathematics, 2University ABSTRACT This paper presents the design and development of Tactile Interfaces with Enhanced Images with Patterns and Textures for Visually Impaired People. In particular, haptic interfaces uses a number of predefine tactile texture and patterns to enhance the experience of the tactile feeling of images for visually impaired people. The open source OpenCv application is used to automatically change group of colours from depth images to tactile texture and patterns. The depth images are captured with the heλp of RGB cameras. The tactile images are then explored by visually impaired people with the use of haptic interfaces. The predefined textures and patterns are more easily distinguished than uniformly colors in depth images. Key Words: Haptics, tactile interfaces, 3D models, depth images, RGB-Depth cameras, haptic geometric patterns 1.INTRODUCTION Touch is a basic sensation for the understanding our surrounding environment. The sensation involves the skin's ability to separate properties of an object, and the kinaesthetic the muscles to determine the body's position. The reaction that we acquire whenever we shape, the texture, the geometry, the elasticity, and the dimensions of objects hands is called tactile feedback. There kinaesthetic feedback which is the stimuli produced in muscles and tendons from translation, limb motion control, and body Visually impaired people are depending and kinaesthetic feedback surroundings. RGB-Depth cameras model of the environment and virtual reality. Haptic interfaces visually impaired people to explore augmented virtuality. Geometric to increase the experience of and enhance the haptic feeling The rest of the paper is organized 2 presents a related work on tactile 3 presents the 3D depth monitoring used to capture and reproduce the surroundings. Section 4 patterns that could be used for of the 3D models. Section OpenCV algorithm that manipulates reproduce the enhanced tactile describes the H3D haptic program that patterned image to haptic objects. concludes the paper. 2.HAPTICDEVICES Virtual objects acquire realistic mass, hardness, texture and interface is an interactive actuator the 3D object and the user in a receives commands from the user 3D virtual objects. It consequently from the virtual environment the user’s skin and muscles. The tactile interfaces are split according to the stimuli that they They are divided into tactile stimuli that they produce refers and to kinesthetic haptic interface refers to muscle of the user and This paper presents the design and development of cameras can record the 3d and replicate it to a 3d interfaces can be used by explore and react with this Geometric patterns can be used the haptic interface user feeling of the 3d objects. ed Images with Patterns and Textures for Visually Impaired People. In particular, haptic interfaces uses a number of predefine tactile texture and patterns to enhance the experience of the tactile feeling of images for visually urce OpenCv application is used to automatically change group of colours from depth images to tactile texture and patterns. The depth organized as follows: Section tactile interfaces. Section monitoring technology that is reproduce the 3D virtual model of proposes the geometric for the tactile enhancement 5 describes the open manipulates the 3d models to tactile 3d objects. Section 6 λp of RGB-Depth cameras. The tactile images are then explored by ith the use of haptic interfaces. The predefined textures and patterns are more easily distinguished than uniformly colors in Haptics, tactile interfaces, 3D models, Depth cameras, haptic geometric describes the H3D haptic program that transforms the patterned image to haptic objects. Section 7, finally understanding of sensation of touch separate the surface kinaesthetic ability of position. realistic properties such as and elasticity. The tactile actuator that interacts with a virtual environment. It user and manipulates the consequently receives feedback environment and creates stimuli on we explore the the mass, the objects with our There is also the stimuli that is from power-force body position. split into two categories they produce to the users. tactile interfaces when the refers to the skin of the user interface when the stimuli and his/her body position. depending in the tactile to explore explore their @ IJTSRD | Available Online @ www.ijtsrd.com www.ijtsrd.com | Volume – 3 | Issue – 1 | Nov-Dec 2018 Dec 2018 Page: 1174

  2. International Journal of Trend in Scientific Research International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456 and Development (IJTSRD) ISSN: 2456-6470 3.DEPTH CAMERAS FOR 3D MODELLING In order to replicate real world objects, RGB-depth cameras RGB-depth camera that is Microsoft Kinect Depth sensor a VGA video camera, a CMOS infrared projector. The projector to the physical object. The physical infrared light back to the CMOS depth sensor captures the reflected time passed from the moment light until the light reaches the the distance between the sensor object. This time is called “time In order to connect the kinect produce custom software for Microsoft provides the Kinect Kit (SDK) and the Kinect Studio. captures the 3D image, the VGA images, as shown in Figure 3. DEPTH CAMERAS FOR 3D MODELLING world objects to 3D virtual Tactile Interface are more common and skin feedback and produce vibration, heat to the users fingertips. They help explore elasticity, the texture and the dimensions virtual 3D objects. Kinesthetic haptic devices provide feedback according to his movements and his body They help the user to explore the mass, and the hardness of the virtual 3D objects. One commonly used haptic devices, Systems Touch Haptic Device” (1), figure both tactile and kinesthetic feedback to the haptic device that user a ring or a stylus with the user. It measures the position fingertip. The user can feel the sense movement constraints, friction and surface and other features of virtual objects. and they support pressure and help the user to dimensions of are used. One low cost commonly used is the sensor (3). It is composed by CMOS depth sensor and an projector emits infrared light physical object reflects the CMOS depth sensor and the reflected infrared light. The moment the projector emits the the depth sensor defines sensor and the physical “time of flight”. feedback to user body position. mass, the friction objects. devices, is the “3D figure 1. It offers the user. It is a stylus to interact position of the user's sense of touch, surface texture, kinect sensor to the pc and for 3D-Depth mapping, Kinect Software Development Studio. The kinect studio VGA image and the depth Figure1. 3D Systems Touch Haptic Haptic Device Another common haptic device is the“Novint (2). It was released in 2007 and it was the device that offered high resolution 3D feedback user, Figure 2. Novint Falcon” the first haptic feedback to the Figure2. Novint Falcon Haptic Device Device The 3D Systems Touch Haptic Device Degrees Of Freedom (DOF) to the user Novint Falcon offers 3 DOF. Both devices servo rate, which means that they can update packets per second. Device offers six user while the devices uses 1kHz produce 1000 Figure3. Images captured from VGA image at the left side, the middle and the 3D image from the Kinect sensor. The the Depth image at the image at the right @ IJTSRD | Available Online @ www.ijtsrd.com www.ijtsrd.com | Volume – 3 | Issue – 1 | Nov-Dec 2018 Dec 2018 Page: 1175

  3. International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456 International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456 International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470 4.MAPPING DEPTH IMAGES TO ENHANCED PATTERNED IMAGES. In order these images to be more easily to a grey scale image as shown in Figure Figure4. MAPPING DEPTH IMAGES TO ENHANCED PATTERNED IMAGES. haptically explored the depth images of figure 3 should be transformed Figure4.Depth .Depth images transformed to gray scale images Let us assume that we No. Red Green 1 255 2 229 3 204 4 178 5 153 6 127 7 102 8 9 10 11 we have an 11 RGB Grey scale. as depicted in figure Green Blue RGB % 255 255 (100,100,100) 229 229 (90,90,90) 204 204 (80,80,80) 178 178 (70,70,70) 153 153 (60,60,60) 127 127 (50,50,50) 102 102 (40,40,40) 76 76 (30,30,30) 51 51 (20,20,20) 25 (10,10,10) 0 (0,0,0) figure 5. Hex Color Code FFFFFF E5E5E5 CCCCCC B2B2B2 999999 7F7F7F 666666 4C4C4C 333333 191919 000000 255 229 204 178 153 127 102 76 51 25 25 0 76 51 25 0 Figure Figure5. Gray scale in the RGB model These 11 Grey levels should be correspond corresponded to the simple geometrical patterns, such such as those of Figure 6. Figure6. Simple Geometrical Geometrical Patterns for enhancing tactile interfaces interfaces @ IJTSRD | Available Online @ www.ijtsrd.com www.ijtsrd.com | Volume – 3 | Issue – 1 | Nov-Dec 2018 Dec 2018 Page: 1176

  4. International Journal of Trend in Scientific Research International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456 and Development (IJTSRD) ISSN: 2456-6470 The image of of Figure 4 is then transformed in Figure 7. Figure Figure7. Haptically enhanced depth image 5.THE USE OF OPENCV FOR AUTOMATIC TRANSFORMATION OF RGB IMAGES TO HAPTIC IMAGES This transformation can automatically the help of the OpenCV (4) image processing and python. The imread function will load the colored RGB image to OpenCV. Next, we convert the color image to grayscale with the help of the cvtColor function. In order to convert the original image from the color space to gray, the code COLOR_BGR2GRAY is used. Gray = cv2.cvtColor (Image, cv2.COLOR_BGR2GRAY) Now we have to correspond the levels of the gray scale of Figure 5 to the geometrical patterns of Figure 6. This is done with the help of the library colorspace. 6.TRANSFORMGRAYSCALEIMAGES HAPTICOBJECTS In order to haptically explore the patterned images produced in the previous section the H3D haptic application is used (5). The H3D uses XML files to pass haptic properties to the objects illustrated in the pictures. The static friction, the dynamic friction, the OF OPENCV FOR AUTOMATIC TRANSFORMATION OF RGB IMAGES TO stiffness and the maximum height of the objects be given to objects with the following XmL code. <DepthMapSurface maxDepth="0.0 staticFriction="0. dynamicFriction="0. whiteIsOut="False <ImageTexture containerField="depthMap" url="patterned_image.png" repeatS="false" repeatT="false" </DepthMapSurface> 7.CONCLUSION The system proposed gives impaired people to haptically surrounding environment with interfaces. Predefined patterns enhance the experience of textures and patterns help visually distinguish more easily the objects a picture. The color of the automatically with the help of to the predefined patterns. These easily distinguished by haptic uniformly colors. maximum height of the objects can with the following XmL code. be done with processing library stiffness="0.6" maxDepth="0.008" staticFriction="0.9" dynamicFriction="0.9" False" > function will load the colored RGB image to OpenCV. Next, we convert the color image to containerField="depthMap" _image.png" repeatS="false" repeatT="false"/> </DepthMapSurface> function. In order to convert the original image from the RGB COLOR_BGR2GRAY , cv2.COLOR_BGR2GRAY) the ability to visually explore images and the with the help of tactile patterns and textures are used to of the user. Predefined visually impaired people to objects that are included in the object is changed of the OpenCv application These patterns are more haptic interfaces than Now we have to correspond the levels of the gray geometrical patterns of Figure 6. This is done with the help of the library colorspace. IMAGESTO haptically explore the patterned images produced in the previous section the H3D haptic . The H3D uses XML files to pass haptic properties to the objects illustrated in the the dynamic friction, the @ IJTSRD | Available Online @ www.ijtsrd.com www.ijtsrd.com | Volume – 3 | Issue – 1 | Nov-Dec 2018 Dec 2018 Page: 1177

  5. International Journal of Trend in Scientific Research International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456 and Development (IJTSRD) ISSN: 2456-6470 4.Jeeva, B., Sanjay, V., Purohit, V., Tauro, D. O., & Vinay, J. Design and Development of Automated Intelligent Robot Using 2018 International Innovations for 3Cs Compute Communicate Control (ICDI3C) . pp. 92- 5.Aondoakaa, A. S., Swash, R. M., & Sadka, A. 3D depth estimation from a holoscopic 3D image. In 3D Image Acquisition Technology, Perception and Applications. DW1F-5. REFERENCES 1.Touch., https://www.3dsystems.com/haptics-devices/touch. [Online] Dec 2018. 2.N. Karbasizadeh, A. Aflakiyan, M. Zarei, M. T. Masouleh and A. Kalhor. Dynamic identification of the Novint Falcon Haptic device. . International Conference on Robotics and Mechatronics (ICROM), Tehran. 2016, pp. 518 523. 3.Zhang, Z. Microsoft kinect sensor and its effect. IEEE multimedia, . Vol. 19(2), 4-10. Jeeva, B., Sanjay, V., Purohit, V., Tauro, D. O., Design and Development of Automated Intelligent Robot Using OpenCV. In 2018 International Innovations for 3Cs Compute Communicate Touch., 3D 3D Systems. Systems. devices/touch. Conference Conference on on Design Design N. Karbasizadeh, A. Aflakiyan, M. Zarei, M. T. -96. Dynamic identification Aondoakaa, A. S., Swash, R. M., & Sadka, A. 3D depth estimation from a holoscopic 3D image. In 3D Image Acquisition chnology, Perception and Applications. 2017, of the Novint Falcon Haptic device. . 4th International Conference on Robotics and 2016, pp. 518- and and Display: Display: Microsoft kinect sensor and its effect. 10. @ IJTSRD | Available Online @ www.ijtsrd.com www.ijtsrd.com | Volume – 3 | Issue – 1 | Nov-Dec 2018 Dec 2018 Page: 1178

More Related