190 likes | 283 Views
Exploration Robot with Stereo v ision. Vladislav Richter richterv2@fit.cvut.cz Miroslav Skrbek FIT, CTU in Prague skrbek@fit.cvut.cz. Outline. Introduction Robot's hardware Control Application Sensor visualization Kinect. Exploration R obot. Android phone or tablet. Wi - Fi. Android.
E N D
Exploration Robot with Stereovision Vladislav Richterrichterv2@fit.cvut.cz Miroslav Skrbek FIT, CTU in Prague skrbek@fit.cvut.cz
Outline • Introduction • Robot's hardware • Control Application • Sensor visualization • Kinect Vladislav Richter, Miroslav Skrbek PESW2014
Exploration Robot Android phone or tablet Wi-Fi Android 3D image (anaglyph) Commands forward backward rotate set_wheel_speed say get_eye_image … Vladislav Richter, Miroslav Skrbek PESW2014
Robot's Hardware Speaker (speech synthesis) Roboard RB110 (1GHz PC,Linux) Audio codec with an amplifier WiFi antena Cameras (stereo pair) Accelerometer, gyroscope, battery voltage, battery current and temperature sensors inside 80 cm distance IR sensor Eye Movement (up/down) Vladislav Richter, Miroslav Skrbek PESW2014
Remote Control Application • Directed for tablets and mobile phones • Android based application • Earlier version Android 2.3 • Latest version Android 4.4 • Motor control • Sensor visualization • Sound localization • Voice output Vladislav Richter, Miroslav Skrbek PESW2014
Sensor Visualization • Accelerometer • Gyroscope • Motor speed • System temperature • Battery • Voltage • Current • IR distance sensors • Sound source location Vladislav Richter, Miroslav Skrbek PESW2014
Accelerometer Data Visualizations Accelerometer data Terrain The model follows the movement of the robot Vladislav Richter, Miroslav Skrbek PESW2014
Sound Source Localization Δt1 Cross correlation I Localization Cross correlation II Δt2 Bar with four microphones (two stereo pairs) Vladislav Richter, Miroslav Skrbek PESW2014
Camera Images 3D View Cameras: 640x480, 30 fps, YUYV Anaglyph Vladislav Richter, Miroslav Skrbek PESW2014
Nearest object detection OpenCV based stereo correspondence calculation Left image Right image Depth map Nearest objects are white Vladislav Richter, Miroslav Skrbek PESW2014
Motor Control Virtual Joystick Movementkeypad Vladislav Richter, Miroslav Skrbek PESW2014
Advanced Feature – GUI Adaptation Android get_xml_info XML file GUI adaptation Description of sensors and commands available on the robot Vladislav Richter, Miroslav Skrbek PESW2014
Kinect on the Robot UDP multi-cast streams - color image - depth map - infrared image - skeleton tracking data - accelerometer - audio Wi-Fi Server 15 fps UDP stream - motor control - stream control Vladislav Richter, Miroslav Skrbek PESW2014
Kinect - Depth Map Image depth is 12 bits. To view the full range, a color scheme is used. Depth map Vladislav Richter, Miroslav Skrbek PESW2014
Kinect - Color Image Head-up Display Vladislav Richter, Miroslav Skrbek PESW2014
Kinect – Infrared Image Vladislav Richter, Miroslav Skrbek PESW2014
Kinect - Skeleton Data Vladislav Richter, Miroslav Skrbek PESW2014
Conclusion • Four-wheeled robot hardware • Stereo camera pair for distance measurement • Reach set of sensors • Sound source location • Voice output • Android based control application • 3D view by anaglyph • Sensor data visualization • Motor control Vladislav Richter, Miroslav Skrbek PESW2014
Thank you for your attention Vladislav Richter, Miroslav Skrbek PESW2014