Tele-Robotics Demonstration (Rover Tele-operation)

1. JUSTSAP-PISCES Symposium 2008 Tele-Robotics Demonstration(Rover Tele-operation) November 11, 2008

2. Agenda • Demonstration overview • Technology and Products of remote control platform • Let’s enjoy “Rover Tele-operation Time Trial”

3. Demonstration overview Let's experience the effect of "latency cancel technology" for teleoperation. Wireless LAN Access point Internet simulate PC Rover Operation PC Simulate data transmission delay in Internet Japan/USA PISCES Internet

4. Technology and Products You can develop advanced remote control system easily and at a low cost. TelePresence over IP Platform for the remote control system which requires the digital video transmitted by wireless LAN and internet.

5. Technology and Products • Advanced remote control system • Operation from everywhere of Internet available places • High operativity using Motion JPEG video • Easily and low cost • I/F for using COTS equipments (WirelessLAN, Internet, PC, game controller, etc.) • Software Library for easy system development by users • All-in-one platform for video-based remote control system

6. Remote control system using TPIP Microphone Sensor NTSC Camera Video & Sound data、Sensor data input WireLess LAN Access point PC Wireless LAN card TPIP Library Target System TPIP Unit Bidirectional WireLess LAN IEEE802.11a/b/g internet controller output Control command data、Sound data Speaker Motor control

7. + Internet Minimization of “Video transmission delay” Processing time 33ms NTSC video capture NTSC Camera Network Camera VGA 15fps QVGA 30fps JPEG compression JPEG Encoder VGA/QVGA 30fps 20ms Wireless LAN IEEE802.11g 20Mbsp 15Kbyte(1/60) 6ms Decoder & rendering JPEG decode & rendering 20ms PC monitor Refresh Rate:60Hz 17ms TPIP Total delay time 96ms≒100ms Web CameraTotal delay time 500ms

8. Minimization of “Video transmission delay” • Performance High-speed image compression 30 frames/sec. (NTSC -> Motion JPEG) No re-transmission Communication UDP/IP High-speed image rendering High-speed Motion JPEG decoding, image processing middleware

9. Removal of “Transmission delay influence” Japan Hawaii 1sec delay 5700km Earth Moon 6sec? delay 384400km 4km/h（ex. walk） moves during delay time Hawaii 1.1m Moon 6.6m 10km/h（ex. model car） Hawaii 2.8m Moon 16.8m

10. Removal of “Transmission delay influence” received imagecaptured at “ T ” T+ΔT Steering angle PointT speed Image transmission delay corrected imageview at T+ΔT specify a red frame part by three data ΔT：Image transmission delay

11. Correction of “Transmission delay fluctuation” Remove the video frame intervals fluctuation which degrades operativity without image buffering Time Stamp Ｔ１ Ｔ３ Ｔ４ Ｔ２ A fluctuation by the transmission delay occurs Send image Ｔ３ Ｔ４ Ｔ１ Ｔ２ Receive image Delay time：t1 （t1=“Time now” -T1） Ｔ２ Ｔ３ Ｔ４ Ｔ１ Display image display images at same interval by time stamps

12. Let’s enjoy “Rover Tele-operation Time Trial” First Gate Second Gate Start and Goal

13. An Introduction “TPIP Unit”

14. The feature of TPIP unit • Value remote control platform • All-in-one platform for video-based remote control system (Image processing board, control board, software library) • Various input/output for remote control system Image processing board Sound input/output : Audio Input image : NTSC Output image : MotionJPEG (VGA/QVGA 30fps) Wireless Communication : IEEE802.11a/b/g TPIP Unit Control board RS232 I/F RC servo I/F Pulse input Digital I/O Analog input DC motor I/F CAN I/F

15. TPIP Unit Hardware

16. V1 V2 V3 S1 S6 S2 S7 S3 S8 Servo S4 S9 Motor S5 S10 I4 I2 I3 I1 TPIP Unit Specifications NTSC CAMERA Target control board Image processing board NTSC display Vout JPEG chip Vout V in Ethernet Other unit WirelwssLAN (11a,11g) RS232 SH2 50MHz CAN I/F Wireless LAN Card RS232 RS232 SH4 240MHz pw pw RS232 5V O2 O4 O６ O８ A8 A6 A4 A2 P2 Audio Other unit O1 O５ O3 A3 O７ A7 A5 A1 P1 Encoder DC6～16V speaker Mike Sensor Analog Input Switch Voltage Sensor Digital INPUT

17. 11a Servo Servo Servo Servo Motor Motor Motor Motor Compatibility with the radio control system TPIP unit Radio control system for hobby Two-way Digital control command Video & Sensor data One-way Analog control command only replaceable Camera Input Image processing board R/C receiver Target control board amp Sensor Input RC servo Output amp amp RC servo Output DC Motor Output DC Motor Output RC parts usable

18. TPIP Unit Software

19. TPIP library User program TPIP Library DLL (Dynamic Link Library) Real-Time Image transmit Software Real-Time Image transmit Software JPEG decode Software High-speed rendering Controller Software Sound control S/W OS : Linux OS : Windows XP Internet UDP / IP UDP / IP JPEG device driver PC/AT compatible machine JPEG compression board H/W Motor control board TPIP Unit

20. TPIP library TPIP library TPIP Unit apply library Base library Image control / transmit function Wireless LAN UDP function UDP-SIO communication function SIO Communication function Other controller RS232 Graphic control function Display Controller input function USB

21. TPIP User Community Activities theme ・Robot Control ・OpenCV Image data processing etc… University Technical junior college Research laboratories Community Technical information TPIP Library source code Research & development results Product information archive Research and development(school) ： 26 members Research and development(Enterprise): 11 members RRC Student member : 23 members Student member : 9 members offer & technical support SanritzAutomation

22. Price in Japan TPIP Platform PACKAGE \200,000 academic price \100,000 You can choose TPIP Platform TPIP Unit TPIP library TPIP Unit Tele-Operation Rover PACKAGE \2,500,000 academic price \1,500,000 TPIP library Image processing board （compression & transmit） \140,000 Image processing board Remote control Target (Rover) TPIP library And… ・TPIP Platform customize ・TPIP Systems construction support