Execution

1. Animation Data Flow 5. Animation Data 1. BVH File Viewer SIM (Region) Server 2. binBVH Data Conversion 3. Request Cache File Execution JointMotionList 4. binBVH Viewer Viewer

2. System RINIONS Real Time Input from NI and Output to Network and Shared Memory System RINIONS Network Network Server Rinions Client NI / NUI Shared Memory NI : Natural Interaction NUI: Natural User Interface アプリケーション

3. Viewer Composition of the System Rinions ……… ……… Repeat Repeat UUID of the animation under execution is compared with the index of a shared memory. Index Joint data from Kinect. Does UUID exist in an index? Calculate Quaternion NO YES Write a UUID and Joint Data to Shared Memory Read Data from Shared Momory Read Data from JointMotionList Shared Memory Data Repeat Animation is performed ……… Repeat ………

4. Joints Index Joints Data （It exists for every joint） ・・・・・・・・・ Channel 1 Channel 21 Channel 0 Animation UUID Joint Info Joint Info Joint Info Channel 0 Animation UUID Animation UUID Animation UUID Position X Position X Position X Channel 1 Animation UUID Animation UUID Animation UUID Position Y Position Y Position Y Channel 2 Position Z Position Z Position Z ・・・・・・・・・ 64bytes Quaternion X Quaternion X Quaternion X Quaternion Y Quaternion Y Quaternion Y ・・・・・・・・・・ ・・・・・・・・・・ Quaternion Z Quaternion Z Quaternion Z Animation UUID Animation UUID Quaternion W Quaternion W Quaternion W Channel 21 36+4 bytes 8 bytes (double) 8 bytes (double) 8 bytes (double) Joint Info = State of Joint（1) + Existence of Data（1) + Joint No.（2） + 0x00000000（4） Byte Unit Structure of Shared Memory

5. Correspondence of joint mSkull mHead XN_SKEL_HEAD mEyeLeft mNeck mWristLeft XN_SKEL_LEFT_HAND XN_SKEL_NECK L_Hand mCollarRight mElbowLeft XN_SKEL_LEFT_ELBOW mChest XN_SKEL_TORSO mShoulderLeft XN_SKEL_LEFT_SHOULDER mTorso mPlevis (ROOT) mHipLeft XN_SKEL_LEFT_HIP mKneeLeft XN_SKEL_LEFT_KNEE mAnkleLeft mFootLeft XN_SKEL_LEFT_FOOT mToeRight Joints of SL/OS Available Joints of Kinect (OpenNI) Kinect SDK

6. b Q2 Q4 θ Q4 Q3 a Q3 c Q1 Cross product c = a × b Dot product cosθ = a・ b Q4 = (cos(θ/2), sin(θ/2)*c) Q4r = ~Q0*Q4*Q0 a,b,c is unit vector Q０ Quaternion of Joint

7. Log Window SaveCopy Clear File Menu Window Menu Setting Menu Help Sensor Window Start Detection Stop Detection Next User Start Record Stop Record Up Tilt Motor Initialize Tilt Motor Down Tilt Motor Current SDKStatus of Network FPS of writing to Shared MemoryReceived FPSReceived Speed

8. Second LifeOpenSim MikuMikuDance MikuMikuDance SL/OS Viewer SL/OS Viewer Shared Memory Shared Memory Kinect Kinect Rinions Rinions UDP UDP anm_server

9. Animation Relay Server anm_server (Linux/Unix) • Relay the UDP Data by Group Unit Relay Process Relay Process Relay Process fork fork fork anm_server

10. UDP Data （every animation） Header （64） Joint Data （32） Command（2） Timer（2） …………… Access Key （16+1） Quaternion Data×4（16） Coordinate Data×3（12） Animation UUID（36+1） Joints No. （2） Received Port of Rinions（2） Number of Joints（2） Various use（2） （　）is Byte Length Total Length = Header(64) + Joint Data(32)*Joint Num(15/11)= 544～416 bytes Status of Joints（1）＋Existence of Data（1）

11. Protocol • Logout • Login anm_server Relay Process Rinions Rinions DELETE LOGIN Access Key, Port Num. Animation UUID Group Key OK LOGOUT Access Key, Port Num. fork Access Key, Port Num. Relay Process REGIST Access Key, Port Num. Animation UUID TRANSFER Joints Data ・・・ TRANSFER • Sever identified the Client by IP address and port Num Joints Data TRANSFER Joints Data ・・・