1 / 23

Vladimir Veevnik Alexander Sorokoletov

Future begins now…. Vladimir Veevnik Alexander Sorokoletov. What is Kinect Capabilities How it works API Ideas What’s next . Agenda. It’s…. … controller-free gaming and entertainment experience … a controller for Xbox … a device, invented by Microsoft ( and Prime Sense)

huy
Download Presentation

Vladimir Veevnik Alexander Sorokoletov

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Future begins now… Vladimir Veevnik Alexander Sorokoletov

  2. What is Kinect • Capabilities • How it works • API • Ideas • What’s next Agenda

  3. It’s… … controller-free gaming and entertainment experience … a controller for Xbox … a device, invented by Microsoft (and Prime Sense) … future that begins now What’s kinect?

  4. Gesture recognition • Facial recognition • Voice recognition • Track up to 6 people (2 active) • Take photo & make video Capabilities

  5. Overview • Hardware (Physics) • Software (Mathematics) How it works

  6. 2) RGB camera 3) Monochrome camera 1) IR source 4) Microphone Array 5) Motorized tilt Led Indicator How it works Infrared source: 830nm infrared light, diffraction grate, class 1 laser RGB Camera 640x480x8bit@30fps Infrared Depth Sensor 640x480x11bit@30fps (monochrome) camera 4 microphones in Array

  7. IR Laser -> Diffraction Grate -> Speckle pattern -> Pattern distortion -> Depth image (monochrome) How it worksHardware (physics)

  8. Dots speckle pattern -> Depth image (colored)

  9. Depth Image -> Randomized trees -> Body parts labeling -> Joint proposals -> Skeleton model How it worksSoftware (mathematics)

  10. 500 000 (100 000) images • 2000 pixels per image • 20 levels per tree • 3 trees -> 1 day calculation on 1000-core cluster • GPU -> 200 fps Randomized trees

  11. Prime Sense & OpenNI • Microsoft Kinect SDK Overview • Some Internals • Code Example • Demonstration (working code) API

  12. OpenNI (Natural Interface) • Device represents a physical device • Depthgenerates depth-maps • Image generates colored image-maps • IR generates IR image-maps • Audio generates an audio stream • Gestures generates callbacks when specific gestures are identified • SceneAnalyzer analyzes a scene (separates background from foreground, etc.) • Hands generates callbacks when hand points are created, their positions change • User generates a representation of a user in the 3D space. API

  13. Microsoft.Research.Kinect.Nui • Microsoft.Research.Kinect.Audio • (Microsoft.Research.Kinect.dll) • Runtime • Depth image • Color image • Skeleton • Device • Camera API(Microsoft SDK)

  14. API(class diagram)

  15. HipCenter Spine ShoulderCenter Head ShoulderLeft ElbowLeft WristLeft HandLeft ShoulderRight ElbowRight WristRight HandRight HipLeft KneeLeft AnkleLeft FootLeft HipRight KneeRight AnkleRight FootRight JointIDenum

  16. privatevoidWindow_Loaded(object sender, EventArgs e) { _nuiRuntime = newRuntime(); try { _nuiRuntime.Initialize(RuntimeOptions.UseSkeletalTracking); } catch (InvalidOperationException) { MessageBox.Show("Runtime initialization failed. Please make sure Kinect device is plugged in."); return; } _nuiRuntime.SkeletonFrameReady += newEventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady); } Code(open IDE)

  17. Aura Light Saber User in 3D Ideas Stereo Photo

  18. Blind/Deaf support • Finger language • Controlling avatar • User 3D model creation and export toXNA / 3DS Max / Maya • Individual training programs Ideas

  19. Higher resolution • Lower response time • Better accuracy • NUI + Augmented Reality What’s next?

  20. http://is.gd/kinect_howitworks_video • http://is.gd/ms_research (ms scientific publication) • http://is.gd/kinect_sdk • http://is.gd/kinect_and_bunny (video of the demo) References

  21. ??

More Related