1 / 24

Kinect Color Image Retrieval and Manipulation

Learn about important classes for retrieving and setting color image formats, capturing frames on demand, saving frames, and extending the KinectCam application with more controls. Also, explore adjusting elevation angles and building KinectCam 2.0.

cathiew
Download Presentation

Kinect Color Image Retrieval and Manipulation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. EEC-492/592Kinect Application Development Lecture 5 Wenbing Zhao wenbing@ieee.org

  2. Outline Important Classes on Color Image Retrieving/Setting Color Image Format Capturing Frames on Demand Frame Number Frame rate Saving frames Extend the KinectCam app with more controls

  3. Important Classes on Color Image • We have dealt with two classes related to Kinect color images • ColorImageStream: handles a stream of ColorImageFrame • ColorImageFrame • ColorImageStream • this.sensor.ColorStream.Enable(); • ColorStream is of ColorImageStream type • ColorImageFrame • ColorImageFrame imageFrame = e.OpenColorImageFrame()

  4. ColorImageStream Sealed class: no other class can inhert a sealed class

  5. ColorImageFrame

  6. More on ColorImageFrame

  7. Retrieving/Setting Color Image Format • Can retrieve the color image format for the current image frame using the ImageFrame.Format property • public ColorImageFormat ImageFormat {get;set;} • Retrieve format • this.ImageFormat = imageFrame.Format; • Setting format • this.sensor.ColorStream.Enable(RgbResolution640x480Fps30);

  8. Capturing Frames on Demand • Polling: request an image frame on demand • ColorImageStream::openNextFrame() • OpenNextFrame() accepts one parameter specifying how long the sensor should wait before it sends the image frame • If there is no frame ready within the provided time, the method will return null int millisecondsWait = 10; if (this.sensor.ColorStream.IsEnabled) { ColorImageFrame colorImageFrame = this.sensor.ColorStream. OpenNextFrame(millisecondsWait); }

  9. Frame Number • Frame number: Every frame has a number to identify the frame • This number is incremented with each new frame generated • It is read only in image frame • int frameNumber=imageFrame.FrameNumber;

  10. Frame Rate Calculation private int TotalFrames; private DateTime lastTime = DateTime.MaxValue; private int LastFrames; int currentFrameRate = 0; private int GetCurrentFrameRate() { ++this.TotalFrames; DateTime currentTime = DateTime.Now; var timeSpan = currentTime.Subtract(this.lastTime); if (this.lastTime == DateTime.MaxValue || // first time to run timeSpan >= TimeSpan.FromSeconds(1)) // average 1 second { currentFrameRate = (int)Math.Round((this.TotalFrames - this. LastFrames) / timeSpan.TotalSeconds); this.LastFrames = this.TotalFrames; this.lastTime = currentTime; } return currentFrameRate; }

  11. Saving Color Image private void SaveImage() { using (FileStream fs = new FileStream(string.Format("{0}.jpg", Guid.NewGuid().ToString()), System.IO.FileMode.Create)) { BitmapSource imageSource = (BitmapSource)image1.Source; JpegBitmapEncoder jpegEncoder = new JpegBitmapEncoder(); jpegEncoder.Frames.Add(BitmapFrame.Create(imageSource)); jpegEncoder.Save(fs); fs.Close(); } }

  12. Saving Color Images Periodically • Call the SaveImage () method on a Tick event of DispatcherTimer • Automatic periodic image saving. 4 steps. • Step1: Define the DispatcherTimer object under the System.Windows.Threading namespace: • private DispatcherTimer timer = new DispatcherTimer();

  13. Saving Color Images Periodically • Step 2: Write the start method with an interval of 10 seconds and attach the Tick event handler: public void StartTimer() { this.timer.Interval = new TimeSpan(0, 0, 10); this.timer.Start(); this.timer.Tick += handleTickEvent; } // handleTickEvent: event handler delegate to be implemented

  14. Saving Color Images Periodically • Step 3: On the Tick event handler, call the SaveImage() method: public void handleTickEvent(object sender, object e) { if (this.sensor.IsRunning && this.sensor.ColorStream.IsEnabled) { this.SaveImage(); } } • Step 4: To launch the timer thread, call timer.StartTimer();

  15. Saving Color Images Periodically • To stop timer public void StopTimer() { this.timer.Stop(); this.timer.Tick -= this.handleTickEvent; }

  16. Adjusting Elevation Angle • Kinect elevation angle can be changed between • MaxElevationAngle (27 degrees) • MinElevationAngle (-27 degrees) • Adjusting angle private void SetSensorAngle(int angleValue) { if(angleValue > sensor.MinElevationAngle || angleValue < sensor.MaxElevationAngle) { this.sensor.ElevationAngle = angleValue; } }

  17. Build KinectCam2.0 • Extend the KinectCam we built last time • Revise GUI design • Adding image controls • Add code to handle image controls

  18. Revised GUI

  19. Settings • Changing frame format on the fly • RgbResolution640x480Fps30, enable this by default • RgbResolution1280x960Fps12 • Customize the checkbox names and the event handlers this time private void ChangeToHighResolution(object sender, RoutedEventArgs e) { if (this.sensor != null) { this.normalresolution.IsChecked = false; this.sensor.ColorStream.Enable( ColorImageFormat.RgbResolution1280x960Fps12); this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength]; } }

  20. Settings • Changing frame format on the fly • Enable one, and disable the other • Need to reset the colorPixels array length for different formats private void ChangeToNormalResolution (object sender, RoutedEventArgs e) { if (this.sensor != null) { this.highresolution.IsChecked = false; this.sensor.ColorStream.Enable( ColorImageFormat.RgbResolution640x480Fps30); this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength]; } }

  21. Settings • Displaying frame rate • Calculated and displayed for each frame received • At the end of void colorFrameReady(object sender, ColorImageFrameReadyEventArgs e) • textBox1.Text = ""+GetCurrentFrameRate(); • Changing elevation angle private void IncrementAngle(object sender, RoutedEventArgs e) { int angleValue = this.sensor.ElevationAngle + 1; if (angleValue < sensor.MaxElevationAngle) { this.sensor.ElevationAngle = angleValue; this.AngleBox.Text = "" + angleValue; } } private void DecrementAngle(object sender, RoutedEventArgs e) { int angleValue = this.sensor.ElevationAngle - 1; if (angleValue > sensor.MinElevationAngle) ……

  22. Settings • Save image on demand (Save Image button) private void SaveImage(object sender, RoutedEventArgs e) { SaveImage(); } private void SaveImage() { using (FileStream fileStream = new FileStream(string.Format("{0}.jpg", Guid.NewGuid().ToString()), System.IO.FileMode.Create)) { BitmapSource imageSource = (BitmapSource)image1.Source; JpegBitmapEncoder jpegEncoder = new JpegBitmapEncoder(); jpegEncoder.Frames.Add(BitmapFrame.Create(imageSource)); jpegEncoder.Save(fileStream); fileStream.Close(); } }

  23. Settings • Save image periodically (via checkbox) • Make sure you designate event handler for both checked and unchecked event!

  24. Settings private void PeriodicSavingChanged (object sender, RoutedEventArgs e) { if (checkBox1.IsChecked == true) { StartTimer(); } else { this.timer.Stop(); } } private DispatcherTimer timer = new DispatcherTimer(); public void StartTimer() { this.timer.Interval = new TimeSpan(0, 0, 10); this.timer.Start(); this.timer.Tick += this.handleTickEvent; } public void StopTimer() { this.timer.Stop(); this.timer.Tick -= this.handleTickEvent; } public void handleTickEvent(object sender, object e) { if (this.sensor.IsRunning && this.sensor.ColorStream.IsEnabled) { this.SaveImage(); } } • Save image periodically (via checkbox)

More Related