1 / 45

GPGPU-based surface inspection from structured white light

GPGPU-based surface inspection from structured white light. Jari Hannuksela, Olli Silvén Machine Vision Group, Infotech Oulu Department of Electrical and Information Engineeering University of Oulu, Finland. Miguel Bordallo 1 , Karri Niemelä 2 , Olli Silvén 1

vivien-william
Download Presentation

GPGPU-based surface inspection from structured white light

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GPGPU-based surface inspection from structured white light Jari Hannuksela, Olli Silvén Machine Vision Group, Infotech Oulu Department of Electrical and Information Engineeering University of Oulu, Finland Miguel Bordallo1, Karri Niemelä 2, Olli Silvén1 1 Center for Machine Vision Research - University of Oulu, Finland 2 VTT - Technical Research Center of Finland, Oulu, Finland

  2. Contents Introduction Automatic Surface Inspection • Phase extraction from white structured light • Practical problems Measuring Prototype • Design and construction • GPU as a computing engine • Experimental setup Description of the system • Algorithms and Implementation Experiments • Qualitative results • Speed and scalability Summary

  3. Motivation • Automatic surface inspection used in the industry: • To detect all kinds of surface defects • To measure the overall quality of a produced piece • Most convenient inspection method should provide exact 3D information • High speed of production lines need: • Fast imaging methods • Lots of computational power • Systems must be cost effective: • Standard PCs • Graphics Processing Units (GPUs)

  4. GPU as a computing engine • Allcomputers and manyembedded systems include a GPU • Standard PCs and components • Cost-effective systems • Highlyscalable • GPU canbetreated as an independententity • Graphics ProcessingUnitsofferimportantparallelizationcapabilities • GPUsoffer ”many-core” computation • Thousands of threadscanbeexecutedconcurrently. • GPU and CPU canbeusedconcurrently • If data transfer is small, CPU loadremainslow (CPU canbeused for othertasks) • CUDA is a highlyoptimized and attractiveacceleratorinterface

  5. Surface topography from white structured light (SLS) • Phase-shifting methods: • Based on fringe pattern projections or structured light • Extensively utilized in topography measurement • Provide for high resolution height measurements on each pixel. • The illuminator projects a sine pattern: • On a moving target • In a synchronized manner • The camera system obtains suitable input pictures using: • Pulse-like illumination • Synchronized camera subsystem • Certain known rate

  6. Phase measurement The input images are defined by the following: If δ1, δ2, δ3 are known: And the height: In practice δ1, δ2, δ3 are not known in beforehand

  7. Phase extraction with syntetic images 120dg Phase Shifted patterns Reconstructed Images Phase/Height Comparison

  8. Problems and errors + + Clipping effect: saturation Wrong phase shift (δ1, δ2, δ3) Wrong frequency = Combined effect

  9. Problems and errors + + Clipping effect: saturation Wrong phase shift (δ1, δ2, δ3) Wrong frequency = + Combined effect Input

  10. Problems and errors + + Clipping effect: saturation Wrong phase shift (δ1, δ2, δ3) Wrong frequency = + = Combined effect Result Input

  11. Prototype design

  12. Prototype design • VTT prototype: Sine period of 250um • Camera: Basler Scout scA 1600-14gm. 1628x1236 pixels, Area 4.4*4.4um2 • Interface: GiGE, 17 frames per second • Optics: Optosigma Telecentric (TC1236). Pixel size 30 µm • Illuminator: 9 Luxeon K2 Red LEDS + collimating lens. 3 channels • Laptop: Lenovo W700 • CPU: Intel Core 2 Extreme QX9300 2.53 GHz • GPU: Nvidia Quadro FX3700 (128 cores) • IDE: Visual Studio. CUDA & C code environments • Motor Line Controller: ATMEL microcontroler and PC • Line speed: 0,3 m/s • Samples used: • Offline: 10 cents coin, printed electronics (10 µm thick) • Online: MDF-fiberboard

  13. Prototypeconstruction

  14. Application flow

  15. Application flow

  16. Application flow

  17. Application flow

  18. Application flow

  19. Application flow

  20. Application flow

  21. Application flow

  22. Input images Full frame size: 1628x1236 pixels, 8 or 10 bpp, grayscale, 17 fps

  23. Input images Full frame size: 1628x1236 pixels, 8 or 10 bpp, grayscale, 17 fps 64x256 correlation area

  24. Image registration • Based on modified phase correlation • Tukey window + FFT-based (+ Gaussian filtering) • Robust to blur (even motion blur) • Robust to image intensity changes • Fast to compute • Easy to parallelize • CUDA FFT routines already optimized • Per-pixel operations • Identifies corresponding pixels • Subpixel level access as a CUDA texture object • Predict initial phase shift for phase computation • Fine tune the motor displacements & camera rate 600x300 ROIs

  25. Correlation algorithm performance

  26. Advanced Phase Shifting Algorithm (APSA) • First introduced by Z. Wang in 2004 • Iterative algorithm • Initial estimation of phase difference (δ1, δ2, δ3) • from correlation and previous frames • Phase of each pixel is computed • Using a CUDA 2-dimensional kernel • Average phase of the image is computed • By adding together the values of all the pixels • Using CUDPP parallel reductions • Average phase is the new phase difference • Iterate until convergent and error < threshold • Result is a phase wrapped image • Range between -πand π Wrapped image

  27. APSA times

  28. Phase unwrapping and surface fitting 600x300 Surface map • Lp Norm algorithm: • Developed in CUDA (Mistry, 2009) • Accurate results • Very high computation times (up to 2.5 seconds) • Not suitable for real-time • Sorting by reliability in noncontinuous path: • Fast two dimensional unwrapping • Developed in C for a CPU (Arevalillo 2004) • Sufficient accuracy • Very fast (about 125 ms.) • Can be executed concurrently with the GPU phase extraction • Surface fitting computes the closer average plane

  29. Display system

  30. Automatic calibration system • Phase maps measured continuously in real time • The information of the phase extraction process can be used to improve further results and conditions. • Synchronizes • Illumination, • Camera capture • Motor speed • Input parameters: • Correlation results (adjust motor speed) • Phase average (adjust illumination and camera capture Phase tuning and system calibration improve the results gradually

  31. Real-time tests: MDF fibreboard sample

  32. Real time tests: 3D representation

  33. Printed electronics sample

  34. Complete system CPU GPU Get input frames N = 1 CopyImages as texture GetCorrelation ROI Performcorrelation Forward correlationvalues GetSurface ROI Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel. Get pixel phase Getaveragephase APSA1 Getaveragephase APSA2 Forward phaseaveragevalues Getphasemap Phaseunwrapping Surfacefitting

  35. Complete system CPU GPU Get input frames N = 1 CopyImages as texture GetCorrelation ROI Performcorrelation Forward correlationvalues GetSurface ROI Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel. Get pixel phase Getaveragephase APSA1 Getaveragephase APSA2 Forward phaseaveragevalues Getphasemap Phaseunwrapping Surfacefitting

  36. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel. Phaseunwrapping Surfaceftting

  37. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Phaseunwrapping Surfacefitting N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel.

  38. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Get input frames N = 2 Phaseunwrapping Surfacefitting N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel.

  39. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Get input frames N = 2 Calculate wrappedphase Image N = 2 Phaseunwrapping Surfacefitting N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel.

  40. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Get input frames N = 2 Calculate wrappedphase Image N = 2 Phaseunwrapping Surfacefitting N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel. Get input frames N = 3 Calculate wrappedphase Image N = 3 Phaseunwrapping Surfacefitting N = 2

  41. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Get input frames N = 2 Calculate wrappedphase Image N = 2 Phaseunwrapping Surfacefitting N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel. Get input frames N = 3 Calculate wrappedphase Image N = 3 Phaseunwrapping Surfacefitting N = 2

  42. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Get input frames N = 2 Calculate wrappedphase Image N = 2 Phaseunwrapping Surfacefitting N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel. Get input frames N = 3 Calculate wrappedphase Image N = 3 Phaseunwrapping Surfacefitting N = 2 Get input frames N = n Calculate wrappedphase Image N = n Phaseunwrapping Surfacefitting N = n-1

  43. Complete system CPU GPU Get input frames N = 1 Calculate wrappedphase Image N = 1 Get input frames N = 2 Calculate wrappedphase Image N = 2 Phaseunwrapping Surfacefitting N = 1 Image size: 3 ROI of 600x300 Computation time: < 150 ms. Frame rate: > 5 fps. Resolution: 30µm per pixel. Get input frames N = 3 Calculate wrappedphase Image N = 3 Phaseunwrapping Surfacefitting N = 2 Get input frames N = n Calculate wrappedphase Image N = n Phaseunwrapping Surfacefitting N = n-1

  44. Summary • A sine projection technique is a suitable method to optically measure a layer-like surface topography • The system could be used in rapid motor lines with proper synchronization • An integrated automatic calibration system helps synchronization and increases quality and robustness • High accuracy can be achieved with fast imaging methods and intensive computation • Time critical algorithms can be executed with GPU-based parallel computing

  45. Thank you! • Any questions ???

More Related