Automated Rivet Inspection System for Aging Aircrafts

1. Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University

2. Contents • Nondestructive Inspection (NDI) • Magneto-optic Imager in NDI • Motion-based Filtering (MBF) • Real-time implementation of MBF • Automated rivet inspection system • Rivet detection • Rivet classification • Results and conclusions • Future work

3. Nondestructive Inspection for Aircrafts • Detect subsurface defects Seam Rivet Crack

4. Nondestructive Inspection for Aircrafts • Increase service life of airplane • Prevent disasters Aloha Airlines B-737-200 lost part of its front fuselage during a flight in Hawaii, 1985

5. CCD Camera Light Source Polarizer Analyzer MO Sensor Induction Sheet Bias Coil Sample Magneto-optic Imager (MOI) • Eddy current excitation • Magneto-optic sensing • Imaging

6. Magneto-optic Imager (cont.) • Produce real-time analog images of inspected part • Images both surface breaking and subsurface cracks • Easy to interpret with minimal training • Applicable both on conducting samples as well as composites by tagging with ferromagnetic particles

7. Sample MOI Images Crack along seam Crack between two rivets, Radial crack on a rivet Corrosion dome

8. Drawback of MOI images • MOI image contains serpentine pattern noises due to the magnetic domain walls in magneto-optic sensor Signals due to domain walls Rivet

9. Moving direction of objects Moving direction of MOI Motion-based Filtering (MBF) • Additive Frame Subtraction In-5 In-4 In-3 In-2 In-1 In D5 D4 D3 D2 D1  Sn

10. Input Image Preprocessing RGB to Gray Additive Frame Subtraction Threshold Post processing Median Filter Stretch Output Image Motion-based Filtering (cont.)

11. MB filtered images Original Filtered

12. MB filtered images (cont.) Original Filtered

13. Frame grabber Record to VHS Collects MOI image Data Web camera Real-time implementation of MBF • Experimental setup for proof of concept Play on a Video player Record to movie file noise Play on a PC monitor

14. Real-time MBF MOI Max Sensing RGB to Gray Subtract Imaging Max Displaying Stretch Median Filtering Threshold Real-time implementation of MBF (cont.) • Data transfer rate • 4.6 Mbytes /sec ( 320240 pixels  16 bit  30 fps ) • Data are down sampled as the input images are dropped while an image is processed • Diagram of real-time Motion-based Filtering

15. Image capture: 20 ms RGB to Gray: 20 ~ 23 ms Additive Frame Subtraction: 1~2 ms/image Threshold: 1 ~ 2 ms Median Filter: 200 ~ 250 ms Stretch: 1 ~ 2 ms Output Image Real-time implementation of MBF (cont.) • Optimizing MBF algorithm in C++

16. Real-time implementation of MBF (cont.) • RGB to Gray conversion • Additive frame subtraction • MAX(I1-I3,I2-I3)  MAX(I1,I2) - I3 • Median filter

17. Real-time implementation of MBF (cont.) • Execution time of MBF algorithm in C++

18. Drawbacks of current MOI inspection • No measure for quantitative interpretation • Data interpretation is subjective • Manual inspection by human operator (more than 10 hours per airplane) • Expensive labor cost • Error due to fatigue

19. Automated MOI inspection system • PRI Research and Development Corporation (PRI) • Developing and improving magneto-optic imager (MOI) • Michigan State University, ECE department • Image processing algorithm for filtering and classification • Boeing Phantom Works • Self-guided, suction cup robot – crawls over airplane skin

20. Motion-based Filtering Rivet detection Rivet classification Automated MOI inspection (cont.) • Currently focusing on radial cracks on rivets • Quantification of defects in MOI images • Implementing real-time rivet inspection algorithm

21. Segment out each rivet Obtain center and radius, c1, r1 Erode rivet with a circle of radius r1 yes, r1 r1-1 Area(rivet) = 0 no Obtain center and radius, c2, r2 Rivet detection • Hough transformation-based method • Circular Hough transformation • Morphological operation-based method

22. Rivet detection (cont.) Hough transformation Original Morphological operation

23. Rivet classification • Two-pass Hough transformation • 1st pass – Rivet detection • 2nd pass – Blob detection good bad Original image Filtered image - After 1st pass After 2nd Pass

24. Rivet classification (cont.) • Bayesian classifier • Feature selection Original image Hough transformation Morphological op.

25. Off-line test • Training • 10 normal, 10 defective rivet images • Obtain mean and variance of feature f1 • Testing • 222 rivet images including 66 defective rivet images Two-pass Hough Hough - Bayes Morph. - Bayes

26. Experimental Results • Accuracies of three algorithms

27. Conclusions • MB filtered image is optimal in image processing for automated rivet inspection • Morphological operation-based rivet detection is superior to Hough-based rivet detection both for execution time and accuracy • Bayesian classifier is superior to Hough-based classifier • Radial crack detection on rivets showed 99% accuracy in off-line test

28. Future work • Implement MBF and rivet inspection algorithms on the Digital Signal Processing (DSP) board • Improve robustness of the algorithms with the feedback from field test • Develop MOI inspection algorithms for other types of defects in aircrafts