NON-DESTRUCTIVE TESTING AND EVALUATION TECHNIQUES  My Past, Present, and Future Research

1. NON-DESTRUCTIVE TESTING AND EVALUATION TECHNIQUES My Past, Present, and Future Research Dr. Guang-Ming Zhang General Engineering Research Institute, Liverpool JMU October 10, 2008

2. Outline of Research Experience • April 2003 - present – Research Fellow in General Engineering Research Institute, Liverpool John Moores University, Liverpool, UK • March 2001 – March 2003 – Researcher in Signals and Systems Group, Uppsala University, Uppsala, Sweden • Sep. 1999 – March 2001 – Postdoc in Institute of Acoustics, Nanjing University, Nanjing, P.R.China. (Promoted as an Associate Professor in Nov. 2000) • Sep. 1996 – June 1999 – Ph.D in Laser and Infrared Research Institute, Xi’an Jiaotong University, Xi’an, P.R.China, Full time • Sep. 1993 – June 1996 – MSc in Laser and Infrared Research Institute, Xi’an Jiaotong University, Xi’an, P.R.China, Full time

3. MSc Research supported by industry. Xi’an Jiaotong University Development of JTUIS Ultrasonic Imaging Testing System My main contributions include: initiating the system plan and structure, development of NDT techniques, designing a PC based data acquisition and control circuit board and software programming. This product has been patented and commercialized. It has been sold in more than eighteen companies including big state-owned companies in China and multi-national companies for example ABB, and companies in Indian.

4. Development of JTUIS Ultrasonic Imaging Testing System

5. Applications Low Voltage Switch High Voltage Vacuum Switch High Voltage Switch Sintered Contact Ultrasonic inspection

6. PhD Research. Xi’an Jiaotong University Theory and Application Study on Time-Frequency Analysis Technology for Ultrasonic NDT • Integrating neural networks techniques for recognition and classification of ultrasonic signals and defects. • Continuous wavelet transform and parameter optimization in ultrasonic non-destructive evaluation. • Adaptive time-frequency decomposition • Flaw identification and noise suppressing using Wavelet Ridge. • 2D wavelet packet transform for image enhancement. • Ultrasonic non-destructive evaluation of weld defects.

7. Xi’an Jiaotong University Infrared Thermography • Developed an infrared image collecting and processing system. It was designed for AGEMA series thermovision. This product has been commercialized and was awarded a science and technology progress prize by National Education Ministry of P.R.China. It has been used in universities and petrochemical companies • Investigated pulse-heating infrared thermography. High power ultrasound was used as the heating source. • Carried out composite material non-destructive testing using the infrared thermal imaging technique.

8. Nanjing University Acoustical Sensors and Actuators Based on MEMS Techniques • Supported by the Natural Science Foundation of China (equivalent to EPSRC). • Developed a surface acoustic wave rotation motor(size at millimetre–level). A prototype MEMS (Micro Electro-Mechanical System) motor has been designed and made using photolithography, and its basic characteristics have been studied by both experiments and computer simulation. • Fabrication and characterisation of ferroelectric thin film using magnetron sputtering.

9. Acoustical Sensors and Actuators Based on MEMS Techniques Schematic diagram of the miniaturized SAW rotary motor Time evolution of the angular velocity both in theoretical simulation and in experiment The stator is made by a 128° rotated y-cut x-propagation LiNbO3 substrate, on which two pairs of IDTs (Inter-digital transducer) are arranged in parallel in the direction of propagation. The rotor is composed of an aluminium disk with several steel balls around the circumference. The size of the stator is 17×11×0.4 mm3, and the size of the rotoris 9×0.7 mm3. With the operating frequency of 30 MHz and the driving voltage of 120 Vp–p, the motor can rotate at a speed of 270 rpm.

10. Ultrasonic Image Compression • Investigated transform and subband coding for ultrasonic Image compression. Part of the European 5th Framework Programme project of ‘Signal processing and improved qualification for non-destructive testing of aging reactors’ . Uppsala University 0.125 bpp Original signal JPEG Wavelet transform (JPEG200) A MATLAB toolbox Karhunen-Loève transform These RF signals were extracted from original and decompressed ultrasonic B-scan images

11. Ultrasonic Image Compression • Carried out research on simultaneous denoising and compression of ultrasonic images using vector quantization of image subbands. The general VQ-based coding scheme The proposed coding scheme 0.47bpp

12. Supported by SKB (the Swedish Nuclear Fuel and Waste Management Co). Uppsala University Inspection of Copper Canisters for Spent Nuclear Fuel • Studied nonlinear acoustical imaging techniques. Phased array Allin ultrasonic imaging system, and RITEC advanced ultrasonic measurement system RAM-5000 were used to carried out experiments. A precision network analyzer from Agilent was used to analyze transducer characteristics. • Performed modelling and computer simulation of nonlinear acoustical fields in immersed solids. • Designed a software called DREAM for acoustic field simulation using discrete representation array modelling

13. Nonlinear acoustical imaging for NDE A C-scan image of the electron beam welding test piece obtained from the fundamental wave The second harmonic image

14. Calibration An ultrasonic pulse measured by a hydrophone at 31mm away from the transducer in water, and its spectrum. It can be seen that nonlinearity in water can be neglected in our experiment. An ultrasonic pulse reflected from a side-drilled hole and its spectrum. Nolinearity is clearly observed.

15. Modelling and computer simulation of nonlinear acoustical fields in immersed solids Acoustical field from the 2.3-MHz spherically annular transducer operating in copper block immersed in water with a 30-mm water path length, with an initial source intensity 21W/cm2. Left: without accounting for the nonlinearity in water; Right: with accounting for the nonlinearity in water. Normalized second harmonic at the welded interface: 30.8854dB, 19.6513dB.

16. Acoustic field simulation using discrete representation array modelling http://www.signal.uu.se/Toolbox/dream/ (Free Download)

17. EPSRC ROPA project. Liverpool JMU Quality and reliability testing of BGA and Flip-Chip solder bonds subjected to mixed-environments testing using AMI • Carried out a comparison study of X-ray inspection and acoustic micro imaging for the evaluation of modern microelectronic packages. • AMI is an effective approach for detecting gap-type defects such as voids, delaminations and cracks due to the strong reflection of ultrasound in solid-air interfaces. These defects are difficult to be found by X-ray inspection owing to low contrast. The contrast of X-ray images relies on the thickness of internal structures/defects, and differences in the atomic mass and density. Thus, X-ray inspection is fit for volumetric defects for example broken wires. However, this kind of defect is very hard or impossible to be detected by AMI. • Was assessed as ‘tending to outstanding’

18. EPSRC ROPA project. Liverpool JMU Quality and reliability testing of BGA and Flip-Chip solder bonds subjected to mixed-environments testing using AMI AMI X-ray inspection X-ray inspection

19. Signal model in AMI (a) (b) AMI echoes at typical boundaries in Flip-Chip package mounted on ceramic substrate (a), and an example A-scan obtained using a 230MHz transducer (b). Ultrasonic signals reflected by defects possess information about defect size and orientation.

20. Imaging modes of ultrasonic imaging (a) A-scan B-scan C-scan

21. Multiple slices of flip-chip solder joints by AMI. From left to right: top slice (silicon-bump interface), middle slice (bump area), and bottom slice (bump-PCB). Lower part: corresponding electronic gate

22. Imaging technology for x-ray inspection • Conventional 2D transmission x-ray • Oblique angled viewing • 3D digital tomography • 3D laminography Principle of digital lexicography

23. Liverpool JMU Advanced Acoustic Micro-Imaging for the Evaluation of Microelectronic Packages • Background: • Modern PCBs and semiconductor devices are difficult to inspect • This will get harder as devices become smaller and more complex • 3D packages as shown in the above picture are emerging, bringing new problems that need solutions • The key acoustic challenges are axial resolution for delamination and cracks at closely-spaced interfaces and penetration through multiple interfaces.

24. Limitations of Conventional AMI Techniques Left: by a 230 MHz transducer; Right: by a 50 MHZ transducer. (Dimension of the flip-chip package is 8.19mm8.30mm0.86mm)

25. Advanced AMI Techniques Basic Principle: • Firstly, performing a-priori selection of a possibly over-complete signal dictionary (a collection of parameterised waveforms) in which the ultrasonic pulses are assumed to be sparsely representable. • Secondly, separating the incident pulses by exploiting their sparse representability. • Thirdly, selecting an appropriate echo and producing a C-scan output.

26. Time-Frequency Domain AMI Techniques Acoustic time-frequency domain imaging. (a) Time-frequency representations of Fig.2a by CWT; (b) Wavelet transform modulus maxima; (c) Significant local maxima obtained by wavelet thresholding.

27. Sparse Signal Representation Based AMI Approaches What is sparse signal representation? The sparse signal representation problem is formulated as: Given a signal , and the overcomplete dictionary seek the sparsest coefficient vector . under the linear model This corresponds to solving the following variational problem: Minimize subject to It is an NP-hard problem. So many approximation solutions were proposed.

28. Sparse Signal Representation Based AMI Approaches • What are the advantages of sparse signal representation? • Recently sparse overcomplete representation is of great interest in many applications such as image compression, denoising of signals, and blind source separation because of its advantages. • One is that there is greater flexibility in capturing structure in the data. Instead of a small set of general basis vectors, there is a larger set of more specialized basis vectors such that relatively few are required to represent any particular signal. These can form more compact representations, because each basis vector can describe a significant amount of structure in the data. • The second is super-resolution. We can obtain a resolution of sparse objects that is much higher than that possible with traditional methods. • An additional advantage is that overcomplete representations increase stability of the representation in response to small perturbations of the signal. • In addition, the redundant representations have the desired shift invariance property.

29. Sparse Signal Representation Based AMI Approaches • Sparse signal representation methods • Basis Pursuit (BP), Matching Pursuit (MP), and Best Orthogonal Basis (BOB) (Wavelet Packet decomposition). • 2) Over-complete dictionaries • Wavelet packets dictionaries, Cosine packet dictionaries, and Gabor dictionaries. • 3) SSRAMI • BP base AMI, MP based AMI, and BOB based AMI.

30. Sparse Signal Representation Based AMI Approaches The echo separation result of an A-scan shown in phase plane and a succession of time-frequency windows. The darkness of the time-frequency image increases with the energy value, and each time-frequency atom selected by the BP method is represented by a Heisenberg box.

31. Sparse Signal Representation Based AMI Approaches The AMI results with simulated A-scans by different AMI techniques.

32. Learning Overcomplete Representation Based AMI Approaches • The success of SSRAMI greatly depends on the overcomplete dictionary. There are several ways to determine the overcomplete dictionary: • To choose a non-adapted overcomplete dictionary from the existing dictionaries, such as Gabor dictionary, and wavelet packet dictionaries. • To learn an adapted overcomplete dictionary. • Dictionary learning methods: Principal Component Analysis (PCA), Independent Component Analysis (ICA), overcomplete ICA, and FOCUSS-based dictionary learning, and recently developed K-SVD. • 3. To combine dictionaries to make bigger, more expressive dictionaries.

33. Learned Overcomplete Dictionaries Learned basis vectors. Left: 50MHz transducer; Right 230MHz transducer.

34. Learning Overcomplete Representations Learning overcomplete representation techniques: Basis Pursuit (BP), Matching Pursuit (MP), Overcomplete ICA, FOCUSS, and Sparse Bayesian Learning (SBL). Sparse representations of ultrasonic signals: (a) A simulated A-scan; the sparse representations of (a) by the overcomplete ICA (b), FOCUSS (c) and SBL (d) algorithms ; (e) the original echo [the first echo in (a)]; (f) the recovered echo from (b); (g) the recovered echo from (c); (h) the recovered echo from (d).

35. Sparse Deconvolution • Investigated sparse deconvolution of ultrasonic NDE Traces accounting for pulse variances by learning overcomplete representations. Deconvolution of (a) a synthetic non-stationary trace segment (SNR=8.6dB). (b) The true reflectivity (b). The reflectivity recovered by (c) the proposed method and (d) the IWM-BD method. Upper: true and estimated local pulses for the first and middle echoes by the proposed method. Solid lines devote the true pulses and dashed lines devote the estimated pulses. Lower: by the IWM-BD method.

36. Ultrasonic Flaw Detection via Overcomplete and Sparse Representations 1. Signal detection for signals with additive white Gaussian noise The proposed algorithm WTSP

37. Ultrasonic Flaw Detection via Overcomplete and Sparse Representations 2. Signal detection for signalswith correlated noise Correlated noise and an original ultrasonic flaw signal The proposed algorithm WTSP

38. Ultrasonic Flaw Detection via Overcomplete and Sparse Representations 3. Estimate the pulse arrival time for defect location, thickness measurement, and sound velocity determination. The error of measure delay time between two pulses against the SNR. Results are presented in sample number.

39. Ongoing EPSRC Project Super-Resolution Acoustic Time-Frequency Domain Imaging Techniques 1.Simulation of Acoustic Microimaging • To elucidate the defect detection mechanism and predict ultrasonic pulses. • To investigate the basic ultrasound limitation – penetration vs. resolution in modern 3D packages. • To optimize the parameters of transducer focusing. • To study the ultrasound nonlinearity. 2.Super-resolution Acoustic Microimaging Approaches • To achieve a sub-wavelength axial resolution. • To study sparse representation based AMI approaches. • To develop fast time-frequency domain AMI approaches. • To develop deconvolution-based AMI • To study 3D AMI

40. Other Related Work • Worked on development of INTRANET network software under the IBM AS-400 computer operating system in Primax Manufacturing LTD, Guangdong, P.R.China in June-September, 1996. • Involved in research of photoacoustic imaging and laser ultrasonics techniques at Nanjing University. • Worked on the ‘TELEPATH’ – an EU funded e-learning project involved in partners from UK, Portugal and China at EDC. • Provided consulting help to local (Merseyside) companies under the EU ERDF funding at EDC. • Co-supervised postgraduate students and Ph.D students.

41. GRANTS AND PROPOSALS Super-resolution acoustic time-frequency domain imaging techniques for the evaluation of modern microelectronic packages. EPSRC responsive mode, Granted (£200,000), 2008 July (Researcher Co-I). Micro-electrical-mechanical ultrasound phased array system. Supported by Postdoctoral Science Foundation of China, 2000 (PI). Ferroelectric thin film manufacturing and study on structure and performance. Supported by State Key Lab. Science Foundation of Nanjing University, 2000 (PI). Nanoscale tomographic imaging of buried structures. ERC Starting Independent Researcher Grant, not granted, April 2007 (PI) (Very good comments were received). Characterisation of future packages using advanced imaging techniques. EPSRC responsive mode, missed by one positions, March 2007 (Co-I). Advanced imaging approaches for diagnosis of electronic assemblies. EPSRC Advanced Research Fellowship, I was invited for interview and the proposal was ranked 11 in the final stage but unfortunately only 9 have been granted, Sep. 2006 (PI).

42. Summary 1) I have developed significant expertise in a broad research area by working in different universities in different countries: • Ultrasonic non-destructive evaluation, ultrasonic engineering, signal and image processing, pattern recognition, information theory, instrumentation development and electronic design. • Xi’an Jiaotong University - Nanjing University - Uppsala University – LJMU • China – Sweden – UK 2) As main researcher I have been involved in multiple research projects supported by Natural Science Foundation of China, European Framework Programme, Sweden, EPSRC, and industries.

43. Summary 3) Have gained experience in project management: As principal investigator managed two projects. Co-managed 3 projects. Co-supervised several postgraduate students and PhD students. 4) Strong research and product development abilities: • Published 40 refereed journal papers (25 as first author), and 7 conference papers (6 as first author). • 1 patent. • 2 products commercialised. • A number of awards have been awarded by Chinese government and other organisations. • Two invited presentations. (Invited by the vice Chancellor of Xi’an Science and Technology University, and by the head of Institute of Acoustics, Nanjing University, in 2005.)

44. Thank You for Your Attention