Signals and Processing

1. ASU MAT 591Image Processing Scienceand Robotic VisionRod PickensPrincipal Research EngineerLockheed Martin, Incorporated

2. Signals and Processing • Signals • Analog and discrete signals • Dimensionality of signals • 1-D signals • Sounds (temporal), echocardiogram, seismic signal • 2-D signals (this presentation) • Images (spatial) • 3-D signals • Video sequences of images (spatial and temporal) • Signal processing • Synthesize and analyze signals • Filter signals using low-pass, band-pass, and high-pass filter • Modify signals such as warp, delay, stretch, rotate, shrink, … • Restore and enhance signals • Recognize patterns and detect signals

3. Signal Processing: Now Animal Robotic Touch Touch Vision Vision Taste Taste Hearing Hearing Smell Smell

4. The Processing Analogy

5. Fourier Synthesis White Light In White Light Out Fourier Analysis Inverse Functions Analysis and Synthesis of Light

6. Fourier Transforms are Inverse Functions

7. Derivative Inv Fourier Trans Inv Radon Trans Warp Correction Integral Fourier Transform Radon Transform Warp Data Inverse Functions

8. Filtering White Light In Filtering removes all but red colors Red Light Out

9. Television Television Stations 3, 5, 6, 13, 15, … Television Filtering removes all but Channel 6 Channel 6

10. Television Television Stations 3, 5, 6, 13, 15, … Television Filtering removes all but Channel 15 Channel 15

11. Radio Radio Stations Radio Stations 91.5, 96.9, 100.7 Radio Station 100.7 Filtering removes all but Station 100.7

12. Radio Radio Stations Radio Stations 91.5, 96.9, 100.7 Radio Station 96.9 Filtering removes all but Station 96.9

13. Vision Scene of a Room: walls, books, desks, chairs, windows,… Robot vision Book Filtering removes all but a book

14. Vision Scene of a Room: walls, books, desks, chairs, windows,… Scene of a Room Robot vision Table Filtering removes all but a table

15. Synthesis All Room Contents Descriptor of scene is D(w) Scene of a Room Graphics to build a scene

16. Signal Approximation of Signal Filter that eliminates less important data. Data compression

17. Data compression goal Signal Approximation of Signal Filter that eliminates less important data.

18. An Example of a Processing Architecture

19. Correct Errors Preprocess Restore Communications Normalize Remove Noise Remove Distortions Remove Sensor Effects Recognize Analyze Decompose Signals Label Signals The Example Architecture Format Format Correct Errors Preprocess Restore Data Recognize Analyze Descriptions Will Discuss in more detail!

20. Preprocess Format Correct Errors Preprocess Preprocess Restore Data Normalize Remove Noise Remove Distortions Analyze Recognize Descriptions

21. Fourier Analysis Clearer Output Image Noisy Input Image Fourier Synthesis Fourier Based Noise Filtering Mostly Noise so is Zeroed Mostly Signal Fourier Transform and Filter the Noise From Jason Plumb at http://noisybox.net/weblog/

22. Filtering and Enhancing Data Math to follow From Mathworks homepage at http://www.mathworks.com/

23. Filtering: Analysis Image Analysis

24. Filtering: Removing Noise Image Filtering: removes noise

25. Filtering: Synthesis Synthesis Image Enhanced

26. Image Enhanced Filtering: removes noise Filtering Synthesis Analysis

27. I2 = m*I1 I2 p(I1) p(I2) I1 I1 I2 Input Image Intensity Histogram Enhance (stretch) Using Linear Mapping Output Image Intensity Histogram (more contrast) I=Intensity Enhancing the Data: Linear map

28. Warping data Suppose we have unwanted camera motion. From Mathworks homepage at http://www.mathworks.com/

29. Warping data We can correct motion errors if we know motion model. From Mathworks homepage at http://www.mathworks.com/

30. Warping data From Mathworks homepage at http://www.mathworks.com/

31. Warping Correction is an Inverse Function Warping Correction Warping

32. y1 x1 y2 x2 Linear Algebra to Flip

33. y1 y2 y2=y1 x1 y1 x2 y2 x1 x2=- x1 x2 Linear Algebra to Flip

34. y2 y2 x2 x2 Linear Algebra to Flip y1 I(x1,y1) y2 y2=y1 x1 y1 x2 x1 x2=- x1

35. y2 y2 y2 x2 x2 x2 Linear Algebra to Flip y1 I(x1,y1) y2 y2=y1 x1 y1 x2 x1 x2=- x1 I(x2,y2)

36. Linear Algebra to Flip y1 I(x1,y1) y2 y2=y1 x1 y1 x2 y2 y2 y2 y2 x1 x2=- x1 x2 x2 x2 x2 I(x2,y2)=I(f(x1),g(y1))

37. Linear Algebra to Flip y1 y1 y1=y2 x1 y2 x1 y2 x2 x1=- x2 x2 I(x2,y2)

38. Linear Algebra to Flip y1 I(f-1(x2), g-1(y2)) y1 y1=y2 x1 y2 x1 y2 x2 x1=- x2 x2 I(x2,y2)

39. Linear Algebra to Flip y1 I(x1,y1)=I(f-1(x2), g-1(y2)) y2 y2=y1 x1 y1 x2 y2 x1 x2=- x1 x2 I(x2,y2)

40. Linear Algebra to Flip and Shrink y1 x1 y2 x2

41. Linear Algebra to Flip and Shrink y1 y2 y2 = -0.5 * y1 x1 y1 x2 y2 x2 = 0.5 * x1 x1 x2

42. Ifwe can determine f(), g(), f-1(), and g-1(),thenwe can correct camera motion! Correcting warped data (camera motion) From Mathworks homepage at http://www.mathworks.com/

43. Restoration Format Correct Errors Preprocess Restore Restore Data Remove Sensor Effects Recognize Analyze Descriptions

44. Restoring data for smear, optics,… From Mathworks homepage at http://www.mathworks.com/ Smear and optics can be viewed as filters that can degrade an image! Uses Linear Systems Theory Next

45. Restoration Restoring data for smear, optics,… From Mathworks homepage at http://www.mathworks.com/ Uses Linear Systems Theory Next

46. Restoration: Analysis Image Analysis

47. Filtering: Removing Smear Image Smr-1(wx,wy) is a filter that removes smear or restores the original object.

48. Object Filtering Smear inverted as a filter Image Image Restored to best look like original Object

49. Restoring data for smear, optics,… From Mathworks homepage at http://www.mathworks.com/ Uses Linear Systems Theory Image(wx,wy) Next

50. Restoring data for smear, optics,… From Mathworks homepage at http://www.mathworks.com/ Smr(wx,wy)*Image(wx,wy) Uses Linear Systems Theory Image(wx,wy) Next