The Basic Theory of Filtering

1. The Basic Theory of Filtering James D. Johnston Audio Architect Microsoft Corporation Steve R. Hastings Software Developer boB Gudgel Self-described Geek

2. Coffee and Tea • Make sure your cup is full • Take a deep breath • Here we go!

3. The syllabus • Morning: • What is a filter • Why does a filter “filter”? • What is this time/frequency thing? • Impulse response • FIR vs. IIR • Convolution? But why? • What are some filter properties? • Things to consider when using a filter • What do I need to know to design a filter? • What is a filterbank, and why do I care?

4. Afternoon: • Steve: • Where can I get some freeware to design filters? • How do I install it? • How do I use it? • How do I figure out how to build my filters? • boB • What does filtering sound like? • What happens when you get it wrong? • How did I implement this, anyhow?

5. Key Points • What does a simple analog filter do? • What does a simple digital filter do? • Impulse response? Why impulse response? • Convolution? Huh? Why does this matter? • Fourier and Z-transform relationships: Convolution vs. multiplication (N.B. No actual transforms are discussed) • Filters: Zeros vs. poles • FIR vs. IIR • Phase response vs. Symmetry • What can be FIR? What can’t be FIR? • Frequency response vs. length • Basic implications • Filter examples

6. More Key Points • What kind of impulse response implies what kind of performance? • What if we want a set of filters • That’s a filterBANK • That’s an entire field of study itself.

7. What is a filter? Simply put, it retains history of the input signal. Take this simple lowpass filter for an example. Vout Vin This simple filter is a kind of integrator, in which the capacitor integrates the charge provided through the resistor. As the rate of change of Vout depends on the current through the resistor, low frequencies will be filtered less than high frequencies.

8. In other words: • The output of a filter is a function not only of the input at the present time, but also of previous events. • That’s what a linear filter does. No more, no less. We are sticking to linear filters today, thank you! • There are many ways to build filters.

9. The Time and Frequency Response of the Analog Filter Time (impulse) Response Frequency Response

10. How does the analog filter exhibit memory? • In the case shown, the capacitor is the memory element. • It retains previous history. • It does so by summing the history into one value, the voltage across the capacitor. • In such a way, a single component can have a long memory.

11. Ok, what’s the “impulse response” Impulse response is the response of the circuit to a (mathematical) signal of infinite height (and power), and infinitely short duration. This “unit impulse” has very special characteristics: • It contains all frequencies • It has energy of “1” at all frequencies. • It describes the behavior of the filter at all frequencies. Completely. POINT 3 is what you need to remember. The impulse response of a filter is a complete description of what it does.

12. Another way to look at the impulse response. • The impulse response of a system shows how a filter captures the HISTORY of the signal. In other words: • The value of the impulse response at a time ‘t’ demonstrates how much of the HISTORY of the signal is added to the output at time ‘t’ later.

13. Remember last month? • Multiple speakers created a “comb filter”? • Yep, that’s a filter. Different distance means “different times in the history of the signal. • You can plot a time response for such a filter just like you can plot the one for the simple RC filter above. • We saw some of that last month.

14. Getting back to the analog filter: • The analog filter’s output can not change instantly as that would require infinite current in the resistor, that means that the output depends on the HISTORY of the signal. • In fact, if you have an input signal, and you integrate the product of the time-reversed impulse response times the signal, you get the filter’s output.

15. DID YOU SAY INTEGRATE? • Well, it can be a sum, rather than an integral, and in fact in the digital domain, it is a sum, rather than an integral. • In the digital domain, rather than having all frequencies, you have all frequencies inside the digital passband, which will be ½ the sample rate wide.

16. I’ll say more about that later. • For now, let’s get back to that analog filter again for a minute.

17. So, what did that filter actually do? • It stored part of the history of the signal. As the time went on, it “forgot”, exponentially, the contributions to its output from previous history. • Not all impulse responses are so simple. • An impulse response may be positive or negative • An impulse response may “ring” or not.

18. How about a “Digital” filter? Output Input + (1-1/8) X Feedback One Sample Delay The digital filter uses history explicitly.

19. That digital delay • In digital terms, a delay of one sample is defined as a multiplication by ‘z’. • Some texts use z-1. • Either works. The rules are the same, except for how you interpret some things that I’ll leave out for now. • Remember a delay of ‘z’ is one sample. If I say I multiplied a signal by z3, it means I delayed it by 3 samples. Yes, it really is that simple.

20. What is it’s time and frequency response? The result is the same (below half the sampling rate)

21. What about not below half the sampling rate? • Inside the ‘z’ domain, there is no such thing. • The ‘z’ domain is also called the “digital domain” by many people. This does involve a potential confusion that I will ignore for the time being. • Remember when we talked about the sampling theorem a long time ago? We used an anti-aliasing filter. • All other “frequencies” are represented by aliases below fs/2, and you don’t let them in in the first place! • All information in the sampled domain is contained inside that bandwidth. All aliases/images contain exactly the SAME information as the baseband spectrum. No more, no less.

22. Why is the response the same? • This filter also stores the history of the signal in an exponentially decaying fashion. In the digital filter, you can see the storage more directly, as the ‘z’ element. In the analog filter, reactive components provide the history in a continuous fashion. • I picked the parameters to provide the same visible response as long as we stay well below half the sampling rate. • Recursive (IIR) filter outputs depend on both previous outputs and input(s). • Analog filters are mostly (but not completely) filters that depend on both output and input.

23. Back to Impulse Response • The response of either filter to an impulse is called its “impulse response”. A digital impulse is simpler (as the bandwidth is finite), and consists of one ‘1’ sample but both have the same use. • This is the “time response” plotted in the proceeding diagrams. • The impulse response of a filter defines its memory (history) of a signal. • Remember. The impulse response of the filter contains exactly all the information about a filter. • This is, you will find out, very handy.

24. Ok, what’s the big deal, JJ? • Since the impulse response of the filter defines its interaction with the signal, this means that we can either use the recursive form shown before to implement the filter, or we can simply multiply the time reversed impulse response by the signal and sum (integrate) the result, to get the filter output. • The two operations are exactly the same.

25. Let us use a 9th order Elliptical filter as an example: Impulse response Noise Input (the two exactly overlap) Red: By time reverse, multiply, and sum Green: By direct filtering

26. Convolution • The process of multiplying the time-reversed signal by the impulse response, and summing (integrating) is called “convolution”. Think of it as a way to expressly include the history of the signal in the filter output.

27. I thought we multiplied transfer functions, jj???? The usual way we see transfer functions expressed: (notice this is in the ‘’, or frequency, domain) S() Y() = S()*H() H() What’s actually happening in the time domain: (Note:  is used here to denote convolution. There are other notations.) s(t)h(t) s(t) y(t) These are two ways of saying the exact same thing!

28. Multiplication in the time domain is the same as convolution in the frequency domain. Multiplication in the FREQUENCY domain is the same as convolution in the TIME domain. It works either way. If you read a DSP text, you will see the word “duality”. This is duality in action.

29. For typical filters, convolution is what happens in the time domain. • Convolution is merely another way of expressing what happens when you filter a signal. • It’s the same as multiplying the signal by the transfer function. This relationship holds for the ‘s’ domain, the ‘z’ domain, and quite some other domains as well.

30. Convolution is important because: • Convolving in the time domain (like we just saw here) is the same as multiplying the Fourier Transform of the signal by the Fourier Transform of the Impulse Response and then taking the Inverse Fourier Transform • This works the other way around, too, but isn’t usually as interesting to discuss in most filtering applications except perhaps as window functions. (There are exceptions, for instance, “TNS”.)

31. Multiplication of Transforms of Signal and Filter Signal Signal Spectrum Filter Filter Spectrum Product of Spectra Inverse transform Of Product

32. An example of Convolution Time Domain Frequency Domain h(t) |H(w)| s(t) |S(w)| y=s(t) h(t) |Y(w)=H(w)*S(w)| IFFT(Y) |FFT(y)| In this plot, it’s easier to see the linear superposition in the time domain because the two parts of the signal s(t) do not overlap.

33. What’s my point? • Filtering is Convolution. Convolution is filtering. They are the same thing expressed in different domains. There are several ways to do filtering: • IIR (Infinite Impulse Response) filters, like the two shown much earlier. They are called IIR because the filter’s impulse response continues to infinity (yes, at infinitely small value for a stable filter). These filters, effectively, use a topology that implements the history inside a few (very important, sensitive) state variables. • FIR (Finite Impulse Response) filters, in other words, just do the convolution using a ( potentially arbitrary) impulse response.

34. So, they are the same? • Well, no. In fact, FIR filters have zeros, and IIR filters have poles. (In reality, nearly all IIR filters have both poles and zeros, which is to say that they have both an FIR and an IIR part. • FIR and IIR filters can have quite different properties, and usually do, they are two different means to an end. Neither one nor the other is always better.

35. More about IIR filters • IIR filters must be implemented using feedback to implement the poles, in order to be truly IIR. • IIR filters are “longer” (in terms of impulse response) compared to the memory they directly use. (i.e. a 2nd order filter can have 1000’s of samples of significant energy in its impulse response.) • The impulse response length is what can determine the sharpness of the filter’s frequency and phase response. • This extension places substantial requirements on the implementation in terms of accuracy, both of coefficients (analog or digital), and of related processes (digital storage, multiplication, addition). • The data is stored in a few variables, so the accuracy required for those variables rises accordingly.

36. FIR Filters • FIR filters are not generally as sensitive to coefficient roundoff • FIR filters often require more computation, because you must do a multiply-add for each term in the impulse response • FIR filters can be constant delay, IIR filters can not. Sometimes this matters.

37. What are the meaningful properties of a filter? • The amplitude response (plotted in terms of amplitude vs. frequency) • The phase response (plotted in terms of phase vs. frequency) • What does phase response mean? • Linear phase (i.e. constant time delay) • Minimum phase • Non-minimum-phase Linear phase is an important subset of this class that has all zeros. Attention: We are talking about single filters here, not filter banks. That is another subject, and one that places more constraints on individual filters!

38. A bit more on phase response • “Linear Phase” (constant delay) • If a filter has a constant delay, the phase shift of the filter will be t*w, where t is the time delay, and w the natural frequency (2 pi f). • This means that a delay can exhibit enormous phase shift. • This phase shift, however, is ONLY delay. • Non-linear delay • This is the part of the phase shift (in and around the filter’s passband) that is not modeled by a straight line) • The part that does not correspond to a straight line constitutes non-constant-time phase shift. • Phase shift of “1 million degrees” in and of itself tells you nothing!

39. Some example plots:IIR FIR 13th order elliptical, (poles and zeros) 512 point Symmetric FIR Impulse Response Notice similar length Magnitude Response Notice similar frequency response Phase Response DIFFERENT PHASERESPONSE Note phase Nonlinearity in passband “Linear” phase

40. Properties of Impulse Responses • Symmetry • Antisymmetry • Asymmetry • DC Gain • Fs/2 Gain • Frequency response • Phase Response

41. DC gain • The DC gain of an impulse response is exactly the sum of all of its non-zero coefficients. • For many applications, one wishes to set this to one. • This is easy. Divide the entire impulse response by the sum of all values of the impulse response.

42. Gain at FS/2 • This is also easy. • Sum all of the EVEN taps • Sum all of the ODD taps • The difference of the two is the gain of the filter at FS/2

43. Some useful things to know(I won’t prove them here) • A symmetric impulse response implies: • The passband phase response (one or multiple passbands) will look like a pure delay (linear phase) • “Linear phase”  phi = w*t, where omega is the natural frequency and ‘t’ is the time delay • An antisymmetric impulse response has some interesting (and special) properties. They are beyond this introductory tutorial, but are worth looking into for some applications. Such filters will have “linear phase” in the passband, but the intercept of such a filter at DC must be at +_ 90 degrees, and the filter must have a zero at DC. • An asymmetric impulse response implies: • The passband phase response is not a pure delay. • Practically speaking this means that the response is the sum of a symmetric and an antisymmetric response.

44. Implications of the previous page No IIR filter can be linear phase. If it were, it would have to extend to infinity on both sides, and have infinite delay. Some IIR filters can “come close” under some circumstances. In such cases, they have substantial “pre-ringing” (as they must). A true IIR filter with linear phase must be “non-causal”, i.e. it must be able to “look ahead” in time 9th order Butterworth Impulse response With 108 gain

45. FIR filters are usually designed as “type 1 linear phase” meaning that they are symmetric, with even filters having two identical center taps, and odd filters symmetric about a single center tap. Symmetric filters with an even number of taps must have a zero at pi and can not be highpass. Odd Even

46. There are other kinds of FIR filters, in particular antisymmetric even tap filters, which have linear phase in the passband, but do not have zero phase shift at DC. Rather, they have + or – 90 degree phase shift at DC, and must have a zero at DC. These “type 4 filters” can not be used as lowpass filters.

47. A completely asymmetric FIR filter is a valid filter, and in some cases (phase compensation, etc) may be used. Such filters are usually for special-purpose applications, however.

48. A comparison of 3 FIR filters

49. Even vs. odd lengthEven Odd Zero at pi Nonzero at pi Delay of 15.5 samples. Delay of 16 samples (we will compare 32 tap vs. 33 tap Lowpass FIR’s with identical parameters except for length

50. So? • If you need an integer delay in the filter, use an odd-length filter. (N.B. In many applications, where even filters are applied twice, you can use two even filters.) • If you need a zero at pi, use an even-length filter. • If you don’t want a zero at pi, you can’t use a symmetric even-length filter. You can use an antisymmetric even length filter if you want a highpass filter, but then you’ll have a zero at DC. • This means that symmetric high pass filters are of odd length.