Effects in frequency domain

1. Effects in frequency domain Stefania Serafin Music Informatics Fall 2004

2. Overview • Time versus frequency domain effects • The phase vocoder • The STFT • Time stretching • Pitch shifting • Cross synthesis

3. Overview • Until now we examined effects based on delays and working in time domain. • What happens if we want to work in the frequency domain? • In this case we talk about SPECTRAL MODELS and SPECTRAL TRANSFORMATIONS

4. The phase vocoder • A phase vocoder represents an ensemble of techniques which take a sound in the time domain,calculate the Fourier transform, perform some manipulations in frequency domain and then reconstruct the sound in time domain.

5. Analysis/synthesis steps INPUT ANALYSIS TRANSFORMATIONS SYNTHESIS OUTPUT

6. Analysis synthesis steps

7. Analysis: STFT

8. STFT • Time-domain audio signal is separated into successive short frames. • Each frame is multiplied by a window function to smooth transitions. • Resulting windows are passed on to FFT module. The FFT process then extracts spectral information on the signal.

9. Spectral transformations • Pitch shifting • Time stretching • Morphing (Cross-synthesis)

10. Pitch shifting • How would you perform a pitch shift in time domain? • What are the drawbacks of doing it?

11. Solution: move to frequency domain • In frequency domain • 1) Calculate the spectrum • 2) Shift it upwards or downwards • 3) Go back to time domain

12. Time stretching • Time-scaling is an application that allows signal length to be stretched or shortened without affecting the frequencies of its components. • 1) Calculate spectrum • 2)Reduce/increase time information • 3) Go back to time domain

13. Morphing • It is the combination of two signals • In frequency domain: • 1) Calculate spectrum of signal A • 2) Calculate spectrum of signal B • 3) C = A B

14. STFT resynthesis • STFT resynthesis is the final stage of the phase vocoder process. It is based on recreating a signal in time domain from the modified spectral components of the input signal.

15. The third stage implements an exact inverse of the process that was used in STFT analysis of the signal. Modified spectral components of the input signal are passed through the Inverse Fourier Transform to recreate a set of time-domain frames.

16. Application • Farinelli

17. Application (ctn.) • Karaoke

18. In Max/MSP • Check the examples Folder. • Convolution and Phase vocoder example.