Digital Sound Representation & Introduction to Software Synthesis

1. Digital Sound Representation & Introduction to Software Synthesis Electronic Music Studio TU Berlin Institute of Communications Research http://www.kgw.tu-berlin.de/ Prof Eduardo R Miranda Varèse-Gastprofessor eduardo.miranda@btinternet.com

2. Topics: • Digital representation of sound • Software sound synthesis fundamentals • Sound synthesis programming with Csound: Introduction to Csound

3. Digital Representation of Sound What is a sound? • From a physical point of view, it is the result of a mechanical disturbance of some object in a physical medium, such as air. • This mechanical disturbance generates vibrations that can be represented as electrical signals by means of a device (for example, a microphone), that converts these vibrations into a time-varying voltage.

4. Analog vs. Digital • An analog sound signal is the result of measuring the voltage that represents the sound. • These kinds of signals are continuous in the sense that they consist of a continuum of constantly changing values. • A digital sound is the result of counting all these values many times per second for a certain defined length/time. • Each measurement value is called a sample, and this kind of process is called sampling. • In order to process sounds on the computer, the analog sound must be converted into a digital format understandable by computer; that is, binary numbers.

5. Analog sound into a digital sound • The process of converting a sound (e.g., coming from an instrument such as • the violin) into a digital representation: • a) a microphone or a line-level source converts sound into voltage • b) an analog-to-digital converter then converts voltage into binary • numbers • c) a computer stores the resulting sound

6. Analog-to-digital converter (ADC) are devices used to convert the waveform into a succession of binary numbers, each representing the voltage level at a given instant. • Digital-to-analog converter (DAC), converts binary numbers (i.e., digital sound representation) into analog signals in order to play the sound. • 16-bit conversion suffices for home-quality recording • Professional quality recording often uses 32-bit and 64-bit words

7. Computers are normally configured to function based upon strings of bits (binary numbers) of fixed size, called words. For example, a computer configured for processing 4-bit words would represent the decimal numbers 0,1,2 and 3 as the binary numbers 0000 = 0 0001 = 1 0010 = 2 0011 = 3 0100 = 5 0101 = 6 etc.

8. Sampling theorem • The analog-to-digital conversion process is called sampling. • The frequency of a sound is equal to the number of cycles which occur every second ("cycles per second", abbreviated "cps" or "Hz"). • In order to convert an analog sound signal into digital representation one needs to sample the signal many times per second. • The frequency of this sampling process is called sampling frequency or sampling rate, and it is measured in Hertz (Hz).

9. The sampling theorem states that in order to accurately represent a sound digitally, the sampling rate must be higher than at least twice the value of the highest frequency contained in the signal. • The average upper limit of human hearing is approximately 18 kHz (18000 Hz), which implies a minimum sampling rate of 36 kHz (36000 Hz). • The sampling rate frequently used in computer sound design systems is 44.1 kHz (44100 Hz).

10. Aliaising distortion • Nyquist frequency is the name of the highest frequency that can theoretically be represented in a digital audio system. • It is calculated as half of the value of the sampling rate. Passing this level causes aliasing distortion, or foldover.

11. Sampling quality • The level of resolution, or quantization of a sampler: depends upon the size of the word used to represent the amplitude of a sampling sound and is determined by the resolution of the ADC and DAC. • A word, for instance, could be 4-bits long or 16-bits long etc. Unsatisfactory lower resolutions are prone to cause a damaging loss of sound quality, referred to as quantization noise.

13. Sound examples • The following four sound examples were sampled from a mono sound, 4027 milliseconds long, sampled at 44100 Hz, 11025 Hz, 2756 Hz and 2000 Hz, respectively. • The last file uses only 17 kB to store the sound, but it is considerably distorted when compared to the first example, which uses 367 kB. a) [Example 1] 44100 samples per second results in a sound file of 367 kB b) [Example 2] 11025 samples per second results is a sound file of 93 kB c) [Example 3] 2756 samples per second results in a sound file of 23 kB d) [Example 4] 2000 samples per second results in a sound file of 17 kB

14. Comments • The ability to adjust the sample rate affects the length of a sample for a given number of samples. • This will affect the size of the digital sound file in a storage system. • There is a penalty to be paid for the lower rate: a narrower band of frequencies will be recorded. • The problem is that acoustic sounds are in fact composed of many sound components called partials. I • In the majority of cases, most of these partials are of very high frequencies. If the sampling frequency is too low, the computer will not capture these high frequency components and this causes distortion.

15. Software Sound Synthesis

16. Unit generators • Synthesis programming languages provide a number of synthesis subroutines, usually referred to as unit generators. (Inspired by the modules of an analog synthesiser). • Synthesis instruments are designed and programmed by connecting a number of unit generators. • Most basic unit generator of a synthesis programming language or systems is the oscillator.

17. Digital oscillator

18. The speed at which the lookup table is scanned defines the frequency of the sound. • The lookup table contains a list of samples for one cycle of the waveform, which does nor necessarily need to be a sinusoid. • The specification of an oscillator on a computer typically involves: • Frequency • Amplitude • Waveform

19. Other synthesis units • Function generators: create lookup tables for the oscillators and for sound transformation and control (“GEN routines” in Csound). • GEN routines fill data tables with values produced with specified procedures or mathematical formulae such as trigonometric functions and polynomials (the length is usually specified as a power of two (29 = 512, 210 = 1024, etc.) • Signal modifiers: filters, envelopes, etc. Envelops add time-varying control to synthesis instruments.

20. Introduction to Csound • Has its origins in the early 1960’s at Bell Telephone Research Lab (New Jersey, USA) by Max Matthews – first computer music language called Music 3. • Early 1970’s Barry Vercoe created a version called Music 11 (for PDP 11 computer) at MIT. • Later Berry Vercoe re-made Music 11 in the C language, thereby creating Csound.

21. Main features • Separate control rate and audio rate processing. • Audio rate is the same as the sampling rate. • Control rate is a slower rate used to shape certain features of the sound. • It is widely portable under C and UNIX. • It is structured in a way that allows for further developments; e.g., in 1992 real-time control under MIDI was introduced. • User can add their own functions.

22. The structure of the language • The composer needs to specify two types of files: orchestra and score files • The orchestra file has 2 sections: headers and instrument definition • Header: specifies global issues of the synthesis algorithm: sampling rate, number of channels, etc. (Note: anything after “;” is regarded as a comment and is not considered when the program runs). • sr = 22050 ; the sampling rate is set to 22050 samples per second • kr = 1470 ; there will be 1470 control point per second • ksmps = 15 ; there will be a control point every 15 sound samples sr/kr • nchnls = 1 ; mono sound eg1

23. Instrument definition: defines the synthesis algorithm. • The following example shows a simple instrument composed of a single sinewave oscillator • instr 1 ; start the definition of instrument 1 • a1 oscil 10000, 440, 1 ; make an audio rate oscillator with an • ; amplitude of 10000, at a frequency of 440 Hz, ; using wavetable number 1 • out a1 ; output audio signal • endin ; end of instrument eg1

24. The complete orchestra file looks like this: • ;----------------------------- • sr = 22050 • kr = 1470 • ksmps = 15 • nchnls = 1 • ;----------------------------- • instr 1 • a1 oscil 10000, 440, 1 • out a1 • endin • ;----------------------------- 440 Hz 10000 GEN 1 eg1

25. Score: defines what the instrument is going to do. The score comprises: • Definition of the wavetable(s) used in the instrument • Timed event triggers; e.g., note sequences • Wavetable definition: function table (hence the use of an “f”) at the beginning of the line. • Example: wavetable 1, comes to take effect from time 0, with 4096 elements, using GEN no. 10 (GEN = generator function), with only one harmonic (which in this case defines a sinewave) • ;name start size GEN relative strength • f1 0 4096 10 1

26. The second part of the score defines a note event for the instrument, starting at time 0 and lasting for 4 secs • ;instr start duration • i1 0 4 • e • In this case, frequency (pitch) and amplitude are state in the orc file

27. The complete orchestra and score files look like this: • ;----------------------------- • sr = 22050 • kr = 1470 • ksamps = 15 • nchanls = 1 • ;----------------------------- • instr 1 • a1 oscil 10000, 440, 1 • out a1 • endin • ;----------------------------- • ;----------------------------- • f1 0 4096 10 1 • ;----------------------------- • i1 0 4 • e 440 Hz 10000 GEN 1 eg1

28. pfields (p = score parameters): • ;----------------------------- • sr = 22050 • kr = 1470 • ksmps = 15 • nchnls = 1 • ;----------------------------- • instr 1 • a1 oscil p4, p5, 1 • out a1 • endin • ;----------------------------- • ;----------------------------- • f1 0 4096 10 1 • ;----------------------------- • ;p1 p2 p3 p4 p5 • ;----------------------------- • i1 0 4 10000 440 • e p5 p4 GEN 1 • p1 = instrument • p2 = start time • p3 = duration • p4 = amplitude • p5 = frequency eg2

29. Examples of scores for the simple oscil instrument: • (by Andy Hunt, Uni of York, from CDP tutorial) • ;-------------------------------------------------- • ; 3 note melody • ;-------------------------------------------------- • f1 0 4096 10 1 • ;-------------------------------------------------- • ;p1 p2 p3 p4 p5 • ;-------------------------------------------------- • i1 0 4 10000 440 • i1 4 2 10000 493 • i1 6 4 10000 440 • e ;----------------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;----------------------------- instr 1 a1 oscil p4, p5, 1 out a1 endin ;----------------------------- eg3

30. Examples of scores for the simple oscil instrument: • (by Andy Hunt, Uni of York, from CDP tutorial) • ;-------------------------------------------------- • ; Overlapping notes • ;-------------------------------------------------- • f1 0 4096 10 1 • ;-------------------------------------------------- • ;p1 p2 p3 p4 p5 • ;-------------------------------------------------- • i1 0 12 3000 220 • i1 2 12 3000 440 • i1 4 12 3000 660 • i1 6 12 3000 880 • i1 8 12 3000 1100 • i1 10 12 3000 1320 • e ;----------------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;----------------------------- instr 1 a1 oscil p4, p5, 1 out a1 endin ;----------------------------- eg4

31. Examples of scores for the simple oscil instrument: • (by Andy Hunt, Uni of York, from CDP tutorial) • ;-------------------------------------------------- • ; Overlapping with different amplitudes • ;-------------------------------------------------- • f1 0 4096 10 1 • ;-------------------------------------------------- • ;p1 p2 p3 p4 p5 • ;-------------------------------------------------- • i1 0 12 3000 220 • i1 2 12 1500 440 • i1 4 12 700 660 • i1 6 12 350 880 • i1 8 12 175 1100 • i1 10 12 85 1320 • e ;----------------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;----------------------------- instr 1 a1 oscil p4, p5, 1 out a1 endin ;----------------------------- eg5

32. Examples of scores for the simple oscil instrument: • (by Andy Hunt, Uni of York, from CDP tutorial) • ;-------------------------------------------------- • ; Simultaneous starts • ;-------------------------------------------------- • f1 0 4096 10 1 • ;-------------------------------------------------- • ;p1 p2 p3 p4 p5 • ;-------------------------------------------------- • i1 0 12 3000 220 • i1 0 12 1500 440 • i1 0 12 700 660 • i1 0 12 350 880 • i1 0 12 175 1100 • i1 0 12 85 1320 • e ;----------------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;----------------------------- instr 1 a1 oscil p4, p5, 1 out a1 endin ;----------------------------- eg6

33. Amplitude envelopes (or envelops) • May synthesisers feature enveloping functions called ADSR: • Attack = time from the start to the peak of the sound • Decay = loss of volume from the initial peak • Sustain = the continuation level of the sound until release • Release = speed at which the sound dies away

34. linen • A Csound unit commonly used for making an envelope. • Specified with the following parameters: • Amplitude = peak amplitude of the sound • Rise = attack time • Duration = total duration of the sound • Decay = period at which the sound dies away

35. Example ;ORC ---------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;----------------------------- instr 1 k1 linen p4, p6, p3, p7 a1 oscil k1, p5, 1 out a1 endin ;----------------------------- ;OSC --------------------------------------- f1 0 4096 10 1 ;---------------------------------------------- i1 0 1 10000 440 0.05 0.7 i1 1.5 1 10000 440 0.9 0.1 i1 3 1 5000 880 0.02 0.99 i1 4.5 1 5000 880 0.7 0.01 i1 6 1 20000 220 0.5 0.5 e • p1 = instrument • p2 = start time • p3 = duration • p4 = amplitude • p5 = frequency • p6 = rise • p7 = decay eg7

36. Perceptual role of attack • As the attack gets shorter and shorter the whole character of the sound alters • ;OSC ---------------------------------------------------------- • f1 0 4096 10 1 • ;----------------------------------------------------------------- • i1 0 1.5 10000 440 0.9 0.5 • i1 2 1.5 10000 440 0.7 0.5 • i1 4 1.5 10000 440 0.5 0.5 • i1 6 1.5 10000 440 0.3 0.5 • i1 8 1.5 10000 440 0.15 0.5 • i1 10 1.5 10000 440 0.07 0.5 • i1 12 1.5 10000 440 0.04 0.5 • i1 14 1.5 10000 440 0.02 0.5 • i1 16 1.5 10000 440 0.015 0.5 • i1 18 1.5 10000 440 0.01 0.5 • i1 20 1.5 10000 440 0.005 0.5 • i1 22 1.5 10000 440 0.004 0.5 • i1 24 1.5 10000 440 0.003 0.5 • i1 26 1.5 10000 440 0.002 0.5 • i1 28 1.5 10000 440 0.001 0.5 • i1 30 1.5 10000 440 0.0005 0.5 • e eg8

37. Pitch notation • N.00, ‘N’ = the octave and ’00’ the pitch • middle C is octave N=8 • .00 = varies from 00 = C to 11 = B • Examples • Csound Note MIDI Frequency • 8.00 C4 60 261.60 Hz • 8.01 C4# 61 277.18 Hz • 8.02 D4 62 293.67 Hz • 8.03 D4# 63 311.13 Hz • 8.04 E4 64 329.63 Hz • 8.05 F4 65 349.23 Hz • 8.06 F4# 66 369.99 Hz • 8.07 G4 67 392.00 Hz • 8.08 G4# 68 415.30 Hz • 8.09 A4 69 440.00 Hz • 8.10 A4# 70 466.16 Hz • 8.11 B4 71 493.88 Hz

38. Example ;ORC ---------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;----------------------------- instr 1 inote=cpspch(p5) k1 linen p4, p6, p3, p7 a1 oscil k1, inote, 1 out a1 endin ;----------------------------- ;OSC --------------------------------------- f1 0 4096 10 1 ;---------------------------------------------- i1 0 1 10000 8.09 0.05 0.7 i1 1.5 1 10000 8.09 0.9 0.1 i1 3 1 5000 9.09 0.02 0.99 i1 4.5 1 5000 9.09 0.7 0.01 i1 6 1 20000 7.09 0.5 0.5 e • p1 = instrument • p2 = start time • p3 = duration • p4 = amplitude • p5 = octave.pitch notation • p6 = rise • p7 = decay eg9

39. Example ;ORC ---------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;----------------------------- instr 1 inote=cpspch(p5) iamp=ampdb(p4) k1 linen iamp, p6, p3, p7 a1 oscil k1, inote, 1 out a1 endin ;----------------------------- ;OSC --------------------------------------- f1 0 4096 10 1 ;---------------------------------------------- i1 0 1 80 8.09 0.05 0.7 i1 1.5 1 80 8.09 0.9 0.1 e • p1 = instrument • p2 = start time • p3 = duration • p4 = amplitude in dB • p5 = octave.pitch notation • p6 = rise • p7 = decay eg10

40. Chorusing effect • Technique to enrich or make the sound fuller. • Done by creating beat frequencies; that is by playing copies of the note at slightly different frequencies.

41. ;ORC CHORUS ---------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;------------------------------------------- instr 1 inote=cpspch(p5) iamp=ampdb(p4) k1 linen iamp, p6, p3, p7 a1 oscil k1, inote, 1 a2 oscil k1, inote*0.99, 1 a3 oscil k1, inote*1.01,1 aout=(a1+a2+a3)/3 out aout endin ;---------------------------------------------- ;OSC CHORUS ------------------------- f1 0 4096 10 1 ;---------------------------------------------- i1 0 4 80 8.09 0.05 0.7 i1 4 4 80 8.09 0.9 0.1 e eg11

42. Timbre-migrating instrument ;ORC MIGRATING ----------------------- sr = 22050 kr = 1470 ksmps = 15 nchnls = 1 ;------------------------------------------------ instr 1 inote=cpspch(p5) ;amplitude envelope k1 linen p4, p6, p3, p7 ;transition control k2 linseg 1, p3*0.2, 1, p3*0.6, 0, p3*0.2, 0 a1 oscil k1, inote, 1 a2 oscil k1, inote, 2 out (a1*k2)+(a2*(1-k2)) endin ;SCO MIGRATING --------------------------------- f1 0 1024 10 1 0.5 0.3 f2 0 1024 10 81 0 27 0 9 0 3 0 1 0 0.3 0 0.1 ;---------------------------------------------------------- i1 0 6 15000 6.00 0.5 0.5 i1 0 1 12000 10.0 0.1 0.2 i1 1 1 12000 9.11 0.1 0.2 i1 2 1 12000 9.07 0.1 0.2 i1 3 1 12000 9.06 0.1 0.2 i1 4 1 12000 9.04 0.2 0.5 eg12

43. Defined pitch-space filled with glissando texture (from CDP Csound Tutorial Instrument Library) eg13

44. Defined pitch-space filled with glissando texture

45. Further reading: • Getting Started with Csound, by Andy Hunt (PDF file tutorial1.pdf) • CDP’s Csound Tutorial Instrument Library (PDF file tutorial2.pdf)