Fundamentals of New Media MUMT 202

1. Fundamentals of New MediaMUMT 202 Cory McKay Jason Hockman

2. course info Fundamentals of New Media (MUMT 202) • lectures: monday 5:35 PM - 8:25 PM in E-106 • labs: thursday 5:35 PM - 7:35 PM in E-106 • additional lab TBA if required • course materials: on blackboard system • office: 550 Sherbrook St. West, suite 500 (dial x0300 on hallway phone) • office hours by appointment

3. course info • course will be in 2 sections • section 1: • instructor: Jason Hockman (jason.hockman@mail.mcgill.ca) • TA: Jung-Suk Lee (jungsuk.lee@mail.mcgill.ca) • section 2: • instructor: Cory McKay (cory.mckay@mail.mcgill.ca) • TA: Bertrand Scherrer (bertrand.scherrer@gmail.ca) • TA: John Ashley Burgoyne (ashley@music.mcgill.ca)

4. main goals to provide: • an overview of new technologies related to music and music production • an understanding of the basics of digital audio and digital signal processing • an exposure to the present state of artistic work in music technology and new media

5. lecture schedule Part 1 • Introduction to New Media • Basic Concepts of Acoustics and Psychoacoustics • Digital Signal Processing Introduction • Music and the Internet • MIDI, OSC and Symbolic Music • Apple Logic

6. lecture schedule Part 2 • Editing and Mixing Audio • Audio Effects • Score Editing • Sound Recording • Artistic Examples • Overview of Music Technology Research Areas (if time permits) • Optional Topics (if time permits)

7. lecture schedule part 1 part 2 • 9/14: class 1 • 9/21: class 2 • 9/28: class 3 • 10/5: class 4 • 10/12: no class - Thanksgiving • 10/19: class 5 • 10/22: class 6 • special Thursday class • 10/26: class 7 • test 1 • 11/2: no class • 11/9: class 8 • 11/16: class 9 • 11/23: class 10 • 11/30: class 11 • 12/3: class 12 • special Thursday class • test 2 • final project presentations

8. deadlines and attendance • attendance to all courses is required • collaboration and mutual aid is encouraged • cheating and plagiarism will not be tolerated • students are expected to be familiar with McGill’s policy on academic integrity • part 1 will have 4 assignments, and one test • part 2 will have 2 or 3 assignments and a final project • all assignments are due at the beginning of class on the due date • late assignment policy (20% off total grade each day late)

9. deadlines and attendance • Part 1: • Assignment 1 (5%) September 21 • Assignment 2 (5%) September 28 • Assignment 3 (5%) October 5 • Assignment 4 (5%) October 19 • Test 1 (30%) October 26

10. deadlines and attendance • Part 2: • Assignment 5/6 (10%) November 23 (Editing and Mixing Audio) • Assignment 7 (5%) November 30 (Score Editing or Sound Recording) • Test 2 (10%) December 3 • Final Project (25%) December 3

11. MTCL entry • registered students should already have access to the lab • any problems with entry: please see Building Director (room E234) • network accounts: max 5 GB, 100 page (per 120 days) printing limit • new user accounts email Darryl: • login will be: lastnamefirstinitial (e.g., hockmanj) • password: login name + 1234 (e.g., hockmanj1234) • network account issues: Darryl Cameron (darryl.cameron@mcgill.ca)

12. introductions . . .

13. what is new media? • media: tools to store and deliver information • advertising • communications technology • print media • can also be related to materials and techniques used by an artist • integration of media into our lives expands with technology • new media: tools developed relatively recently that relate to computerized communication and information distribution • audio, video, internet and computer technologies

14. what is new media? • new media development has led to: • globalization • increased communal knowledge across nation boundaries • e.g., online higher education, wikipedia • social and political change • response to this year’s Iran election • influence in determining recent US election

15. what is new media? Audio • production and performance technologies: • sequencers: Apple Logic, Steinberg Cubase, Abelton Live • synthesis: Access Virus Snow • auto-effects: ARTMA, BBCUT • control and performance: Lemur, Reactable, Tenori-On, T-Stick, Serato Scratch, Mixxx

16. what is new media? Video • video and visual technologies: • video editing software: Apple Final Cut

17. what is new media? Video • video and visual technologies: • video editing software: Apple Final Cut • image editing software: Photoshop, GIMP

18. what is new media? Computational technologies • internet • engines: Google, Yahoo • social networks: Facebook, Myspace • streaming audio: lastFM, Pandora, Grooveshark • streaming video: Youtube, Vimeo • media stores: itunes store, Beatport, Digital-tunes, Bleep, Boomkat • torrent: pirate bay, mininova

19. outline of today’s material • Basic Concepts of Acoustics and Psychoacoustics • source-medium-receptor chain • acoustic and musical parameters • simple harmonic motion and sinusoids • complex tones • superposition of waves / Fourier analysis

20. break

21. Psychoacoustics • definition: the study of subjective human perception of sounds • the study of psychoacoustics seeks to explain how we experience the phenomena of sound through an understanding of the intricate relationships between sound waves and our organs for perception • why its important: • we need to know how sound propagates and how we perceive it • basis for space-saving compression techniques • used for distribution of sound within space - a.k.a. spatialization

22. Psychoacoustics • area includes a broad range of topics: the ear, brain, and vocal organs, as wel;l as a neurological and psychological approach to concepts such as pitch, loudness, timbre, and spatialization • also includes memory (important for timing measurements)

23. Psychoacoustics • what is sound? • traveling wave(s) that are oscillating through a medium with parameters that are in accordance with the range of the heard frequencies (20 Hz - 20 kHz) • involves 3 systems: source-medium-receptor model • performer singing on a stage - source • air in the concert venue - medium • audience members - receptors • source creates the energy, while the medium transmits the sound, and the receptor receives the signal via senses, and then perceives the sound through higher level cognition.

24. Psychoacoustics • source is comprised of 3 components: • primary excitation (e.g., blowing, bowing, striking) • energy source for system • vibrating element (e.g., string on a guitar, head on a drum) • excitation elicits oscillation modes that generate musical pitch • upper harmonics influence timbre • resonator (e.g., body on guitar or violin) • to transfer the oscillation from the vibrating element to the external environment with greater amplification (and richer timbre)

25. Psychoacoustics • medium (e.g., air, water) • when the traveling oscillations reach boundaries such as walls, ceilings, floors, etc., the are either reflected or absorbed • various materials have different absorption characteristics • this determines the quality of room acoustics and reverberation

26. Psychoacoustics • receptor • process of audition begins with our ears • sound first hits our pinna (collects sound and filters it). pinna is also used for propagation analysis (2 ears) • sound then into auditory canal • eardrum at end of canal - sound waves cause vibration in eardrum, causing 3 bones on opposite side to vibrate (malleus, incus, stapes) • nerve impulses are then created in the cochlea in a region called the basilar membrane (lined with ~3500 hairs that sense vibration) • nerve impulses are then interpreted within frequency groupings, that ranging from 20 Hz - 20 kHz

27. Psychoacoustics • Cochlea

28. Psychoacoustics • Cochlea lower frequencies video higher frequencies

29. Psychoacoustics • Pitch, Loudness and Timbre • sound stimuli broken down into these 3 subjective measures • pitch represents the fundamental frequency of a sound • sine tones contain a single frequency • complex tones have several frequencies (more on this later) • loudness is a sensation of the intensity of a sound • timbre is everything that is not pitch and loudness that can be used to differentiate between two sounds • tone burst must be tens of periods before it is recognized as pitch • noise is without pitch, but contains both loudness and a timbre • e.g., waterfall, fricatives

30. Psychoacoustics • Pitch, Loudness and Timbre • as mentioned these are subjective, and measuring these is difficult • there are 3 objective measures sound identification • fundamental frequency (oscillation rate of lowest freq. in harmonic series) • amplitude (amplitude of the oscillation - equivalent to pressure on eardrum) • spectrum (arrangement of partials)

31. Auditory Perception • Fletcher: pitch perception varies with frequency • high frequency perception is result of cochlea excitation • low frequency perception involves a “time mechanism” • Fletcher-Munson (1933): sound intensity level varies with frequency • i.e., given equally loud sinusoids of different frequencies, we hear them as being different levels. • most audible = 2000 Hz - 3000 Hz • least = below 100 Hz and above 10000 Hz • phon represents the perceived loudness of a sound wiht a given frequency

32. Equal-Loudness Contours • Fletcher-Munson curves (1933): created by asking large number of subjects to rate loudness of different frequency sinusoids

33. Periodic Motion • motion that is repeated in equal intervals of time • e.g., rocking chair, tuning fork, orbit around sun • period: time it takes for one cycle • frequency: inverse of period • e.g., tuning fork may have a frequency of 1000 Hz (repeats motion 1000 times per second, but its period is then 1/1000 or 1 millisecond) • simple harmonic motion • point comes back repeatedly to the same position at exactly equal intervals of time and exactly repeating the same type of motion in between • can be represented as the projection of uniform circular motion on the diameter of a circle • aka sinusoidal motion

34. Periodic Motion • think of the displacement of a spring over time

35. Periodic Motion

36. Periodic Motion • Sinusoid is then fully described by its • Frequency (f) • Amplitude (A) • Phase (φ) • measured in radians • 2π = 360 degrees = full cycle • s

37. Periodic Motion

38. Complex Tones • what happens when 2 or more sinusoidal waves with same frequency are heard simultaneously? • constructive interference: if the 2 waves share a common frequency and phase, the result will be a sum of the amplitudes of each of the original sinusoids • destructive interference: if the 2 waves share only a common frequency, and is 1/2 cycle out of phase (φ = π), then the amplitude will be the difference between the 2 amplitudes

39. Complex Tones • what happens when 2 waves with different frequencies are added? • resulting signal is no longer a sinusoid since it doesn't follow pattern of simple harmonic motion • resulting signal freq. = lower of the 2 original frequencies • amplitude of the resulting signal = sum of the originals • if two sinusoids are `in phase' then peaks and troughs coincide • If sinusoids are out of phase then peaks and troughs oppose each other and they will cancel each other out

40. Complex Tones tone A tone B combined

41. Complex Tones • this leads us to Fourier Theory, which implies that any complex periodic waveform can be decomposed into a set of sinusoids with different amplitudes, frequencies and phases • following this model then, all sounds that we hear may be reduced into the linear combination of simple sinusoids