Human Auditory Cognition

Human Auditory Cognition Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, DimitraEmmanouilidou, DenizErdogmus, James OSullivan, Mehmet Ozdas, Lakshmi Krishnan, Michael Crosse, NimaMesgarani, Jose L Pepe Contreras-Vidal, Thusitha Chandrapala Organizers: Shihab Shamma (Univ. of Maryland), Malcolm Slaney (Microsoft),Barbara Shinn-Cunningham (Boston University), Edmund Lalor (Trinity College, Dublin)

We hear sounds Cocktail Party Effect We imagine and expect sounds Imagined Sounds

Others Muscles… Simple stimuli (beeps, etc) Simple responses (ERPs) Us Natural speech Not imagined movement Imagined speech!!! EEG Research

One basket, one big egg

Record EEG signals Three Approaches: Mick/Nima Covariance of channels/SVM Malcolm DCT of EEG/Nearest Neighbor James Decode audio envelope/Correlate Decoding

Scientific Results Percentage Correct

Ask question Imagine answer Decode/Classify Display result Demo

What would the Queen not like to see?

Applause Quiet please

What's up in the sky?

Andromeda galaxy is formed by billions of ...

Over the Thames river you can find...

What is the Capital of England?

Why should you use sunscreen?

What was Hitler's dream?

What can be seen in the night sky?

What would cause gridlock in a major city?

What melody did Mozart compose as a child?

What is the airspeed velocity of an unladen swallow?

Thank you

What is the most shining thing you've ever seen?

What is the main source of energy for our planet?

When I look up into the sky I see...

What would an Irishman like to see?

Human Auditory Cognition