1 / 30

74.419 Artificial Intelligence 2004 Speech & Natural Language Processing

This article explores the areas of speech and natural language processing, including spoken language understanding, speech recognition, signal processing, and phonetic analysis. It discusses the different components of speech signals, the analysis process, and recognition methods using hidden Markov models and neural networks.

lanelong
Download Presentation

74.419 Artificial Intelligence 2004 Speech & Natural Language Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 74.419 Artificial Intelligence 2004 Speech &Natural Language Processing • Natural Language Processing • written text as input • sentences (well-formed) • Speech Recognition • acoustic signal as input • conversion into written words • Spoken Language Understanding • analysis of spoken language (transcribed speech)

  2. Speech & Natural Language Processing • Areas in Natural Language Processing • Morphology • Grammar & Parsing (syntactic analysis) • Semantics • Pragamatics • Discourse / Dialogue • Spoken Language Understanding • Areas in Speech Recognition • Signal Processing • Phonetics • Word Recognition

  3. Speech Production & Reception • Sound and Hearing • change in air pressure sound wave • reception through inner ear membrane / microphone • break-up into frequency components: receptors in cochlea / mathematical frequency analysis (e.g. Fast-Fourier Transform FFT)  Frequency Spectrum • perception/recognition of phonemes and subsequently words (e.g. Neural Networks, Hidden-Markov Models)

  4. Speech Recognition Phases • Speech Recognition • acoustic signal as input • signal analysis - spectrogram • feature extraction • phoneme recognition • word recognition • conversion into written words

  5. Speech Signal • Speech Signal • composed of different (sinus) waves with different frequencies and amplitudes • Frequency- waves/second  like pitch • Amplitude - height of wave  like loudness • + noise (not sinus wave) • Speech Signal • composite signal comprising different frequency components

  6. Waveform (fig. 7.20) Amplitude/ Pressure Time "She just had a baby."

  7. Waveform for Vowel ae(fig. 7.21) Amplitude/ Pressure Time Time

  8. Speech Signal Analysis • Analog-Digital Conversion of Acoustic Signal • Sampling in Time Frames (“windows”) • frequency = 0-crossings per time frame •  e.g. 2 crossings/second is 1 Hz (1 wave) •  e.g. 10kHz needs sampling rate 20kHz • measure amplitudes of signal in time frame •  digitized wave form • separate different frequency components • FFT (Fast Fourier Transform) • spectrogram • other frequency based representations • LPC (linear predictive coding), • Cepstrum

  9. Waveform and Spectrogram (figs. 7.20, 7.23)

  10. Waveform and LPC Spectrum for Vowel ae(figs. 7.21, 7.22) Amplitude/ Pressure Time Energy Formants Frequency

  11. Speech Signal Characteristics From Signal Representation derive, e.g. • formants - dark stripes in spectrum strong frequency components; characterize particular vowels; gender of speaker • pitch– fundamental frequency baseline for higher frequency harmonics like formants; gender characteristic • change in frequency distribution characteristic for e.g. plosives (form of articulation)

  12. Video of glottis and speech signal in lingWAVES (from http://www.lingcom.de)

  13. Phoneme Recognition Recognition Process based on • features extracted from spectral analysis • phonological rules • statistical properties of language/ pronunciation Recognition Methods • Hidden Markov Models • Neural Networks • Pattern Classification in general

  14. Pronunciation Networks / Word Models as Probabilistic FAs (fig 5.12)

  15. Pronunciation Network for 'about' (fig 5.13)

  16. Word Recognition with Probabilistic FA / Markov Chain (fig 5.14)

  17. Viterbi-Algorithm The Viterbi Algorithm finds an optimal sequence of states in continuous Speech Recognition, given an observation sequence of phones and a probabilistic (weighted) FA. The algorithm returns the path through the automaton which has maximum probability and accepts the observed sequence.

  18. Viterbi-Algorithm (fig 5.19) function VITERBI(observations of len T,state-graph) returns best-path num-statesNUM-OF-STATES(state-graph) Create a path probability matrix viterbi[num-states+2,T+2] viterbi[0,0]1.0 for each time step t from 0 to T do for each state s from 0 to num-states do for each transition s0 from s specified by state-graph new-scoreviterbi[s, t] * a[s,s0] * bs0 (ot ) if ((viterbi[s0,t+1] = 0) jj (new-score > viterbi[s0, t+1])) then viterbi[s0, t+1]new-score back-pointer[s0, t+1]s Backtrace from highest probability state in the final column of viterbi[] and return path

  19. Speech Recognition Acoustic / sound wave Filtering, Sampling Spectral Analysis; FFT Frequency Spectrum Features (Phonemes; Context) Signal Processing / Analysis Phoneme Recognition: HMM, Neural Networks Phonemes Grammar or Statistics Phoneme Sequences / Words Grammar or Statistics for likely word sequences Word Sequence / Sentence

  20. Speech Recognizer Architecture (fig. 7.2)

  21. Speech Processing - Important Types and Characteristics single word vs. continuous speech unlimited vs. large vs. small vocabulary speaker-dependent vs. speaker-independent training Speech Recognition vs. Speaker Identification

  22. Additional References Hong, X. & A. Acero & H. Hon: Spoken Language Processing. A Guide to Theory, Algorithms, and System Development. Prentice-Hall, NJ, 2001 Figures taken from: Jurafsky, D. & J. H. Martin, Speech and Language Processing, Prentice-Hall, 2000, Chapters 5 and 7. lingWAVES (from http://www.lingcom.de

More Related