1 / 26

Computer speech

Computer speech. Speech synthesis Recording and sampling Speech recognition Apr. 5. Speech synthesis. Wolfgang von Kempelen (1734-1804) constructed one of the first working synthesizers. It had a reed that kept vibrating by an airstream from bellows.

cora
Download Presentation

Computer speech

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer speech Speech synthesis Recording and sampling Speech recognition Apr. 5

  2. Speech synthesis • Wolfgang von Kempelen (1734-1804) constructed one of the first working synthesizers. • It had a reed that kept vibrating by an airstream from bellows. • The sound from the reed was applied to a box made of leather and wood (the vocal tract), a movable flap inside it (the tongue), and a shutter at one end (lips). LING 001 Introduction to Linguistics, Spring 2010

  3. Speech synthesis • At the beginning of the 20th century, the progress in electrical engineering made it possible to synthesize speech sounds by electrical means. • The first device of this kind that attracted the attention of a wider public, was the VODER, developed by Homer Dudley at Bell Labs. LING 001 Introduction to Linguistics, Spring 2010

  4. Speech synthesis • The VODER was based on VOCODER (VOice enCODER), which uses a series of band pass filters to analyze, transmit, and synthesize speech sounds. LING 001 Introduction to Linguistics, Spring 2010

  5. Speech synthesis • Pattern playback was developed by Frank Cooper at the Haskins Labs and completed in 1950. It works like an inverse of a spectrograph. • Light from a lamp goes through a rotating disk then through spectrogram into photovoltaic cells. • The amount of light that gets transmitted at each frequency band corresponds to amount of acoustic energy at that band shown on the spectrogram. LING 001 Introduction to Linguistics, Spring 2010

  6. Computer speech synthesis • Articulatory synthesizers model the movement of the articulators and the acoustics of the vocal tract. • Articulatory synthesis has never made it out of the laboratory. • Formant synthesizers start with the acoustics, based on the source-filter model of speech production. • Formant synthesizers enjoyed a long commercial history while computers were relatively underpowered. • Concatenative systems use databases of stored speech to assemble new utterances. • Today most commercial systems are concatenative, with many being so-called unit selection approaches. LING 001 Introduction to Linguistics, Spring 2010

  7. Formant synthesis LING 001 Introduction to Linguistics, Spring 2010

  8. Concatenative synthesis • A speech segment is synthesized by simply playing back a recorded waveform with matching phoneme string. • An utterance is synthesized by concatenating together several speech segments. • Issues: • What type of speech segment (unit) to use? Phoneme: ~ 40 Diphone: ~1500 Syllable: ~15K Word: ~100k - 1.5M Sentence: ∞ • How to select the best string of speech segments from a given library of segments? • How to alter the prosody of the speech segments to best match the desired output prosody? LING 001 Introduction to Linguistics, Spring 2010

  9. Diphone Synthesis • Units are diphones; middle of one phone to middle of next. A diphone r-ih, for example, includes from the middle of the r phoneme to the middle of the ih phoneme. • Mid phone is more stable than edge; the transition between two phones is retained. LING 001 Introduction to Linguistics, Spring 2010

  10. Diphone synthesis [From Richard Sproat] LING 001 Introduction to Linguistics, Spring 2010

  11. Unit Selection synthesis • Larger and variable units: from diphones to sentences. • Large representative database, 10 hours of speech or more, multiple copies of each unit type. • Use search to find best sequence of units based on target and joint costs. • Prosodic modification is often avoided, as selected targets may already be close to desired prosody, little or no signal processing applied to each unit. LING 001 Introduction to Linguistics, Spring 2010

  12. Unit Selection synthesis [From Dan Jurafsky] LING 001 Introduction to Linguistics, Spring 2010

  13. Text-To-Speech Demos • ATT: • http://www.research.att.com/~ttsweb/tts/demo.php • Festvox: • http://festvox.org/voicedemos.html • IBM • http://www.research.ibm.com/tts/coredemo.shtml • Nuance: • http://212.8.184.250/tts/demo_login.jsp • http://www.ivona.com/ • http://www.neospeech.com/ • http://www.cereproc.com/products/voices • Roger Ebert gets his new voice! (YouTube) LING 001 Introduction to Linguistics, Spring 2010

  14. Recording • Digital recording: The process of converting speech waves into computer-readable format is called digitization, or A/D conversion. LING 001 Introduction to Linguistics, Spring 2010

  15. Sampling • In order to transform sound into a digital format, you must sample the sound. The computer takes a snapshot of the sound level at small time intervals while you are recording. • The number of samples taken each second is called the sampling rate. The more samples that are taken, the better sound quality. But we also need more storage space for higher quality sound. • For speech recordings, in most cases a sampling rate of 10k Hz is enough. 44100 Hz 22050 Hz 11025 Hz 8000 Hz 5000 Hz LING 001 Introduction to Linguistics, Spring 2010

  16. Sampling • Nyquist-Shannon theorem: • When sampling a signal (e.g., converting from an analog signal to digital), the sampling frequency must be greater than twice the highest frequency in the input signal in order to be able to reconstruct the original perfectly from the sampled version. • Aliasing: If the sampling frequency is less than twice the highest frequency component, then frequencies in the original signal that are above half the sampling rate will be "aliased" and will appear in the resulting signal as lower frequencies. • Anti-Aliasing filter: typically a low-pass filter that is applied before sampling to ensure that no components with frequencies greater than half the sample frequency remain. LING 001 Introduction to Linguistics, Spring 2010

  17. Audio file formats • There are a number of different types of Audio files. • “.wav” files are commonly used for storing uncompressed sound files, which means that they can be large in size - around 10MB per minute of music. • “.mp3” files use the "MPEG Layer-3" codec (compressor-decompressor). “mp3” files are compressed to roughly one-tenth the size of an equivalent .wav file while maintaining good audio quality. • “.aiff” is the standard audio file format used by Apple. It is like a wav file for the Mac. LING 001 Introduction to Linguistics, Spring 2010

  18. Praat: doing phonetics by computer LING 001 Introduction to Linguistics, Spring 2010

  19. Speech recognition • Goal: to convert an acoustic signal O into a word sequence W. • Statistics-based approach: What is the most likely sentence out of all sentences in the language L given some acoustic input O? • Treat acoustic input O as sequence of individual observations • O = o1,o2,o3,…,ot • Define a sentence as a sequence of words: • W = w1,w2,w3,…,wn LING 001 Introduction to Linguistics, Spring 2010

  20. Speech recognition architecture • Solution: search through all possible sentences. Pick the one that is most probable given the waveform/observation. Bayes’ rule P(O) is the same for each W LING 001 Introduction to Linguistics, Spring 2010

  21. Speech recognizer components • Acoustic modeling: Describes the acoustic patterns of phones in the language. 1. Feature extraction 2. Hidden Markov Model • Lexicon (pronouncing dictionary): Describes the sequences of phones that make up words in the language. • Language modeling: Describes the likelihood of various sequences of words being spoke in the language. LING 001 Introduction to Linguistics, Spring 2010

  22. Acoustic modeling • A vector of 39 features is extracted at every 10 ms from 20-25 ms of speech. • Each phone is represented as an Hidden Markov Model (HMM) that consists of three states: the beginning part (s1), the middle part (s2), and the end part (s3). Each state is represented by a Gaussian model on the 39 features. LING 001 Introduction to Linguistics, Spring 2010

  23. Lexicon • The CMU pronouncing dictionary: a pronunciation dictionary for American English that contains over 125,000 words and their phone transcriptions. http://www.speech.cs.cmu.edu/cgi-bin/cmudict • CMU dictionary uses 39 phonemes (in ARPABET), word stress is labeled on vowels: 0 (no stress); 1 (primary stress); 2 (secondary stress). PHONETICS F AH0 N EH1 T IH0 K S COFFEE K AA1 F IY0 COFFEE(2) K AO1 F IY0 RESEARCH R IY0 S ER1 CH RESEARCH(2) R IY1 S ER0 CH LING 001 Introduction to Linguistics, Spring 2010

  24. Language Modeling • We want to compute the probability of a word sequence, p(w1,w2,w3,…,wn). • Using the Chain rule, we have, for example: p(speech, recognition, is, very, fun) = p(speech)*p(recognition|speech)*p(is|speech, recognition)*p(very|speech, recognition, is)*p(fun|speech, recognition, is, very) • Learn p(fun|speech, recognition, is, very) from data? - we’ll never be able to get enough data to compute the probabilities of long sentences. • Instead, we need to make some Markov assumptions: • Zeroth order: p(fun|speech, recognition, is, very) = p(fun) - unigram • First order: p(fun|speech, recognition, is, very) = p(fun|very) - bigram • Second order: p(fun|speech, recognition, is, very) = p(fun|is, very) - trigram … LING 001 Introduction to Linguistics, Spring 2010

  25. State-of-the-art ASR performance LING 001 Introduction to Linguistics, Spring 2010

  26. Challenges in ASR • Robustness and Adaptability – to changing conditions (different mic, background noise, new speaker, different speaking rate, etc.) • Language Modelling – the role of linguistics in improving the language models? • Out-of-Vocabulary (OOV) Words –Systems must have some method of detecting OOV words, and dealing with them in a sensible way. • Spontaneous Speech – disfluencies (filled pauses, false starts, hesitations, ungrammatical constructions etc) remain a problem. • Prosody –Stress, intonation, and rhythm convey important information for word recognition and the user's intentions (e.g., sarcasm, anger). • Accent, dialect and mixed language – non-native speech is a huge problem, especially where code-switching is commonplace. LING 001 Introduction to Linguistics, Spring 2010

More Related