1 / 16

Faster and Smaller N-Gram LMs Adam Pauls and Dan Klein

Faster and Smaller N-Gram LMs Adam Pauls and Dan Klein. Presented by SUN Jun. Overview. N-gram LMs A short review of LM implementation Trie Array: implicit Trie This work: Combination of Multiple techniques Implicit Encoding of query word Variable length encoding for compression

maddock-walters
Download Presentation

Faster and Smaller N-Gram LMs Adam Pauls and Dan Klein

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Faster and Smaller N-Gram LMsAdam Pauls and Dan Klein Presented by SUN Jun

  2. Overview • N-gram LMs • A short review of LMimplementation • Trie • Array: implicit Trie • This work: Combination of Multiple techniques • Implicit Encoding of query word • Variable length encoding for compression • Speed up for decoder

  3. Back-Off LM • LM: An n-gram LM represents the probability of a word sequence, given history • Back-Off LM: Trust the highest order language model that contains n-gram

  4. Implementation of Back-off LM • File based • Trie • Reverse Trie • Array-a: implicit Trie • Array-b: implicit Trie with reverse index to parent

  5. This paper • This work: Combination of Multiple techniques • Implicit Encoding of query word • Variable length encoding for compression • Speed up for decoder

  6. Implicit Encoding of query word • Sorted array

  7. Implicit Encoding of query word • Hash Table

  8. Implicit Encoding of query word • We can exploit this redundancy by storing only the context offsets in the main array, using as many bits as needed to encode all context offsets (32 bits for Web1T). • In auxiliary arrays, one for each n-gram order, we store the beginning and end of the range of the trie array in which all (wi; c) keys are stored for each wi.

  9. Variable length encoding for compression

  10. Speed up decoder • Repetitive Queries • By cache • Scrolling Queries

  11. Experiements

  12. Exp

  13. Exp

  14. Exp

  15. END

More Related