1 / 37

Advanced Seminar in Data Structures

Advanced Seminar in Data Structures. 28/12/2004: An Analysis of the Burrows-Wheeler Transform (Giovanni Manzini). Presented by Assaf Oren. Topics. Introduction Burrows-Wheeler Transform Move – to – Front Empirical Entropy Order0 coder Analysis of the BW0 algorithm Run-Length encoding

gotzon
Download Presentation

Advanced Seminar in Data Structures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Advanced Seminar in Data Structures 28/12/2004: An Analysis of the Burrows-Wheeler Transform (Giovanni Manzini) Presented by Assaf Oren Data Structures Seminar

  2. Topics • Introduction • Burrows-Wheeler Transform • Move–to–Front • Empirical Entropy • Order0 coder • Analysis of the BW0 algorithm • Run-Length encoding • Analysis of BW0RLalgorithm Data Structures Seminar

  3. Introduction • BWT-based algorithm: • Takes the input string s • Transforms it to bwt(s) • |bwt(s)| = |s| • Compress bwt(s) with compressor A • The compressed string will be A(bwt(s)) Data Structures Seminar

  4. Introduction (con’t) • Notations • Recording scheme:tra() A transformation with no compression • Coding scheme : A() An algorithm which designed to reduce the size of the input Data Structures Seminar

  5. BWT-based Alg’ Properties • Even when using a simple compression alg’, A(bwt(s)) will have a good compression ratio • The very simple and clean alg’ from Nelson[1996], outperforms the PkZip package. • Other, more advanced BWT compressors are Bzip [Seward 1997] and Szip [Schindler 1997]. • BWT-based compressors achieve a very good compression ratio using relatively small resources • Arnold and Bell [2000], Fenwick [1996a] Data Structures Seminar

  6. nova 25% man bzip2 bzip2(1) bzip2(1) NAME bzip2, bunzip2 - a block-sorting file compressor, v1.0.2 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files SYNOPSIS bzip2 [ -cdfkqstvzVL123456789 ] [ filenames ... ] bunzip2 [ -fkvsVL ] [ filenames ... ] bzcat [ -s ] [ filenames ... ] bzip2recover filename DESCRIPTION bzip2 compresses files using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, and approaches the performance of the PPM family of sta­ tistical compressors. Data Structures Seminar

  7. BWT-based Alg’ Properties (cont’) • Works very well in practice, but no satisfactory proof has been given for their compression ratio. • Previous proofs were done: • Assuming the input string is a finite-order Markov source • Sadakane [1997;1998] • To get bounds on the speed at which the average compression ratio approaches the entropy. • Effros [1999] Data Structures Seminar

  8. The Burrows-Wheeler Transform • Background • Part of a research for DIGITAL released at 1994 • Based on a previously unpublished transformation discovered by Wheeler in 1983 • Technical • The resulting output block contains exactly the same data elements that it started with • Performed on an entire block of data at once • Reversible Data Structures Seminar

  9. The Burrows-Wheeler Transform(cont’) • Append # to the end of s • # is unique and smaller then anyother character • Form a Matrix M whoserows are the cyclic shifts of s# • Sort the rows right toleft Data Structures Seminar

  10. The Burrows-Wheeler Transform(cont’) • The output of BWT is the column F = “msspipissii” and the number 3 (the position of #) Data Structures Seminar

  11. The Burrows-Wheeler Transform(cont’) • Observations: • Every column of M is a permutation of s#. • Each character in L is followed in s# by the corresponding character in F. • For any character c, the ith occurrence of c in F corresponds the the ith occurrence of c in L. • How to reconstruct s: • Sort bwt(s) to get column L. (column F is bwt(s)) • F1 is the first character of s. • By applying observation3 we get that ‘m’ (is the same ‘m’ from L6), and obsetvation2 will tell us that F6 is the next character of s. Data Structures Seminar

  12. The Burrows-Wheeler Transform(cont’) Data Structures Seminar

  13. The Burrows-Wheeler Transform(cont’) • Why this transform is so helpful ? • BWT collects together the symbols following a given context. • Formally: • For each substring w of s, the characters following w in s are grouped together inside bwt(s) • More formally!!! Data Structures Seminar

  14. Move–to–Front (mtf ) • Another recording scheme • Suggested be B&W to be used after applying BWT on string s • s` = mtf(bwt(s)) • |mtf(bwt(s))| = |bwt(s)| = |s| • If s is over {a1, a2, … , ah} then s` is over {0, 1, …, h-1} Data Structures Seminar

  15. Move–to–Front (cont’) • For each letter (left-to-right): • Write the number of other letters since the last time the current letter appeared. • Example: a a b a c a a c c b a 0 0 1 1 2 1 0 1 0 2 2 a a b a c a a c c b a Data Structures Seminar

  16. Move–to–Front (cont’) • Why this transform is helpful ? • Transforms the local homogeneous of bwt(s) to global homogeneous • Formally if we had After mtf both strings will probably have the same small numbers. Data Structures Seminar

  17. Huffman coding • Sets binary values to letters according to their frequency • For example: • A = {a, b, c} • In our string the frequency is: • The coding will be: a = 300 b = 150 c = 150 a = `0` b = `10` c = `11` Data Structures Seminar

  18. Arithmetic coding Data Structures Seminar

  19. The Empirical Entropy of a string • s = our string • n = |s| • A = our Alphabet • h = |A| • ni = number of occurrences of the symbol ai inside s • H0(s) = the zeroth order empirical entropy of s Data Structures Seminar

  20. Intuition for the Empirical Entropy For each symbol For each appearance of this symbol in the text The number of bits that will be needed torepresent it with an ultimate uniquely decodable code Data Structures Seminar

  21. The kth order Empirical Entropy • We can achieve a greater compression if the codeword depends on the k symbols that precedes the coded symbol • For example: s = “abcabcabd” the codeword for ‘ab’ could be abs = ccd • And formally we can define: Data Structures Seminar

  22. Examples of Hk(s) • Example 1: • K=1, s = mississippi • ms = i, is = ssp, ss = sisi, ps = pi • Example 2: • K=1, s = cc(ab)n • as = bn, bs = an-1, cs = ca 0 0 1 Data Structures Seminar

  23. The modified Empirical Entropy • Modified in order to avoid cases of Data Structures Seminar

  24. Empirical Entropy and BWT • We saw that…………… • We know that………… • If we had an Ideal algorithm A…………… We get: We reduced the problem of compressing up to kth order entropy to the problem of compressing distinct portions of the input string up to their zeroth order entropy. Data Structures Seminar

  25. An Order0 coder • A coder with a compression ratio that is close to the zeroth order empirical entropy. • Formally: • For static Huffman coding,  = 1 • For a simple arithmetic coder,  =~10-2 • Howard and Vitter [1992a] Data Structures Seminar

  26. Analysis of the BW0 algorithm • BW0(s) = Order0(mtf(bwt(s))) • We would like to achieve: • For now lets assume Theorem 4.1 on mtf(s): Data Structures Seminar

  27. Proof of BW0 • We saw that if then for t ≤ hk • For combined with theorem 4.1: • With our knowledge on Order0:  Get get: Data Structures Seminar

  28. Proof of Theorem 4.1 • Lemma 4.3 • Lemma 4.4 Data Structures Seminar

  29. Proof of Theorem 4.1 (cont’) • Lemma 4.5 • Lemma 4.6 • Lemma 4.7 Data Structures Seminar

  30. Proof of Theorem 4.1 (cont’) • Lemma 4.8 • It is sufficient to prove that: Data Structures Seminar

  31. Proof of Theorem 4.1 (cont’) • By applying Lemma 4.3 and 4.5 we get: • And: • And: Data Structures Seminar

  32. Analysis of BW0RL algorithm • BW0RL(s) = Order0(RLE(mtf(bwt(s)))) • RLE(s) • Let 0 and 1 be two symbols that are not belong to the alphabet • For m ≥ 1, B(m) = m+1 written in binary with 0 and 1,discarding the MSB • B(1) = 0, B(2) = 1, B(3) = 00. B(4) = 01, B(5) = 10… • RLE(s) will replace 0m zeros in s with B(m) • Given s = “110022013000”, RLE(s) = “1112201300 ” • |RLE(s)| ≤ |s|, since log(m+1) ≤ m Data Structures Seminar

  33. Analysis of BW0RL (cont’) • Theorem 5.1 • … • … • Theorem 5.8 Data Structures Seminar

  34. Analysis of BW0RL (cont’) • Locally -Optimal Algorithm • For all t > 0, there exists a constant ct, that for any partition s1, s2, … , stof the string s we have: • A locally -Optimal Algorithm combined with BWT is bounded by: Data Structures Seminar

  35. A bit of practicality • A nice article by Mark Nelson • http://www.dogma.net/markn/articles/bwt/bwt.htm • Includes source code + measurements • Usage: RLE input-file| BWT | MTF | RLE | ARI > output-file UNARI input-file |UNRLE|UNMTF|UNBWT|UNRLE > output-file Data Structures Seminar

  36. A bit of practicality (cont’) Data Structures Seminar

  37. A bit of practicality (cont’) The End Data Structures Seminar

More Related