370 likes | 609 Views
Advanced Seminar in Data Structures. 28/12/2004: An Analysis of the Burrows-Wheeler Transform (Giovanni Manzini). Presented by Assaf Oren. Topics. Introduction Burrows-Wheeler Transform Move – to – Front Empirical Entropy Order0 coder Analysis of the BW0 algorithm Run-Length encoding
E N D
Advanced Seminar in Data Structures 28/12/2004: An Analysis of the Burrows-Wheeler Transform (Giovanni Manzini) Presented by Assaf Oren Data Structures Seminar
Topics • Introduction • Burrows-Wheeler Transform • Move–to–Front • Empirical Entropy • Order0 coder • Analysis of the BW0 algorithm • Run-Length encoding • Analysis of BW0RLalgorithm Data Structures Seminar
Introduction • BWT-based algorithm: • Takes the input string s • Transforms it to bwt(s) • |bwt(s)| = |s| • Compress bwt(s) with compressor A • The compressed string will be A(bwt(s)) Data Structures Seminar
Introduction (con’t) • Notations • Recording scheme:tra() A transformation with no compression • Coding scheme : A() An algorithm which designed to reduce the size of the input Data Structures Seminar
BWT-based Alg’ Properties • Even when using a simple compression alg’, A(bwt(s)) will have a good compression ratio • The very simple and clean alg’ from Nelson[1996], outperforms the PkZip package. • Other, more advanced BWT compressors are Bzip [Seward 1997] and Szip [Schindler 1997]. • BWT-based compressors achieve a very good compression ratio using relatively small resources • Arnold and Bell [2000], Fenwick [1996a] Data Structures Seminar
nova 25% man bzip2 bzip2(1) bzip2(1) NAME bzip2, bunzip2 - a block-sorting file compressor, v1.0.2 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files SYNOPSIS bzip2 [ -cdfkqstvzVL123456789 ] [ filenames ... ] bunzip2 [ -fkvsVL ] [ filenames ... ] bzcat [ -s ] [ filenames ... ] bzip2recover filename DESCRIPTION bzip2 compresses files using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, and approaches the performance of the PPM family of sta tistical compressors. Data Structures Seminar
BWT-based Alg’ Properties (cont’) • Works very well in practice, but no satisfactory proof has been given for their compression ratio. • Previous proofs were done: • Assuming the input string is a finite-order Markov source • Sadakane [1997;1998] • To get bounds on the speed at which the average compression ratio approaches the entropy. • Effros [1999] Data Structures Seminar
The Burrows-Wheeler Transform • Background • Part of a research for DIGITAL released at 1994 • Based on a previously unpublished transformation discovered by Wheeler in 1983 • Technical • The resulting output block contains exactly the same data elements that it started with • Performed on an entire block of data at once • Reversible Data Structures Seminar
The Burrows-Wheeler Transform(cont’) • Append # to the end of s • # is unique and smaller then anyother character • Form a Matrix M whoserows are the cyclic shifts of s# • Sort the rows right toleft Data Structures Seminar
The Burrows-Wheeler Transform(cont’) • The output of BWT is the column F = “msspipissii” and the number 3 (the position of #) Data Structures Seminar
The Burrows-Wheeler Transform(cont’) • Observations: • Every column of M is a permutation of s#. • Each character in L is followed in s# by the corresponding character in F. • For any character c, the ith occurrence of c in F corresponds the the ith occurrence of c in L. • How to reconstruct s: • Sort bwt(s) to get column L. (column F is bwt(s)) • F1 is the first character of s. • By applying observation3 we get that ‘m’ (is the same ‘m’ from L6), and obsetvation2 will tell us that F6 is the next character of s. Data Structures Seminar
The Burrows-Wheeler Transform(cont’) Data Structures Seminar
The Burrows-Wheeler Transform(cont’) • Why this transform is so helpful ? • BWT collects together the symbols following a given context. • Formally: • For each substring w of s, the characters following w in s are grouped together inside bwt(s) • More formally!!! Data Structures Seminar
Move–to–Front (mtf ) • Another recording scheme • Suggested be B&W to be used after applying BWT on string s • s` = mtf(bwt(s)) • |mtf(bwt(s))| = |bwt(s)| = |s| • If s is over {a1, a2, … , ah} then s` is over {0, 1, …, h-1} Data Structures Seminar
Move–to–Front (cont’) • For each letter (left-to-right): • Write the number of other letters since the last time the current letter appeared. • Example: a a b a c a a c c b a 0 0 1 1 2 1 0 1 0 2 2 a a b a c a a c c b a Data Structures Seminar
Move–to–Front (cont’) • Why this transform is helpful ? • Transforms the local homogeneous of bwt(s) to global homogeneous • Formally if we had After mtf both strings will probably have the same small numbers. Data Structures Seminar
Huffman coding • Sets binary values to letters according to their frequency • For example: • A = {a, b, c} • In our string the frequency is: • The coding will be: a = 300 b = 150 c = 150 a = `0` b = `10` c = `11` Data Structures Seminar
Arithmetic coding Data Structures Seminar
The Empirical Entropy of a string • s = our string • n = |s| • A = our Alphabet • h = |A| • ni = number of occurrences of the symbol ai inside s • H0(s) = the zeroth order empirical entropy of s Data Structures Seminar
Intuition for the Empirical Entropy For each symbol For each appearance of this symbol in the text The number of bits that will be needed torepresent it with an ultimate uniquely decodable code Data Structures Seminar
The kth order Empirical Entropy • We can achieve a greater compression if the codeword depends on the k symbols that precedes the coded symbol • For example: s = “abcabcabd” the codeword for ‘ab’ could be abs = ccd • And formally we can define: Data Structures Seminar
Examples of Hk(s) • Example 1: • K=1, s = mississippi • ms = i, is = ssp, ss = sisi, ps = pi • Example 2: • K=1, s = cc(ab)n • as = bn, bs = an-1, cs = ca 0 0 1 Data Structures Seminar
The modified Empirical Entropy • Modified in order to avoid cases of Data Structures Seminar
Empirical Entropy and BWT • We saw that…………… • We know that………… • If we had an Ideal algorithm A…………… We get: We reduced the problem of compressing up to kth order entropy to the problem of compressing distinct portions of the input string up to their zeroth order entropy. Data Structures Seminar
An Order0 coder • A coder with a compression ratio that is close to the zeroth order empirical entropy. • Formally: • For static Huffman coding, = 1 • For a simple arithmetic coder, =~10-2 • Howard and Vitter [1992a] Data Structures Seminar
Analysis of the BW0 algorithm • BW0(s) = Order0(mtf(bwt(s))) • We would like to achieve: • For now lets assume Theorem 4.1 on mtf(s): Data Structures Seminar
Proof of BW0 • We saw that if then for t ≤ hk • For combined with theorem 4.1: • With our knowledge on Order0: Get get: Data Structures Seminar
Proof of Theorem 4.1 • Lemma 4.3 • Lemma 4.4 Data Structures Seminar
Proof of Theorem 4.1 (cont’) • Lemma 4.5 • Lemma 4.6 • Lemma 4.7 Data Structures Seminar
Proof of Theorem 4.1 (cont’) • Lemma 4.8 • It is sufficient to prove that: Data Structures Seminar
Proof of Theorem 4.1 (cont’) • By applying Lemma 4.3 and 4.5 we get: • And: • And: Data Structures Seminar
Analysis of BW0RL algorithm • BW0RL(s) = Order0(RLE(mtf(bwt(s)))) • RLE(s) • Let 0 and 1 be two symbols that are not belong to the alphabet • For m ≥ 1, B(m) = m+1 written in binary with 0 and 1,discarding the MSB • B(1) = 0, B(2) = 1, B(3) = 00. B(4) = 01, B(5) = 10… • RLE(s) will replace 0m zeros in s with B(m) • Given s = “110022013000”, RLE(s) = “1112201300 ” • |RLE(s)| ≤ |s|, since log(m+1) ≤ m Data Structures Seminar
Analysis of BW0RL (cont’) • Theorem 5.1 • … • … • Theorem 5.8 Data Structures Seminar
Analysis of BW0RL (cont’) • Locally -Optimal Algorithm • For all t > 0, there exists a constant ct, that for any partition s1, s2, … , stof the string s we have: • A locally -Optimal Algorithm combined with BWT is bounded by: Data Structures Seminar
A bit of practicality • A nice article by Mark Nelson • http://www.dogma.net/markn/articles/bwt/bwt.htm • Includes source code + measurements • Usage: RLE input-file| BWT | MTF | RLE | ARI > output-file UNARI input-file |UNRLE|UNMTF|UNBWT|UNRLE > output-file Data Structures Seminar
A bit of practicality (cont’) Data Structures Seminar
A bit of practicality (cont’) The End Data Structures Seminar