exercise in the previous class (1)

1. exercisein the previous class (1) • Define (one of) (15, 11) Hamming code: • construct a parity check matrix, and • determine the corresponding generator matrix m = 4 parity check matrix 24 – 1 nonzero vectors generator matrix

2. exercisein the previous class (2) • Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4. • weight 1: HuT= hi = 0...contradiction • weight 2: HuT= hi + hj= 0 ...contradiction • weight 3: HuT= hi+hi + hl = 0, but... • column vectors have odd weight, and hi+ hihas even weight • hi+ hi= hl never happens...contradiction • weight 4: we have 10001110C

3. previous class • Hamming code • one-bit error correcting, one of perfect codes • minimum Hamming distance • smallest distance between two codewords • gives the maximum number of correctable errors • easy but “rough” estimation of performance

4. today’s class two independent topics: • performance evaluation using weight distribution • slightly complicated (ちょっと複雑) • precise and detailed estimation is possible • cyclic code • a subclass of linear codes • advantages in realizations

5. is the minimum distance everything? If the minimum Hamming distance of C is dmin, thenC can correct (dmin – 1) / 2 or less bit errors. Consider three codes with the same code length... C1 C2 C3 ? < = dmin dmin dmin small # of neighbors many neighbors C2 seems better than C3...

6. weight distribution C={000000,100111,010110,110001, 001011,101100,011101,111010}. from the view of 000000, 1 codeword at distance 0 4 codewordsat distance 3 3 codewordsat distance 4 3 distance distribution from 000000 000000 4 in linear codes, all codewords have the same distance distribution 1 codeword with weight 0 4 codewordswith weight 3 3 codewordswith weight 4 weight distribution

7. performance evaluation with weight distribution assume... • BSC with bit error probability p • one-bit error correcting • send all-zero codewordand evaluate the probabilities of... • correctly decoded • wrongly decoded • error detected C={000000,100111,010110,110001, 001011,101100,011101,111010}. 1 codeword with weight 0 4 codewordswith weight 3 3 codewordswith weight 4 000000 decoder ? r received vector

8. the domain of the received vector • the received vectorrV6 • V6 contains eight decoding territories C={000000,100111,010110,110001, 001011,101100,011101,111010}. • correctly decoded; if r falls in the territory of 000000 • wrongly decoded; if r falls in the territory of the other codewords • error detected; if r does not fall in the territories (not all vectors shown)

9. probability of correct decoding correctly decoded if r belongs to T ={000000, 000001, 000010, 000100, 001000, 010000, 100000} (1 – p)6 prob. that rT and wH(r) = 0 (1 – p)6 prob. that rT and wH(r) = 1 6p( 1 – p)5 p( 1 – p)5 r is decoded correctly with probability Pc= (1 – p)6 + 6p(1-p)5

10. probability of wrong decoding (1) case1: r belongs to the territory of a weight-3codeword001011 T = {001011, 001010, 001001, 001111, 000011, 011011, 101011} weight 2 weight 3 prob. that rT and wH(r) = 3 p3(1 – p)3 weight 4 prob. that rT and wH(r) = 2 3p2( 1 – p)4 prob. that rT and wH(r) = 4 3p4( 1 – p)2 r is decoded to the wrong codeword 001011 with probability 3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2 ... the same discussion holds for other codewords with weight 3

11. probability of wrong decoding (2) case2: r belongs to the territory of a weight-4codeword100111 T = {100111, 100110, 100101, 100011, 101111, 110111, 000111} weight 3 weight 4 prob. that rT and wH(r) = 4 p4(1 – p)2 weight 5 prob. that rT and wH(r) = 3 4p3( 1 – p)3 prob. that rT and wH(r) = 5 2p5( 1 – p) r is decoded to the wrong codeword 100111 with probability 4p3(1 – p)3+ p4(1 – p)2+ 2p5(1 – p) ... the same discussion holds for other codewords with weight 4

12. probability of wrong decoding (3) wrongly decoded; Pe= 4×(falls in the territory of one of weight-3 codewords) + 3×(falls in the territory of one of weight-4 codewors) = 4×(3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2 ) + 3×(4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p)) = 12p2(1 – p)4 + 16p3(1 – p)3 + 15p4(1 – p)2 + 6p5(1 – p) error detected with probability Pd = 1 – Pc – Pe C={000000,100111,010110,110001, 001011,101100,011101,111010}. 1 codeword with weight 0 4 codewordswith weight 3 3 codewordswith weight 4

13. the probabilities in a graph

14. comparison example • (7,4) Hamming code: 0000000, 1000101, 0100111, 0010110, 0010110, 1010011, 0110001, 1110100, 0001011, 1001110, 0101100, 1101001, 0011101, 1011000, 0111010, 1111111 • (9, 4) 2D code: 000000000, 000101011, 001001101, 001100110, 010010011, 010111000, 011011110, 011110101, 100010101, 100111110, 101011000, 101110011, 110000110, 110101101, 111001011, 111100000 weight 0 3 4 7 #codewords 1 7 7 1 weight 0 4 6 #codewords 1 9 6

15. comparison of two codes Hamming code seems better, but...

16. “retransmission” re-transmission of broken data is allowed in some systems 2D code + one-time re-transmission is better for p ≤ 0.28.

17. half the way... two independent topics: • performance evaluation using weight distribution • slightly complicated (ちょっと複雑) • precise and detailed estimation is possible • cyclic code • a subclass of linear codes • advantages in realizations

18. codes linear codes cyclic codes cyclic codes • Linear codes are characterized by matrix operations. • Encoders/decoders are realized by combinatorial circuits. • simple but not scalable ... O(n2) for the code length n • cyclic codes (巡回符号) • a subclass of linear codes • Encoders/decoders are realized by shift registers. • complex but scalable ... O(n) complexity

19. x4 + x3 + x2 + 1 x3 + x + 1 x4 + x3 + x2 + 1 x3 + x + 1 ×) +) x4 + x3 + x2 + 1 x4 + x2 + x x5 + x4 + x3 + x x7 + x6 + x5+ x3 x7 + x6+ x3 + x2+ x + 1 preliminary (1) We represent binary vectors by binary polynomials (多項式). addition of polynomials: (subtraction) 11101 01011 +) 11101 x4 + x3 + x2 + x + 1 10110 11101 01011 multiplication of polynomials: ×) 11101 11101 11101 11001111 multiply by xm = left-shift of m bits

20. 111 11101 ) 1010000 11101 10010 11101 11110 11101 11 preliminary (2) • addition = subtraction = XOR of coefficients • the division circuit is easily implemented by a shift register division of polynomials: x2 + x + 1 x4 + x3 + x2 + 1 ) x6 + x4 x6 + x5 + x4 + x2 x5 + x2 x5 + x4 + x3 + x x4 + x3 + x2 + x x4 + x3 + x2 + 1 x + 1

21. division circuit (1) • divide p(x) = x6+ x4byq(x) = x4+ x3 + x2 + 1... 1 1 1 0 1 q(x) p(x) 1 0 1 0 0 0 0 quotient (商) remainder (剰余) store p(x) to the registers if MSB = 1, then AND gates are activated, and registers are XOR’ed with q(x) left-shift, and go to step 1

22. 111 11101 ) 1010000 11101 010010 11101 11110 11101 11 division circuit (2) 1 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 1 • one-cycle of operations • =one-step of the calculus • contents of the register • = remainder of the division 0 0 0 1 0 0 1

23. 1101 11101 ) 10000001 11101 1101001 11101 11101 11101 0 definition of cyclic codes How to construct an (n, k) cyclic code: • length n, k information symbols, m = n –k parity symbols • Step 1: choose an order-m polynomial G(x) which divides xn + 1. • Step 2: C = {multiples of G(x) with order < n} Example: n = 7, m = 4: G(x) = x4 + x3 + x2 + 1 divides x7+1, because x7+1 = (x3 + x2 + 1) (x4 + x3 + x2 + 1). G(x)...generator polynomial x7+1 G(x)

24. construction of a cyclic code • Step 1: choose an order-m polynomial G(x) which divides xn + 1. • Step 2: C = {multiples of G(x) with order < n} n = 7, m = 4: G(x) = x4 + x3 + x2 + 1 (x2+x+1)× (x2+x+1)× (x2+x+1)× (x2+x+1)× (x2+x+1)× (x2+x+1)× (x2+x+1)× (x2+x+0)× 0000000 0011101 0111010 0100111 G(x) G(x) G(x) G(x) G(x) G(x) G(x) G(x) = C= 1110100 1101001 1001110 1010011

25. properties of cyclic codes (1) Lemma: a cyclic code is a linear code proof: show that, for c1, c2C, we have c1+c2C • c1 C  c1 = f1(x)G(x) • c2C  c2= f2(x)G(x) • c1+c2 = (f1(x)+f2(x)) G(x) C

26. properties of cyclic codes (2) Lemma: if (an-1, an-2, ..., a0)C, then(an-2, ..., a0, an-1)C proof: LetW(x) = an-1xn-1 + ... + a0 and W’(x) = an-2xn-1+ ... + a0x + an-1. • W(x) is a multiple of G(x) • W’(x) = an-2xn-1 + ... + a0x + an-1 = an-1xn + an-2xn-1 + ... + a0x + an-1+ an-1xn = xW(x)+an-1(xn+ 1) multiple of G(x) multiple of G(x)  W’(x) is a multiple of G(x), and (an-2, ..., a0, an-1)C. A cyclic code C is closed for a cyclic shift.

27. three approaches for encoding three approaches for an encoding procedure: • matrix approach • use a generator matrix ... no advantage of cyclic codes • multiplication approach • codeword = (info. symbols) × G(x) • the code not systematic ( p.24) • division approach • slightly complicated (for human) • make the code systematic • easily implemented by shift registers

28. x4 encoding by division • represent information symbols by a polynomial A(x) • divide A(x)xm by G(x), and let B(x) be the remainder • let A(x)xm + B(x) be a codeword • encoding 011, with n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1 • A(x) = x + 1 • A(x)x4 = x5 + x4 = x(x4 + x3 + x2 + 1) + (x3 + x), and B(x) = x3 + x • A(x)x4+ B(x)= x5 + x4 + x3 + x, the codeword = 0111010 A(x) W(x) = A(x) x4 + B(x) B(x) ÷ dividend remainder G(x) divisor quotient D(x)

29. did we really make “encoding”? simple question: Is A(x)xm + B(x) really a codeword?  is A(x)xm + B(x) divided by G(x)? Yes, note that... • B(x) is a remainder of A(x)xm • in a binary world, A(x)xp + B(x) = A(x)xp – B(x) • “A(x)xp– B(x)” = “remove the remainder” • if A(x)xp – B(x) is divided by G(x), then there is no remainder...

30. example n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1 data 000 001 010 011 100 101 110 111 A(x) x2+x+0 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1 A(x)x4 0000000 0010000 0100000 0110000 1000000 1010000 1100000 1110000 B(x) x3+x2+x+0 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 A(x)xm + B(x) 0000000 0011101 0100111 0111010 1001110 1010011 1101001 1110100 • systematic code • encoder ≈ division circuit  O(n) < O(n2) of matrix operation

31. error “detection” of cyclic codes error “detection” is easy for cyclic codes • uC  u (in a polynomial representation) is divided by G(x) = 0 ... no error  0 ... error received u ÷ dividend remainder G(x) divisor quotient • one division circuit is used for encoding and error detection  reduces the cost of realization • Cyclic Redundancy Check (CRC) ... used in many communication systems

32. error “correction” of cyclic codes • general algorithm for all cyclic codes • error-trapping decoder [Kasami 61] TadaoKasami 1930-2007 • special algorithms for special cyclic codes • Berlekamp-Massey algorithm for... • BCH (Bose-Chaudhuri-Hocquenghem) codes • Reed-Solomon Codes I. Reed, 1923- (left) and G. Solomon, 1930-1996 E. Berlekamp, 1940- J. L. Massey, 1934-

33. summary • performance evaluation using weight distribution • slightly complicated (ちょっと複雑) • precise and detailed estimation is possible • cyclic code • a subclass of linear codes • advantages in realizations

34. exercise • Consider the following code C. • determine the weight distribution of C • compute the “three” probabilities (p. 7), and draw a graph • We want to construct a cyclic code with n = 7, k = 4, and m =3. • confirm that G(x) = x3 + x2 + 1 can be a generator polynomial • encode 0110 • decide if 0001100 is a correct codeword or not