3F4 Error Control Coding

1. 3F4 Error Control Coding Dr. I. J. Wassell

2. Introduction • Error Control Coding (ECC) • Extra bits are added to the data at the transmitter (redundancy) to permit error detection or correction at the receiver • Done to prevent the output of erroneous bits despite noise and other imperfections in the channel • The positions of the error control coding and decoding are shown in the transmission model

3. Error Control Coding Modulator (Transmit Filter, etc) Digital Source Source Encoder Line Coding X(w) Hc(w) Channel Transmitter N(w) Noise + Error Control Decoding Demod (Receive Filter, etc) Digital Sink Source Decoder Line Decoding Y(w) Receiver Transmission Model

4. Error Models • Binary Symmetric Memoryless Channel • Assumes transmitted symbols are binary • Errors affect ‘0’s and ‘1’s with equal probability (i.e., symmetric) • Errors occur randomly and are independent from bit to bit (memoryless) 1-p 0 0 p is the probability of bit error or the Bit Error Rate (BER) of the channel p IN OUT p 1 1 1-p

5. Error Models • Many other types • Burst errors, i.e., contiguous bursts of bit errors • output from DFE (error propagation) • common in radio channels • Insertion, deletion and transposition errors • We will consider mainly random errors

6. Error Control Techniques • Error detection in a block of data • Can then request a retransmission, known as automatic repeat request (ARQ) for sensitive data • Appropriate for • Low delay channels • Channels with a return path • Not appropriate for delay sensitive data, e.g., real time speech and data

7. Error Control Techniques • Forward Error Correction (FEC) • Coding designed so that errors can be corrected at the receiver • Appropriate for delay sensitive and one-way transmission (e.g., broadcast TV) of data • Two main types, namely block codes and convolutional codes. We will only look at block codes

8. Block Codes • We will consider only binary data • Data is grouped into blocks of length k bits (dataword) • Each dataword is coded into blocks of length n bits (codeword), where in general n>k • This is known as an (n,k) block code

9. Block Codes • A vector notation is used for the datawords and codewords, • Dataword d = (d1d2….dk) • Codeword c = (c1 c2……..cn) • The redundancy introduced by the code is quantified by the code rate, • Code rate = k/n • i.e., the higher the redundancy, the lower the code rate

10. Block Code - Example • Dataword length k = 4 • Codeword length n = 7 • This is a (7,4) block code with code rate = 4/7 • For example, d = (1101), c = (1101001)

11. 1000 1000 Channel coder Channel decoder Channel Error Control Process Source code data chopped into blocks Codeword (n bits) 101101 Dataword (k bits) Codeword + possible errors (n bits) Dataword (k bits) Error flags

12. Error Control Process • Decoder gives corrected data • May also give error flags to • Indicate reliability of decoded data • Helps with schemes employing multiple layers of error correction

13. Parity Codes • Example of a simple block code – Single Parity Check Code • In this case, n = k+1, i.e., the codeword is the dataword with one additional bit • For ‘even’ parity the additional bit is, • For ‘odd’ parity the additional bit is 1-q • That is, the additional bit ensures that there are an ‘even’ or ‘odd’ number of ‘1’s in the codeword

14. Parity Codes – Example 1 • Even parity (i) d=(10110) so, c=(101101) • d=(11011) so, c=(110110)

15. Dataword Codeword Parity Codes – Example 2 • Coding table for (4,3) even parity code

16. Parity Codes • To decode • Calculate sum of received bits in block (mod 2) • If sum is 0 (1) for even (odd) parity then the dataword is the first k bits of the received codeword • Otherwise error • Code can detect single errors • But cannot correct error since the error could be in any bit • For example, if the received dataword is (100000) the transmitted dataword could have been (000000) or (110000) with the error being in the first or second place respectively • Note error could also lie in other positions including the parity bit

17. Parity Codes • Known as a single error detecting code (SED). Only useful if probability of getting 2 errors is small since parity will become correct again • Used in serial communications • Low overhead but not very powerful • Decoder can be implemented efficiently using a tree of XOR gates

18. Hamming Distance • Error control capability is determined by the Hamming distance • The Hamming distance between two codewords is equal to the number of differences between them, e.g., 10011011 11010010 have a Hamming distance = 3 • Alternatively, can compute by adding codewords (mod 2) =01001001 (now count up the ones)

19. Hamming Distance • The Hamming distance of a code is equal to the minimum Hamming distance between two codewords • If Hamming distance is: 1 – no error control capability; i.e., a single error in a received codeword yields another valid codeword XXXXXXX X is a valid codeword Note that this representation is diagrammatic only. In reality each codeword is surrounded by n codewords. That is, one for every bit that could be changed

20. Hamming Distance • If Hamming distance is: 2 – can detect single errors (SED); i.e., a single error will yield an invalid codeword XOXOXO X is a valid codeword O in not a valid codeword See that 2 errors will yield a valid (but incorrect) codeword

21. Hamming Distance • If Hamming distance is: 3 – can correct single errors (SEC) or can detect double errors (DED) XOOXOOX X is a valid codeword O in not a valid codeword See that 3 errors will yield a valid but incorrect codeword

22. 10011011 X This code corrected this way 11011011 O 11010011 O This code corrected this way 11010010 X Hamming Distance - Example • Hamming distance 3 code, i.e., SEC/DED • Or can perform single error correction (SEC) X is a valid codeword O is an invalid codeword

23. Hamming Distance • The maximum number of detectable errors is • That is the maximum number of correctable errors is given by, where dmin is the minimum Hamming distance between 2 codewords and means the smallest integer

24. Linear Block Codes • As seen from the second Parity Code example, it is possible to use a table to hold all the codewords for a code and to look-up the appropriate codeword based on the supplied dataword • Alternatively, it is possible to create codewords by addition of other codewords. This has the advantage that there is now no longer the need to held every possible codeword in the table.

25. Linear Block Codes • If there are k data bits, all that is required is to hold k linearly independent codewords, i.e., a set of k codewords none of which can be produced by linear combinations of 2 or more codewords in the set. • The easiest way to find k linearly independent codewords is to choose those which have ‘1’ in just one of the first k positions and ‘0’ in the other k-1 of the first k positions.

26. Linear Block Codes • For example for a (7,4) code, only four codewords are required, e.g., • So, to obtain the codeword for dataword 1011, the first, third and fourth codewords in the list are added together, giving 1011010 • This process will now be described in more detail

27. Linear Block Codes • An (n,k) block code has code vectors d=(d1d2….dk) and c=(c1 c2……..cn) • The block coding process can be written as c=dG where G is the Generator Matrix

28. Linear Block Codes • Thus, • ai must be linearly independent, i.e., • Since codewords are given by summations of the ai vectors, then to avoid 2 datawords having the same codeword the ai vectors must be linearly independent

29. Linear Block Codes • Sum (mod 2) of any 2 codewords is also a codeword, i.e., Since for datawords d1 and d2 we have; So,

30. Linear Block Codes • 0 is always a codeword, i.e., Since all zeros is a dataword then,

31. Error Correcting Power of LBC • The Hamming distance of a linear block code (LBC) is simply the minimum Hamming weight (number of 1’s or equivalently the distance from the all 0 codeword) of the non-zero codewords • Note d(c1,c2) = w(c1+ c2) as shown previously • For an LBC, c1+ c2=c3 • So min (d(c1,c2)) = min (w(c1+ c2)) = min (w(c3)) • Therefore to find min Hamming distance just need to search among the 2k codewords to find the min Hamming weight – far simpler than doing a pair wise check for all possible codewords.

32. Linear Block Codes – example 1 • For example a (4,2) code, suppose; a1 = [1011] a2 = [0101] • For d = [1 1], then;

33. Linear Block Codes – example 2 • A (6,5) code with • Is an even single parity code

34. Systematic Codes • For a systematic block code the dataword appears unaltered in the codeword – usually at the start • The generator matrix has the structure, k R R = n - k • P is often referred to as parity bits

35. Systematic Codes • I is k*k identity matrix. Ensures dataword appears as beginning of codeword • P is k*R matrix.

36. Decoding Linear Codes • One possibility is a ROM look-up table • In this case received codeword is used as an address • Example – Even single parity check code; Address Data 000000 0 000001 1 000010 1 000011 0 ……… . • Data output is the error flag, i.e., 0 – codeword ok, • If no error, dataword is first k bits of codeword • For an error correcting code the ROM can also store datawords

37. Decoding Linear Codes • Another possibility is algebraic decoding, i.e., the error flag is computed from the received codeword (as in the case of simple parity codes) • How can this method be extended to more complex error detection and correction codes?

38. Parity Check Matrix • A linear block code is a linear subspace Ssub of all length n vectors (Space S) • Consider the subset Snull of all length n vectors in space S that are orthogonal to all length n vectors in Ssub • It can be shown that the dimensionality of Snull is n-k, where n is the dimensionality of S and k is the dimensionality of Ssub • It can also be shown that Snull is a valid subspace of S and consequently Ssub is also the null space of Snull

39. Parity Check Matrix • Snull can be represented by its basis vectors. In this case the generator basis vectors (or ‘generator matrix’ H) denote the generator matrix for Snull -of dimension n-k = R • This matrix is called the parity check matrix of the code defined by G, where G is obviously the generator matrix for Ssub- of dimension k • Note that the number of vectors in the basis defines the dimension of the subspace

40. Parity Check Matrix • So the dimension of H is n-k (= R) and all vectors in the null space are orthogonal to all the vectors of the code • Since the rows of H, namely the vectors bi are members of the null space they are orthogonal to any code vector • So a vector y is a codeword only if yHT=0 • Note that a linear block code can be specified by either G or H

41. Parity Check Matrix • So H is used to check if a codeword is valid, R = n - k • The rows of H, namely, bi, are chosen to be orthogonal to rows of G, namely ai • Consequently the dot product of any valid codeword with any bi is zero

42. Parity Check Matrix • This is so since, and so, • This means that a codeword is valid (but not necessarily correct) only if cHT = 0. To ensure this it is required that the rows of H are independent and are orthogonal to the rows of G • That is the bi span the remaining R (= n - k) dimensions of the codespace

43. Parity Check Matrix • For example consider a (3,2) code. In this case G has 2 rows, a1 and a2 • Consequently all valid codewords sit in the subspace (in this case a plane) spanned by a1 and a2 • In this example the H matrix has only one row, namely b1. This vector is orthogonal to the plane containing the rows of the G matrix, i.e., a1 and a2 • Any received codeword which is not in the plane containing a1 and a2 (i.e., an invalid codeword) will thus have a component in the direction of b1 yielding a non- zero dot product between itself and b1

44. c1 a2 c2 a1 c3 b1 Parity Check Matrix • Similarly, any received codeword which is in the plane containing a1 and a2 (i.e., a valid codeword) will not have a component in the direction of b1 yielding a zero dot product between itself and b1

45. Error Syndrome • For error correcting codes we need a method to compute the required correction • To do this we use the Error Syndrome, s of a received codeword, cr s = crHT • If cr is corrupted by the addition of an error vector, e, then cr = c + e and s = (c + e) HT = cHT + eHT s = 0 + eHT Syndrome depends only on the error

46. Error Syndrome • That is, we can add the same error pattern to different codewords and get the same syndrome. • There are 2(n - k) syndromes but 2n error patterns • For example for a (3,2) code there are 2 syndromes and 8 error patterns • Clearly no error correction possible in this case • Another example. A (7,4) code has 8 syndromes and 128 error patterns. • With 8 syndromes we can provide a different value to indicate single errors in any of the 7 bit positions as well as the zero value to indicate no errors • Now need to determine which error pattern caused the syndrome

47. Error Syndrome • For systematic linear block codes, H is constructed as follows, G = [ I | P] and so H = [-PT | I] where I is the k*k identity for G and the R*R identity for H • Example, (7,4) code, dmin= 3

48. Error Syndrome - Example • For a correct received codeword cr = [1101001] In this case,

49. Error Syndrome - Example • For the same codeword, this time with an error in the first bit position, i.e., cr = [1101000] • In this case a syndrome 001 indicates an error in bit 1 of the codeword

50. Comments about H • The minimum distance of the code is equal to the minimum number of columns (non-zero) of H which sum to zero • We can express Where do, d1, dn-1 are the column vectors of H • Clearly crHT is a linear combination of the columns of H