1 / 29

236601 - Coding and Algorithms for Memories Lecture 5

236601 - Coding and Algorithms for Memories Lecture 5. Write-Once Memories (WOM). Introduced by Rivest and Shamir , “ How to reuse a write-once memory ” , 1982

javan
Download Presentation

236601 - Coding and Algorithms for Memories Lecture 5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 236601 - Coding and Algorithms for MemoriesLecture 5

  2. Write-Once Memories (WOM) • Introduced by Rivest and Shamir, “How to reuse a write-once memory”, 1982 • The memory elements represent bits (2 levels) and are irreversibly programmed from ‘0’ to ‘1’ Q:How many cells are required to write 100 bits twice? P1: Is it possible to do better…? P2: How many cells to write kbits twice? P3: How many cells to write kbits ttimes? P3’: What is the total number of bits that is possible to write in n cells in t writes? 1st Write 2nd Write

  3. Definition: WOM Codes • Definition: An [n,t;M1,…,Mt] t-write WOM codeis a coding scheme which consists of n cells and guarantees any t writes of alphabet size M1,…,Mt by programming cells from zero to one • A WOM code consists of t encoding and decoding maps Ei, Di, 1 ≤i≤ t • E1: {1,…,M1}  {0,1}n • For 2 ≤i≤ t, Ei: {1,…,Mi}×{0,1}n {0,1}nsuch that for all (m,c)∊{1,…,Mi}×{0,1}n, Ei(m,c) ≥ c • For 1 ≤i≤ t, Di: {0,1}n {1,…,Mi}such that for Di(Ei(m,c)) =m for all (m,c)∊{1,…,Mi}×{0,1}n • The sum-rate of the WOM code is R = (Σ1t logMi)/n RivestShamir: [3,2;4,4], R = (log4+log4)/3=4/3

  4. The Capacity of WOM Codes • The Capacity Region for two writes C2-WOM={(R1,R2)|∃p∊[0,0.5],R1≤h(p), R2≤1-p} h(p) – the entropy functionh(p) = -plog(p)-(1-p)log(1-p) • p – the prob to program a cell on the 1st write, thus R1≤ h(p) • After the first write, 1-p out of the cells aren’t programmed, thus R2≤1-p • The maximum achievable sum-rate ismaxp∊[0,0.5]{h(p)+(1-p)} = log3 achieved for p=1/3: R1 = h(1/3) = log(3)-2/3 R2= 1-1/3 = 2/3

  5. The Capacity of WOM Codes • The Capacity Region for two writes C2-WOM={(R1,R2)|∃p∊[0,0.5],R1≤h(p), R2≤1-p} h(p) – the entropy functionh(p) = -plog(p)-(1-p)log(1-p) • The Capacity Region for t writes: Ct-WOM={(R1,…,Rt)| ∃p1,p2,…pt-1∊[0,0.5], R1≤ h(p1), R2 ≤ (1–p1)h(p2),…, Rt-1≤ (1–p1)(1–pt–2)h(pt–1) Rt≤ (1–p1)(1–pt–2)(1–pt–1)} • p1- probto prog. a cell on the 1st write: R1≤ h(p) • p2- probto prog. a cell on the 2nd write (from the remainder): R2≤(1-p1)h(p2) • pt-1- prob to prog. a cell on the (t-1)thwrite (from the remainder): Rt-1≤(1–p1)(1–pt–2)h(pt–1) • Rt≤ (1–p1)(1–pt–2)(1–pt–1) because(1–p1)(1–pt–2)(1–pt–1) cells weren’t programmed • The maximum achievable sum-rate is log(t+1)

  6. James Saxe’s WOM Code • [n,n/2-1; n/2,n/2-1,n/2-2,…,2] WOM Code • Partition the memory into twoparts of n/2 cells each • First write: • input symbol m∊{1,…,n/2} • program the ith cell of the 1stgroup • The ith write, i≥2: • input symbol m∊{1,…,n/2-i+1} • copy the first group to the second group • program the ith available cell in the 1stgroup • Decoding: • There is always one cell that is programmed in the 1st and not in the 2nd group • Its location, among the non-programmed cells, is the message value • Sum-rate: (log(n/2)+log(n/2-1)+ … +log2)/n=log((n/2)!)/n≈ (n/2log(n/2))/n ≈ (log n)/2

  7. The Coset Coding Scheme • Cohen, Godlewski, and Merkx ‘86 – The coset coding scheme • Use Error Correcting Codes (ECC) in order to construct WOM-codes • Let C[n,n-r] be an ECC with parity check matrix H of size r×n • Write r bits: Given a syndrome s of r bits, find a length-n vector e such that H⋅eT = s • Use ECC’s that guarantee on successive writes to find vectors that do not overlap with the previously programmed cells • The goal is to find a vector e of minimum weight such that only 0s flip to 1s

  8. The Coset Coding Scheme • C[n,n-r] is an ECC with an r×nparity check matrix H • Write r bits: Given a syndrome s of r bits, find a length-n vector e such that H⋅eT = s • Example: H is the parity check matrix of a Hamming code • s=100, v1 = 0000100: c = 0000100 • s=000, v2 = 1001000: c = 1001100 • s=111, v3 = 0100010: c = 1101110 • s=010, …  can’t write! • This matrix gives a [7,3:8,8,8] WOM code

  9. Binary Two-Write WOM-Codes • First Write: program only vectors v such that rank(Hv) = rVC = { v ∊ {0,1}n | rank(Hv) = r} • For H we get |VC| = 92 - we can write 92 messages • Assume we write v1= 0 1 0 1 1 0 0 v1= (0 1 0 1 1 00)

  10. Binary Two-Write WOM-Codes • First Write: program only vectors v such that rank(Hv) = r, VC = { v ∊ {0,1}n | rank(Hv) = r} • Second Write Encoding: • Second Write Decoding: Multiply the received word by H: H⋅(v1 + v2) = H⋅v1 + H⋅v2 = s1+ (s1 + s2) = s2 • s2 = 001 • s1 = H⋅v1 = 010 • Hv1⋅v2 = s1+s2 = 011a • v2 = 0 0 0 0 0 1 1 • v1+v2 = 0 1 0 1 1 1 1 • Write a vector s2 of r bits • Calculates1=H⋅v1 • Find v2 such that Hv1⋅v2= s1+s2a • v2 exists since rank(Hv1) = ra • Write v1+v2 to memory v1= (0 1 0 1 1 00)

  11. Sum-rate Results • The construction works for any linear code C • For any C[n,n-r] with parity check matrix H, VC = { v ∊ {0,1}n | rank(Hv) = r} • The rate of the first write is: R1(C) = (log2|VC|)/n • The rate of the second write is: R2(C) = r/n • Thus, the sum-rate is: R(C) = (log2|VC| + r)/n • In the last example: • R1= log(92)/7=6.52/7=0.93, R2=3/7=0.42, R=1.35 • Goal: Choose a code C with parity check matrix H that maximizes the sum-rate

  12. Capacity Achieving Results • The Capacity region C2-WOM={(R1,R2)|∃p∊[0,0.5],R1≤h(p), R2≤1-p} • Theorem: For any (R1, R2)∊C2-WOM and ε>0, there exists a linear code C satisfying R1(C) ≥ R1-εand R2(C) ≥ R2–ε • By computer search • Best unrestricted sum-rate 1.4928 (upper bound 1.58) • Best fixed sum-rate 1.4546 (upper bound 1.54)

  13. Capacity Region and Achievable Rates of Two-Write WOM codes

  14. Non-Binary WOM Codes • Definition: An [n,t; M1,…,Mt]q t-write WOM code is a coding scheme that consists of n q-ary cells and guarantees any t writes of alphabet size M1,…,Mt only by increasing the cell levels • The sum-rate of the WOM-code is R = (Σi=1tlogMi)/n

  15. WOM Capacity • The capacity of non-binary WOM-codes was given by Fu and Han Vinck, ‘99 • The maximal sum-rate using t writes and q-ary cells is • There is no tight upper bound on the sum-rate in case equal amount of information is written on each write

  16. Basic Constructions • Construction 1: Assume t=q–1and there are nq-ary cells • On each write, n binary bits are written • Write to the cells level by level • The sum-rate is n(q–1)/n = q-1 (upper bound ≈ 2(q-1)) • Construction 2: Assume t=2 and q is odd • First write: use only levels 0,1,…,(q-1)/2 • Second write: use only levels (q-1)/2,…,q-1 • The sum-rate is 2log((q+1)/2) • The difference between the upper bound is

  17. Construction 3 • Assume q is a power of 2, q=2m • Every cell is converted into m bits • For 1 ≤ i ≤ m, the ith bits from each cell comprise a t-write binary WOMcode • The m WOM codes write into their corresponding bits in the cells independently • Since every bit can only change from 0 to 1, the level of each cell cannot decrease • Use the binary expansion ϕ:{0,1}m→{0,…,2m-1} x=(x0,…,xm-1),ϕ(x) = ∑j 2m-j xj

  18. Example – Construction 3 • Let q = 8 = 23 and n = 3 • Every cell corresponds to 3 bits • Use Rivest-Shamir WOM-code • The ith bits from each cell comprise a WOM-code • The sum-rate is 6∙2/3 = 4 (upper bound is 5.17)

  19. Construction 3 • Theorem: If q=2m and there exists a t-write binary WOM-code of sum-rate R, then there exists a t-write q-ary WOM-code of sum-rate mR

  20. Binary Two-Write WOM-Codes • First Write: program only vectors v such that rank(Hv) = r, VC = { v ∊ {0,1}n | rank(Hv) = r} • Second Write Encoding: • Second Write Decoding: Multiply the received word by H: H⋅(v1 + v2) = H⋅v1 + H⋅v2 = s1+ (s1 + s2) = s2 • s2 = 001 • s1 = H⋅v1 = 010 • Hv1⋅v2 = s1+s2 = 011a • v2 = 0 0 0 0 0 1 1 • v1+v2 = 0 1 0 1 1 1 1 • Write a vector s2 of r bits • Calculates1=H⋅v1 • Find v2 such that Hv1⋅v2= s1+s2a • v2 exists since rank(Hv1) = ra • Write v1+v2 to memory v1= (0 1 0 1 1 00)

  21. Non-Binary Two-Write WOM-Codes • Let C[n,n-r] be a linear code w/ p.c.mH of size r×n over GF(q) • For a vector v ∊ GF(q)n, let Hv be the matrix H with 0’s in the columns that correspond to the positions of the nonzeros’s in v • First Write: program only vectors v such that rank(Hv)=r, VC = { v ∊GF(q)n | rank(Hv) = r} • Second Write Encoding: Write a vector s2 of rsymbols overGF(q) • Let v1 be the programmed vector on the first write, and let s1=H⋅v1 • Remember that rank(Hv1) = ra • Find v2 such that Hv1⋅v2= -s1+s2F • A solution v2 exists since rank(Hv1) = ra • Write v1+v2 to memory • Second Write Decoding: Multiply the received word by H: H⋅(v1 + v2) = H⋅v1 + H⋅v2 = s1+ (-s1 + s2) = s2 • This construction works for any linear code over GF(q)

  22. Code Rate • The construction works for any linear code C over GF(q) • For any C[n,n-r] with parity check matrix H,VC = { v ∊GF(q)n | rank(Hv) = r} • The rate of the first write is: R1(C) = (log2|VC|)/n • The rate of the second write is: R2(C) = r⋅log2q/n • Thus, the total rate is: R(C) = (log2|VC| + r⋅log2q)/n • By choosing the parity check matrix H uniformly at random, it is possible to achieve the following capacity region: {(R1,R2)|∃p∊[0,1-1/q], R1≤h(p)+p⋅log2(q-1), R2≤(1-p)⋅log2q} • The maximum achievable rate log2(2q-1) is achieved for p=(q-1)/(2q-1) • Remarks: • This is not an optimal non-binary two-write WOM code construction • Cells cannot be programmed twice • However, these codes are useful to construct multiple-write WOM codes

  23. Three-Write WOM Codes • Let C3 be an [n,2; 2nR1,2nR2] two-write WOM code over GF(3) • Construct a binary [2n,3; 2nR1,2nR2,2n]3-writeWOM code as follows: • The code has 2n cells, which are broken into n2-cell blocks • First write: use the first write of the code C3 • Write a message m1 from {1,…, 2nR1} • This message is supposed to be written as a ternary vector v1 ∊ GF(3)n • Write the vector v1 into the n 2-cell blocks, using the mapping Φ :GF(3)→ (GF(2),GF(2)),where Φ(0)=(0,0), Φ(1)=(1,0),Φ(2)=(0,1) • Second write: use the second write of the code C3 • Write a message m2 from {1,…, 2nR2} • The ternary vector v2 is written into the n 2-cell blocks using the mapping Φ • Each 2-cell block is written at most once and at most one cell is programmed m1∊{1,…, 2nR1} C3 Encoder v1∊GF(3)n u1∊{0,1}2n Φ m2∊{1,…, 2nR2} v2∊GF(3)n u2∊{0,1}2n C3 Encoder Φ

  24. Three-Write WOM Codes • Let C3 be an [n,2; 2nR1,2nR2] two-write WOM code over GF(3) • Construct a binary [2n,3; 2nR1,2nR2,2n]3-writeWOM code as follows: • The code has 2n cells, which are broken into n2-cell blocks • First write: use the first write of the code C3 • Write a message m1 from {1,…, 2nR1} • This message is supposed to be written as a ternary vector v1 ∊ GF(3)n • Write the vector v1 into the n 2-cell blocks, using the mapping Φ :GF(3)→ (GF(2),GF(2)),where Φ(0)=(0,0), Φ(1)=(1,0),Φ(2)=(0,1) • Second write: use the second write of the code C3 • Write a message m2 from {1,…, 2nR2} • The ternary vector v2 is written into the n 2-cell blocks using the mapping Φ • Each 2-cell block is written at most once and at most one cell is programmed • Third Write: write n bits • Each bit is stored in a 2-cell block: the bit value is 1iff both cells are programmed • It is possible to write a bit in each block since at most one cell in each block was programmed

  25. Three-Write WOM Code - Example • Ex: C3 is a 2-write WOM code over GF(3) with n=7 cells • First Write • Write a vector v1 over GF(3) according to the first write of C3 • Assume the vector is v1 = 2 0 1 0 2 1 0 • Using the mapping Φ, v1 is written into the 2n = 14 binary cells 2 0 1 0 21 0→Φ 01.00.10.00.01.10.00 • Second Write • Write a vector v2 over GF(3) according to the second write of C3 • Assume the vector is v2 = 2 1 1 0 2 1 2 • Using the mapping Φ, v2 is written into the 2n = 14 binary cells 2 1 1 0 2 1 2 →Φ01.10.10.00.01.10.01 • Third Write: Write 7 bits • Assume the message is v3 = 0 11 0 1 0 1 , then write to memory 01.11.11.00.11.10.11

  26. Code Rate • Theorem: Let C3 be an [n,2; 2nR1,2nR2]2-write WOM code over GF(3) of sum-rate R1+R2, then there exists a binary [2n,3; 2nR1,2nR2,2n]3-write WOM code of sum-rate (R1+R2+1)/2 • The best sum-rate for the code C3 is log5 = 2.32 • Conclusion: It is possible to find a binary 3-write WOM code of sum-rate (log5 +1)/2 = 1.66 • Using a computer search, the best sum-rate found for WOM codes over GF(3) is 2.22 and thus there exists a 3-write WOM code of sum-rate 1.61

  27. Four-Write WOM Codes • Let C3 be an [n,2; 2nR3,1,2nR3,2] two-write WOM code over GF(3)and letC2be an[n,2; 2nR2,1,2nR2,2]binary two-write WOM code • Construct a [2n,4; 2nR1,2nR2, 2nR2,1,2nR2,2]4-writeWOM code as follows: • The code has 2n cells, which are broken into n2-cell blocks • First write: use the first write of the code C3 • Write a message from {1,…,2nR3,1} • This message is supposed to be written as a ternary vector v ∊ GF(3)n • Write the vector v into the n 2-cell blocks, using the mapping Φ :GF(3)→ (GF(2),GF(2)),where Φ(0)=(0,0), Φ(1)=(1,0),Φ(2)=(0,1) • Second write: use the second write of the code C3 • Write a message from {1,…,2nR3,2} • The ternary vector is written into the n 2-cell blocks using the mapping Φ • Note that each 2-cell block is written only once • Third and fourth writes: use the first and second writes of C2 • Each bit is stored in a 2-cell block: the bit value is 1iff both cells are programmed • It is possible to write a bit in each block since at most one cell in each block was programmed

  28. Code Rate • Theorem: Let C3 be an [n,2; 2nR3,1,2nR3,2]2-write WOM code over GF(3) (found before) of sum-rate R1+R2, and letC2be an[n,2; 2nR2,1,2nR2,2] binary two-write WOM codethen there exists a binary [2n,4; 2nR3,1,2nR3,2,2nR2,1,2nR2,2]4-write WOM code of sum-rate (R3,1+R3,2+R2,1+R2,2)/2a • The best sum-rate for the code C3 is log5 = 2.32 and for the codeC2islog3 = 1.58 • Conclusion: It is possible to find a binary 3-write WOM code of sum-rate (log5 + log3)/2 = 1.95 • Using a computer search, the best sum-rate found for codes over GF(3) is 2.22and for codes over GF(2) is 1.49. Thus there exists a 4-write WOM code of sum-rate 1.855

  29. Five-Write WOM-Codes • Use a 2-write WOM code C3 over GF(3) of sum-rates R3,1, R3,2 and a binary 3-write WOM code C2 of sum-rates R2,1, R2,2, R2,3a • The first and second writes are implemented as before using the first and second writes of the code C3 • The third, fourth, and fifth writesare also implemented as before by using the first, second, and third writes of C2 • It is possible to achieve a five-write WOM code of rate (log5+(log5+1)/2)/2=1.99 and there exists a code of rate 1.915 • For six writes, we will change the code C2 to be a 4-write WOM code such that it is possible to achieve WOM-rate (log5+1.95)/2 = 2.14 and there exists a code of rate 2.0375

More Related