Design of Optimal Short-Length LT Codes Using Evolution Strategies

1. Design of Optimal Short-Length LT Codes Using Evolution Strategies John K. Zao*, Martin Hornansky, Pei-lunDiao WCCI 2012 IEEE World Congress on Computational Intelligence June, 2012 - Brisbane, Australia

2. Outline • Introduction • LT codes • Performance models • Optimization problem • Optimization methods • Optimization scheme • Experiments and results

3. Introduction • LT codes : • With large symbol blocks ( ) : The asymptotic behaviors have been deduced analytically • With short symbol blocks ( ) : A proficient method for finding the optimal degree distributions is still absent [8] E. Hyytia, T. Tirronen, and J. Virtamo, “Optimal degree distribution for LT codes with small message length,¨in the 26th IEEE INFOCOM 2007, pp.2576-2580. [9] E. A. Bodine and M. K. Cheng, “Characterization of Lubytransform codes with small message Size for low-latency decoding,¨IEEE International Conference in Communications, 2008.

4. In this paper • To employ evolution strategies in designing optimal SL-LT codes with decoding performance that suit different applications. • A new performance model : • Coding overhead ε • Failure ratio r • Failure occurrence probability p

5. How to ensure proper use of the evolution strategies • The selection of evolution strategies • The choice of decision variables • The specification of fitness functions • The choice of initial population • The criteria for selecting population samples in every generation

6. LT codes • A. Encoding and Decoding Operations • B. Degree Distribution • C. Code Performance • D. Code Applications

7. LT codes • A. Encoding and Decoding Operations • B. Degree Distribution Ideal solitondistribution Robust soliton distribution

8. LT codes • C. Code Performance • There is a high chance that up to 70% of source symbols may not be recovered if only a small number of codewords were used for decoding. • Reducing the maximum failure rate or the probability of high failure instances.

9. LT codes • D. Code Applications • Erasure protection for lossless data transfer • File downloads • Perfect data reception among the receivers • As few symbols as possible • Data transfer with limited overhead allowance • Video streaming • Can tolerate small amount of decoding failures • Can not tolerate large increase in bandwidth or latency • Postcodingin rateless composite codes • Adding precoding • Only need to bring the decoding failure rates below certain threshold

10. Performance models • Overhead • The ratio between the number of extra encoded symbols received and the number of source symbols. • Failure Rate • The fraction of the unrecovered source symbols during a decoding process • Failure probability • The probability of the decoding failure rate to be higher than a threshold value r while the code is decoded with an overhead ε .

11. LT codes • D. Code Applications • Erasure protection for lossless data transfer minimize ε • File downloads • Perfect data reception among the receivers • As few symbols as possible • Data transfer with limited overhead allowance reduce r • Video streaming • Can tolerate small amount of decoding failures • Can not tolerate large increase in bandwidth or latency • Postcodingin rateless composite codes minimize p • Adding precoding • Only need to bring the decoding failure rates below certain threshold

12. Optimal problem • A. Design Variables • B . Fitness Function Evaluation • C. Optimization Scenarios

13. Optimal problem :A. Design Variables • We created two M-tuples : • Degree tuple • Probability tuple • Each captures a non-trivial entry of the probability mass function: • Degree elements need to be rounded to the closest integers . • Several degree elements may arrive at the same value . • Some probability elements may become insignificant in the final results.

14. Optimal problem • B . Fitness Function Evaluation • Evaluating the performance of different SL-LT code samples in each generation by means of numerical simulation of the actual decoding process. • Fitness function f(x) • Fitness value • C. Optimization Scenarios • Fixed Degree Scenario : The components of the degree tuple are kept constant. • Variable Degree Scenario: The design variables consist of the components of both the degree and the probability tuples.

15. Optimization methods • Covariance Matrix Adaptation Evolution Strategy • Iteratively updating the covariance matrix of a multivariate normal distribution of mutated population. • Especially successful in solving badly conditioned, multimodal and noisy problemswith high-dimensional rugged search landscape.

16. Optimization methods • Natural Evolution Strategy • A new numerical optimization method • Performing gradient ascent along the natural gradient in the population parameter space • Preventing oscillations, premature convergence, and other undesired effects • Differential Evolution • An evolution strategy • Creating new candidate solutions by combining existing ones according to simple formulae based on vector differences. • Good on noisy constrained optimization problems with multidimensional real-valued functions and problems that change over time.

17. Optimization scheme : Decision Variables • Transformations between Decision and Design Variables: • The design variables d and p , have bounded value ranges • We defined two decision variabletuples : • How to transform ? • CMA-ES and NES are stochastic strategies that use Gaussian distributions to produce random off-springs over an unbounded variable space • DE is a population-based evolution strategy that can evolve its off-springs in a bounded variable space.

18. Optimization scheme : Decision Variables • Probability Transformation for CMA-ES and NES: • Probability Normalization for DE: • Degree Rounding and Constraining:

19. Optimization scheme : Decision Variables • CMA-ES and NES can generate unbounded values for their variables affects both, degrees and probabilities. • Adding monotonically increasing penalty :

20. Optimization scheme : Initial Degree Distribution • Choices of Initial Degrees: • We chose their initial degrees from the range of [1, K/5] • Either the prime numbers or the powers of two

21. Optimization scheme : Specification of Initial Probabilities • Sparse Robust Soliton Distribution: • Gathering probability values under the adjacent degrees of the robust soliton distribution to those under the selected initial degrees . • is the probability mass function of robust soliton distribution

22. Optimization scheme : Specification of Initial Probabilities • Sampled Ideal Soliton Distribution: • The last component then absorbed the remaining probability [20]: • Uniform Distribution: • Every component of the probability tuple was assigned the same value. [20] G. G. Yan, H. C. Chang, “Research on separable UEP-LT code”, M.S. thesis, Dept. of Electron. Eng., National Chiao Tung University, Hsinchu, Taiwan, 2007, pp. 45-46.

23. Experiments and results • Specific Values of Performance Measurements • We tried to find optimal SL-LT codes based on • We fixed the values of two performance measurements while optimizing the third one.

24. Experiments and results • Parameters of Optimization Methods • CMA-ES and NES Cases: • Users only need to specify the initial decision variable values and their standard deviations. • For the standard deviations: • [10, 30] for decision variables representing degrees • [0.02, 0.2] for decision variables representing probabilities • DE Parameters: • The optimization problem is continuous, noisy, and exhibit respective problem dimensionality. • We assigned • 0.5 for crossover probability • 0.7 for scaling factor [19] R. Storn, K. Price, "Differential evolution -- a simple and efficient heuristic for global optimization over continuous spaces". Journal of Global Optimization 11, 1997, pp. 341-359. [20] G. G. Yan, H. C. Chang, “Research on separable UEP-LT code”, M.S. thesis, Dept. of Electron. Eng., National Chiao Tung University, Hsinchu, Taiwan, 2007, pp. 45-46. [21] M. E. H. Pedersen, “Good parameters for differential evolution”, Technical Report no. HL1002, Hvass Laboratories, 2010.

25. Experiments and results • Comparison of Convergence Behaviors : • We traced the fitness function values and average degrees of every generation throughout the evolution process.

30. Experiments and results • Consistency among Optimized Performance : ε γ p

31. Experiments and results : Observations • Evolution strategies were shown to be a practical method for designing optimal SL-LT codes. • All three strategies are managed to converge and produce degree distributions. • CMA-ES and NES showed similar convergence • NES appeared to be the most robust evolutionary strategy . • NES > CMA-ES > DE • The optimization scenarios with fixed and variable degrees produce similar performance measurement values.