Communication Under Normed Uncertainties

1. Robust Capacity of White Gaussian Noise with Uncertainty Communication Under Normed Uncertainties S. Z. Denic School of Information Technology and Engineering University of Ottawa, Ottawa, Canada C. D. Charalambous Department of Electrical and Computer Engineering University of Cyprus, Nicosia, Cyprus S. M. Djouadi Department of Electrical and Computer Engineering University of Tennessee, Knoxville, USA Dec 9, 2004

2. Overview • Importance of Uncertainty in Communications • Shannon’s Definition of Capacity • Review of Maximin Capacity • Paper Contributions

3. Overview • Main Results • Examples • Conclusion and Future Work

4. Importance of Communication Subject to Uncertainties • Channel measurement errors • Network operating conditions • Channel modeling • Communication in presence of jamming • Sensor networks • Teleoperations

5. Shannon’s Definition of Capacity

6. Shannon’s Definition of Capacity Source Decoder Sink + • Model of communication system Encoder Channel

7. Shannon’s Definition of Capacity • Discrete memoryless channel • Channel capacity depends on channel transition matrix Q(y|x) that is known

8. Shannon’s Definition of Capacity • What if Q(y|x) is unknown ? • Example: compound BSC • What is the channel capacity ? 1-  0 0   1 1 1- 

9. Shannon’s Definition of Capacity • Additive Gaussian Channels • Random Process Case n y x +

10. Shannon’s Definition of Capacity • Random process case derivation

11. Shannon’s Definition of Capacity • Capacity of continuous time additive Gaussian channel

12. Shannon’s Definition of Capacity • Water-filling

13. Review of Maximin Capacity

14. Review of Minimax Capacity • Example: compound DMC • This result is due to Blackwell et. al. [6]. Also look at Csiszar [8], and Wolfowitz [21] • Blachman [5], and Dobrushin [12] were first to apply game theoretic approach in computing the channel capacity with mutual information as a pay-off function for discrete channels

15. Review of Minimax Capacity • The existence of saddle point ? • For further references see Lapidoth, Narayan [18]

16. Paper Contributions

17. Paper Contributions • Modeling of uncertainties in the normed linear spaces H∞, and L1 • Explicit channel capacity formulas for SISO communication channels that depend on the sizes of uncertainty sets for uncertain channel, uncertain noise, and uncertain channel, and noise • Explicit water-filling formulas that describe optimal transmitted powers for all derived channel capacities formulas depending on the size of uncertainty sets

18. Main Results

19. Communication system model • Model n y x +

20. Communication system model + + • Uncertainty models: additive and multiplicative

21. Communication system model • Example Im / Re /(1+) /(1-)

22. Communication system model • The uncertainty set is described by the ball in frequency domain centered at and with radius of

23. Channel capacity with uncertainty • Define four sets

24. Channel capacity with uncertainty • Overall PSD of noise is and uncertainty is modeled by uncertainty of filter or by the set A4

25. Channel capacity with uncertainty I • Three problems could be defined • Noise uncertainty • Channel uncertainty

26. Channel capacity with uncertainty I • Channel – noise uncertainty

27. Channel capacity with uncertainty I • Channel capacity is given parametrically

28. Channel capacity with uncertainty I • Maximization gives water – filling equation

29. Channel capacity with uncertainty I • Water – filling

30. Channel capacity with uncertainty II • Jamming • Noise uncertainty • Channel – noise uncertainty

31. Channel capacity with uncertainty II • The lower value C- of pay-off function is defined as and is given by Theorem 2. The upper value C+ is defined by

32. Channel capacity with uncertainty II • Channel capacity is given as where are Lagrange multipliers

33. Channel coding theorem • Define the frequency response of equivalent channel with impulse response and ten sets

34. Channel coding theorem • Positive number Ri is called attainable rate for the set of channels Ki if there exists a sequence of codes such that when then uniformly over set Ki. • Theorem 1. The operational capacities Ci (supremum of all attainable rates Ri) for the sets of communication channels with the uncertainties Ki are given by corresponding computed capacity formulas. • Proof. Follows from [15], and [20] (see [11])

35. Example 1 • Uncertain channel, white noise • Transfer function

36. Example 1 • Channel capacity

37. Example 1 P = 10-2 W N0 = 10-8 W/Hz  = 1000 rad/s 

38. Example 2 • Uncertain noise • Transfer function • Noise uncertainty description

39. Example 2

40. Example 2

41. Example 3 • Uncertain channel, uncertain noise • Damping ration is uncertain • The noise uncertainty is modelled as in the Example 2

42. Example 3

43. Example 3

44. Future work • Currently generalizing these results to uncertain MIMO channels

45. References [1] Ahlswede, R., “The capacity of a channel with arbitrary varying Gaussian channel probability functions”, Trans. 6th Prague Conf. Information Theory, Statistical Decision Functions, and Random Processes, pp. 13-31, Sept. 1971. [2] Baker, C. R., Chao, I.-F., “Information capacity of channels with partially unknown noise. I. Finite dimensional channels”, SIAM J. Appl. Math., vol. 56, no. 3, pp. 946-963, June 1996.

46. References [4] Biglieri, E., Proakis, J., Shamai, S., “Fading channels: information-theoretic and communications aspects,” IEEE Transactions on Information Theory, vol. 44, no. 6, pp. 2619-2692, October, 1998. [5] Blachman, N. M., “Communication as a game”, IRE Wescon 1957 Conference Record, vol. 2, pp. 61-66, 1957. [6] Blackwell, D., Breiman, L., Thomasian, A. J., “The capacity of a class of channels”, Ann. Math. Stat., vol. 30, pp. 1229-1241, 1959. [7] Charalambous, C. D., Denic, S. Z., Djouadi, S. M. "Robust Capacity of White Gaussian Noise Channels with Uncertainty", accepted for 43th IEEE Conference on Decision and Control.

47. References [8] Csiszar, I., Korner, J., Information theory: Coding theorems for discrete memoryless systems. New York: Academic Press, 1981. [9] Csiszar, I., Narayan P., “Capacity of the Gaussian arbitrary varying channels”, IEEE Transactions on Information Theory, vol. 37, no. 1, pp. 18-26, Jan., 1991. [10] Denic, S. Z., Charalambous, C. D., Djouadi, S.M., “Capacity of Gaussian channels with noise uncertainty”, Proceedings of IEEE CCECE 2004, Canada. [11] Denic, S.Z., Charalambous, C.D., Djouadi, S.M., “Robust capacity for additive colored Gaussian uncertain channels,” preprint.

48. References [12] Dobrushin, L. “Optimal information transmission through a channel with unknown parameters”, Radiotekhnika i Electronika, vol. 4, pp. 1951-1956, 1959. [13] Doyle, J.C., Francis, B.A., Tannenbaum, A.R., Feedback control theory, New York: McMillan Publishing Company, 1992. [14] Forys, L.J., Varaiya, P.P., “The -capacity of classes of unknown channels,” Information and control, vol. 44, pp. 376-406, 1969. [15] Gallager, G.R., Information theory and reliable communication. New York: Wiley, 1968.

49. References [16] Hughes, B., Narayan P., “Gaussian arbitrary varying channels”, IEEE Transactions on Information Theory, vol. 33, no. 2, pp. 267-284, Mar., 1987. [17] Hughes, B., Narayan P., “The capacity of vector Gaussian arbitrary varying channel”, IEEE Transactions on Information Theory, vol. 34, no. 5, pp. 995-1003, Sep., 1988.  [18] Lapidoth, A., Narayan, P., “Reliable communication under channel uncertainty,” IEEE Transactions on Information Theory, vol. 44, no. 6, pp. 2148-2177, October, 1998. [19] Medard, M., “Channel uncertainty in communications,” IEEE Information Theory Society Newsletters, vol. 53, no. 2, p. 1, pp. 10-12, June, 2003.

50. References [20] Root, W.L., Varaiya, P.P., “Capacity of classes of Gaussian channels,” SIAM J. Appl. Math., vol. 16, no. 6, pp. 1350-1353, November, 1968. [21] Wolfowitz, Coding Theorems of Information Theory, Springer – Verlang, Belin Heildelberg, 1978. [22] McElice, R. J., “Communications in the presence of jamming – An information theoretic approach, in Secure Digital Communications, G. Longo, ed., Springer-Verlang, New York, 1983, pp. 127-166. [23] Diggavi, S. N., Cover, T. M., “The worst additive noise under a covariance constraint”, IEEE Transactions on Information Theory, vol. 47, no. 7, pp. 3072-3081, November, 2001.