1 / 13

ECE 101 An Introduction to Information Technology Information Theory

ECE 101 An Introduction to Information Technology Information Theory. Information Path. Source of Information. Digital Sensor. Information Display. Information Receiver and Processor. Information Processor & Transmitter. Transmission Medium. Information Theory.

zenaida-stephenson
Download Presentation

ECE 101 An Introduction to Information Technology Information Theory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ECE 101An Introduction to Information TechnologyInformation Theory

  2. Information Path Source of Information Digital Sensor Information Display Information Receiver and Processor Information Processor & Transmitter Transmission Medium

  3. Information Theory • Source generates information by producing data units called symbols • Measurement of information present • measure randomness (value of information) • do this mathematically using probability • amount of information present is measure of “entropy”

  4. Probability • Study of random outcomes • The experiment • The outcome • P[Xi] = probability of an a particular outcome (Xi) • 0 < P[Xi] < 1 • where N= number of different outcomes

  5. Measuring Information • Symbol - data units of information • Entropy • average amount of energy that a source produces, measured in bits/symbol

  6. Logarithms – Base 2 • In information theory we need logs to the base 2, not 10 (log10 N = x or 10x = N) (logs are exponents) • log2 N = x or 2x = N • 20 = 1; log2 1 = 0 • 21 = 2; log2 2 = 1 • 22 = 4; log2 4 = 2 • 23 = 8; log2 8 = 3 • 24 = 16; log2 16 = 4 • 25 = 32; log2 32 = 5

  7. Logarithms – Base “a” then a=2 • Conversion of bases in general: • loga N = x or ax = N • So log2 N = x or 2x = N • loga N = (log10 N)/ (log10 a) • If a = 2, then use log10 2 = .301 • log2 N = 3.32 (log10 N) • loga MN = (loga M) + (loga N) • loga M/N = (loga M) - (loga N) • loga Nm = m(loga N)

  8. Measuring Information • Symbol - data units of information • Entropy • average amount of energy that a source produces, measured in bits/symbol

  9. Effective Probability and Entropy • Measurement of entropy when probability is not known • estimate probability when it is not known • effective probability = Pe[Xi] = NXi/N

  10. Simulating Randomness by Computer • Information is an unexpected quality • Model it an an experiment that produces random outcomes • Common method: pseudo-random number generator (PRNG) • PRNG uses Modular Arithmetic

  11. Modular Arithmetic • [B]mod(N) = modulo-N value of integer B • Divide B by N: B/N = I + R/N • where I is integer quotient and R is remainder • 0  R  (N-1) • [B]mod(N) = R = B - (I  N) • or R = (B/N - I)  N, where B/N = I.xxx

  12. Pseudo-Random Number Generator • Create a random number from a sequence X1, X 2, X3 , … , Xn, … where Xn is the nth integer in the sequence • Find Xn = [A  Xn-1 + B]mod(N) where • A is an arbitrary multiplier of Xn-1 • N is the base of the modulus • B prevents the sequence from degenerating into a set of zeroes • to get started we need an arbitrary X0, or seed

  13. Arbitrary Range for Pseudo-Random Numbers • Desire range other than an integer number then

More Related