1 / 19

Information Theory

Information Theory. John A. Cagle, Ph.D. Communication California State University, Fresno. Claude Shannon. Shannon & Weaver (1947). Norbert Weiner’s Cybernetics added the notion of feedback to this communication model. Information Theory.

kirti
Download Presentation

Information Theory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Information Theory John A. Cagle, Ph.D. Communication California State University, Fresno Claude Shannon

  2. Shannon & Weaver (1947) Norbert Weiner’s Cybernetics added the notion of feedback to this communication model.

  3. Information Theory • In a perfect communication system, the sender and receiver have identical knowledge of the code. • All possible messages are known in advance. • The source makes a choice to send a message from the set of possible messages. • The receiver needs to know what choice the sender made.

  4. Information is not content • A message has information if it reduces the uncertainty about what choice the sender made. • If the choice is already known to the receiver, the message is redundant. • Information is not content in information theory.

  5. Information is not meaning • Information is different from the content and meaning of messages. • Information is not the interpretation of information. • Peter Drucker wrote of the difference between informating and communicating in an organization.

  6. Entropy • Entropy is randomness, chaos, the lack of organization and predictability. • Entropy is uncertainty. • Information reduces entropy in a communication system. • Entropy is variable in most situations.

  7. Measurement of Information • The smallest unit of information is a bit • Eight bits = one byte • Four bytes = one word • These terms are still at the core of computer science • E.g., 32-bit word processors in the CPU

  8. Measurement of Information • I = - log2 piis the formula for measuring the information value of each message sent against the probability of that message in the field of all the messages that could be sent.

  9. Measurement of Information • H = - ∑ pi log2 piis the formula for measuring the amount of information of all the messages that could be sent in a communication system.

  10. Choices • Signal: we make choices about which signal to send (sounds, letters, etc.) • Semantics: we make choices in a given situation about which meaning to send. • Lexical choice • Meaning • Pragmatics: we make choices in a given situation about which behaviors to enact.

  11. Communication behavior is a specific form of molar behavior _____1_____ occurs in a situation or field possessing specified properties, ____2_____ part of which are in interdependent relationship with each _____3_____. A theory of such behavior is concerned with forces, _____4_____, social, and physical, which determine the course of this _____5_____ and its outcomes in relation to the culture in _____6_____ it occurs.

  12. Information theory analysis

  13. Ring a-round the roses, A pocket full of posies, _____1_____! Ashes! We all fall down! Three blind mice, See how they run! They all _____2_____ after a farmer's wife, Who cut off their tails _____3_____ a carving knife. Did you ever see such a ____4______ in your life, As three blind mice?

  14. Brian está en el aeropuerto de Barajas en Madrid. _____1_____ y otros estudiantes del drupo esperan la llegada del _____2_____ para ir a Leób. Deben esperar una hora. ¿Qué ____3______ hacer? 1 Alice 2 vuelo 3 deciden

  15. Samuel Becker (1968) We construct messages which "are, in effect, overlayed to form the large and complex communication environment or 'mosaic' in which each of us exists. This mosaic consists of an immense number of fragments or bits of information on an immense number of topics. . . . These bits are scattered over time and space and modes of communication. Each individual must grasp from this mosaic those bits which serve his needs, must group them into message sets which are relevant for him at any given time, and within each message set must organize the bits and close the gaps between them in order to arrive at a coherent picture of the world to which he can respond."

More Related