1 / 24

( Deterministic ) Communication amid Uncertainty

( Deterministic ) Communication amid Uncertainty. Madhu Sudan Microsoft, New England. Based on joint works with: (1) Adam Kalai (MSR), Sanjeev Khanna ( U.Penn ), Brendan Juba (Harvard ) a nd (2) Elad Haramaty ( Technion ). Classical Communication. The Shannon setting

lucky
Download Presentation

( Deterministic ) Communication amid Uncertainty

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. (Deterministic) Communication amid Uncertainty MadhuSudan Microsoft, New England Based on joint works with: (1) Adam Kalai(MSR), SanjeevKhanna(U.Penn), Brendan Juba (Harvard) and (2)EladHaramaty(Technion) TIFR: Deterministic Communication Amid Uncertainty

  2. Classical Communication • The Shannon setting • Alice gets chosen from distribution • Sends some compression to Bob. • Bob computes • (with knowledgeof ). • Hope . • Classical Uncertainty: • Today’s talk: Bob knows . TIFR: Deterministic Communication Amid Uncertainty

  3. Outline Part 1: Motivation Part 2: Formalism Part 3: Randomized Solution Part 4: Issues with Randomized Solution Part 5: Deterministic Issues. TIFR: Deterministic Communication Amid Uncertainty

  4. Motivation: Human Communication • Human communication (dictated by languages, grammars) very different from Shannon setting. • Grammar: Rules, often violated. • Dictionary: Often multiple meanings to a word. • Redundant: But not as in any predefined way (not an error-correcting code). • Theory? • Information theory? • Linguistics? (Universal grammars etc.)? TIFR: Deterministic Communication Amid Uncertainty

  5. Behavioral aspects of natural communication • (Vast) Implicit context. • Sender sends increasingly long messages to receiver till receiver “gets” (the meaning of) the message. • Where do the options come from? • Sender may use feedback from receiver if available; or estimates receiver’s knowledge if not. • How does estimation influence message. • Language provides sequence of (increasingly) long ways to represent a message. • How? Why? What features are good/bad. TIFR: Deterministic Communication Amid Uncertainty

  6. Model: • Reason to choose short messages: Compression. • Channel is still a scarce resource; still want to use optimally. • Reason to choose long messages (when short ones are available): Reducing ambiguity. • Sender unsure of receiver’s prior (context). • Sender wishes to ensure receiver gets the message, no matter what its prior (within reason). TIFR: Deterministic Communication Amid Uncertainty

  7. Back to Problem • Design encoding/decoding schemes () so that • Sender has distribution on • Receiver has distribution on • Sender gets • Sends to receiver. • Receiver receives • Decodes to • Want: (provided close), • While minimizing TIFR: Deterministic Communication Amid Uncertainty

  8. Contrast with some previous models • Universal compression? • Doesn’t apply: P,Q are not finitely specified. • Don’t have a sequence of samples from P; just one! • K-L divergence? • Measures inefficiency of compressing for Q if real distribution is P. • But assumes encoding/decoding according to same distribution Q. • Semantic Communication: • Uncertainty of sender/receiver; but no special goal. TIFR: Deterministic Communication Amid Uncertainty

  9. Closeness of distributions: is -close to if for all , -close to (symmetrized, “worst-case” KL-divergence) TIFR: Deterministic Communication Amid Uncertainty

  10. Dictionary = Shared Randomness? • Modelling the dictionary: What should it be? • Simplifying assumption – it is shared randomness, so … • Assume sender and receiver have some shared randomness and are independent of . • Want TIFR: Deterministic Communication Amid Uncertainty

  11. Solution (variant of Arith. Coding) • Use to define sequences • … • where chosen s.t. Either Or TIFR: Deterministic Communication Amid Uncertainty

  12. Performance • Obviously decoding always correct. • Easy exercise: • ( “binary entropy”) • Limits: • No scheme can achieve • Can reduce randomness needed. TIFR: Deterministic Communication Amid Uncertainty

  13. Implications • Reflects the tension between ambiguity resolution and compression. • Larger the ((estimated) gap in context), larger the encoding length. • Coding scheme reflects the nature of human process (extend messages till they feel unambiguous). • The “shared randomness’’ is a convenient starting point for discussion • Dictionaries do have more structure. • But have plenty of entropy too. • Still … should try to do without it. TIFR: Deterministic Communication Amid Uncertainty

  14. Deterministic Compression? • Randomness fundamental to solution. • Needs independent of to work. • Can there be a deterministic solution? • Technically: Hard to come up with single scheme that compresses consistently for all (). • Conceptually: Nicer to know “dictionary” and context can be interdependent. TIFR: Deterministic Communication Amid Uncertainty

  15. Challenging special case • Alice has permutation on • i.e., 1-1 function mapping • Bob has permutation • Know both are close: • (say ) • Alice and Bob know (say ). • Alice wishes to communicate to Bob. • Can we do this with few bits? • Say bits if . TIFR: Deterministic Communication Amid Uncertainty

  16. Model as a graph coloring problem X • Consider family of graphs • Vertices = permutations on • Edges = close permutations with distinct messages. (two potential Alices). • Central question: What is ? TIFR: Deterministic Communication Amid Uncertainty

  17. Main Results [w. EladHaramaty] • Claim: Compression length for toy problem • Thm 1: • ( times) • . • Thm 2: uncertain comm. schemes with • (-error). • -error). • Rest of the talk: Graph coloring TIFR: Deterministic Communication Amid Uncertainty

  18. Restricted Uncertainty Graphs X • Will look at • Vertices: restrictions of permutations to first coordinates. • Edges: TIFR: Deterministic Communication Amid Uncertainty

  19. Homomorphisms • homomorphic to () if • Homorphisms? • is -colorable • Homomorphisms and Uncertainty graphs. • Suffices to upper bound TIFR: Deterministic Communication Amid Uncertainty

  20. Chromatic number of For , Let Claim: is an independent set of Claim: Corollary: TIFR: Deterministic Communication Amid Uncertainty

  21. Better upper bounds: Say Lemma: For TIFR: Deterministic Communication Amid Uncertainty

  22. Better upper bounds: TIFR: Deterministic Communication Amid Uncertainty • Lemma: • For • Corollary: • Aside: Can show: • Implies can’t expect simple derandomization of the randomized compression scheme.

  23. Future work? • Open Questions: • Is ? • Can we compress arbitrary distributions to ? or even • On conceptual side: • Better mathematical understanding of forces on language. • Information-theoretic • Computational • Evolutionary TIFR: Deterministic Communication Amid Uncertainty

  24. Thank You TIFR: Deterministic Communication Amid Uncertainty

More Related