1 / 30

Communication amid Uncertainty

Communication amid Uncertainty. Madhu Sudan Microsoft, Cambridge, USA. Based on:. -Universal Semantic Communication – Juba & S. (STOC 2008) -Goal-Oriented Communication – Goldreich , Juba & S. (JACM 2012) -Compression without a common prior … – Kalai, Khanna, Juba & S. (ICS 2011)

rodney
Download Presentation

Communication amid Uncertainty

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Communication amid Uncertainty MadhuSudan Microsoft, Cambridge, USA Based on: -Universal Semantic Communication – Juba & S. (STOC 2008) -Goal-Oriented Communication – Goldreich, Juba & S. (JACM 2012) -Compression without a common prior … – Kalai, Khanna, Juba & S. (ICS 2011) -Efficient Semantic Communication with Compatible Beliefs – Juba & S. (ICS 2011) -Deterministic Compression with uncertain priors – Haramaty & S. (ITCS 2014) Cornell: Uncertainty in Communication

  2. Classical theory of communication Shannon (1948) Alice Bob Encoder Decoder Cornell: Uncertainty in Communication • Clean architecture for reliable communication. • Remarkable mathematical discoveries: Prob. Method, Entropy, (Mutual) Information • Needs reliable encoder + decoder (two reliable computers).

  3. Uncertainty in Communication? Cornell: Uncertainty in Communication • Always has been a central problem: • But usually focusses on uncertainty introduced by the channel • Standard Solution: • Use error-correcting codes • Significantly: • Design Encoder/Decoder jointly • Deploy Encoder at Sender, Decoder at Receiver

  4. New Era, New Challenges: Cornell: Uncertainty in Communication • Interacting entities not jointly designed. • Can’t design encoder+decoder jointly. • Can they be build independently? • Can we have a theory about such? • Where we prove that they will work? • Hopefully: • YES • And the world of practice will adopt principles.

  5. Example 1 Cornell: Uncertainty in Communication • Intersystem communication? • Google+ Facebook friendship ? • Skype Facetime chat? • Problem: • When designing one system, it is uncertain what the other’s design is (or will be in the future)!

  6. Example 2 Cornell: Uncertainty in Communication • Heterogenous data? • Amazon-marketplace spends N programmer hours converting data from mom-n-pop store catalogs to uniform searchable format. • Healthcare analysts spend enormous #hours unifying data from multiple sources. • Problem: Interface of software with data: • Challenge: • Software designer uncertain of data format. • Data designer uncertain of software.

  7. Example 3 Cornell: Uncertainty in Communication • Archiving data • Physical libraries have survived for 100s of years. • Digital books have survived for five years. • Can we be sure they will survive for the next five hundred? • Problem: Uncertainty of the future. • What systems will prevail? • Why aren’t software systems ever constant? • Problem: • When designing one system, it is uncertain what the other’s design is (or will be in the future)!

  8. Modelling uncertainty Semantic Communication Model A1 B1 B2 A2 Channel A B B3 A3 New Class of Problems New challenges Needs more attention! Bj Ak Cornell: Uncertainty in Communication Classical Shannon Model

  9. Nature of uncertainty Cornell: Uncertainty in Communication • differ in beliefs, but can be centrally programmed/designed. • [Juba,Kalai,Khanna,S.’11] : Compression in this context has graceful degradation as beliefs diverge. • [Haramaty,S’13]: Role of randomness in this context. • differ in behavior: • Nothing to design any more (behavior already fixed). • Best hope: Can identify certain ’s (universalists) that can interact successfully with many ’s. Can eliminate certain ’s on the grounds of “limited tolerance”. • [Juba,S’08; Goldreich,J,S’12; J,S’11]: “All is not lost, if we keep goal of communication in mind” • [Leshno,S’13]: “Communication is a Coordination Game” • Details don’t fit in margin …

  10. II: Compression under uncertain beliefs/priors Cornell: Uncertainty in Communication

  11. Motivation Cornell: Uncertainty in Communication • New era of challenges needs new solutions. • Most old solutions do not cope well with uncertainty. • The one exception? • Natural communication (Humans Humans) • What are the rules for human communication? • “Grammar/Language” • What kind of needs are they serving? • What kind of results are they getting? (out of scope) • If we were to design systems serving such needs, what performance could they achieve?

  12. Role of Dictionary (/Grammar/Language) … Cornell: Uncertainty in Communication • Dictionary: maps words to meaning • Multiple words with same meaning • Multiple meanings to same word • How to decide what word to use (encoding)? • How to decide what a word means (decoding)? • Common answer: Context • Really Dictionary specifies: • Encoding: context meaning word • Decoding: context word meaning • Context implicit; encoding/decoding works even if context used not identical!

  13. Context? Cornell: Uncertainty in Communication • In general complex notion … • What does sender know/believe • What does receiver know/believe • Modifies as conversation progresses. • Our abstraction: • Context = Probability distribution on potential “meanings”. • Certainly part of what the context provides; and sufficient abstraction to highlight the problem.

  14. The problem Cornell: Uncertainty in Communication • Wish to design encoding/decoding schemes (E/D) to be used as follows: • Sender has distribution on • Receiver has distribution on • Sender gets • Sends to receiver. • Receiver receives • Decodes to ) • Want: (provided close), • While minimizing

  15. Closeness of distributions: Cornell: Uncertainty in Communication is -close to if for all , -close to .

  16. Dictionary = Shared Randomness? Cornell: Uncertainty in Communication • Modelling the dictionary: What should it be? • Simplifying assumption – it is shared randomness, so … • Assume sender and receiver have some shared randomness and independent of . • Want

  17. Solution (variant of Arith. Coding) Cornell: Uncertainty in Communication • Use R to define sequences • … • where chosen s.t. Either Or

  18. Performance Cornell: Uncertainty in Communication • Obviously decoding always correct. • Easy exercise: • Limits: • No scheme can achieve • Can reduce randomness needed.

  19. Implications Cornell: Uncertainty in Communication • Reflects the tension between ambiguity resolution and compression. • Larger the ((estimated) gap in context), larger the encoding length. • Entropy is still a valid measure! • Coding scheme reflects the nature of human process (extend messages till they feel unambiguous). • The “shared randomness’’ assumption • A convenient starting point for discussion • But is dictionary independent of context? • This is problematic.

  20. III: Deterministic Communication Amid Uncertainty Cornell: Uncertainty in Communication

  21. A challenging special case Cornell: Uncertainty in Communication • Say Alice and Bob have rankings of N players. • Rankings = bijections • = rank of ith player in Alice’s ranking. • Further suppose they know rankings are close. • Bob wants to know: Is • How many bits does Alice need to send (non-interactively). • With shared randomness – • Deterministically?

  22. Model as a graph coloring problem X Cornell: Uncertainty in Communication • Consider family of graphs • Vertices = permutations on • Edges = -close permutations with distinct messages. (two potential Alices). • Central question: What is ?

  23. Main Results [w. EladHaramaty] Cornell: Uncertainty in Communication • Claim: Compression length for toy problem • Thm 1: • ( times) • . • Thm 2: uncertain comm. schemes with (-error). • -error). • Rest of the talk: Graph coloring

  24. Restricted Uncertainty Graphs X Cornell: Uncertainty in Communication • Will look at • Vertices: restrictions of permutations to first coordinates. • Edges:

  25. Homomorphisms Cornell: Uncertainty in Communication • homomorphic to () if • Homorphisms? • is -colorable • Homomorphisms and Uncertainty graphs. • Suffices to upper bound

  26. Chromatic number of Cornell: Uncertainty in Communication For , Let Claim: is an independent set of Claim: Corollary:

  27. Better upper bounds: Say Lemma: For Cornell: Uncertainty in Communication

  28. Better upper bounds: Cornell: Uncertainty in Communication • Lemma: • For • Corollary: • Aside: Can show: • Implies can’t expect simple derandomization of the randomized compression scheme.

  29. Future work? Cornell: Uncertainty in Communication • Open Questions: • Is ? • Can we compress arbitrary distributions to ? or even • On conceptual side: • Better understanding of forces on language. • Information-theoretic • Computational • Evolutionary • Game-theoretic • Design better communication solutions!

  30. Thank You Cornell: Uncertainty in Communication

More Related