1 / 48

Information Complexity Lower Bounds

Information Complexity Lower Bounds. Rotem Oshman, Princeton CCI Based on: Bar-Yossef,Jayram,Kumar,Srinivasan’04 Braverman,Barak,Chen,Rao’10. Communication Complexity. = ?. Yao ‘79, “Some complexity questions related to distributive computing ”. Communication Complexity.

jael
Download Presentation

Information Complexity Lower Bounds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Information Complexity Lower Bounds Rotem Oshman, Princeton CCI Based on: Bar-Yossef,Jayram,Kumar,Srinivasan’04 Braverman,Barak,Chen,Rao’10

  2. Communication Complexity = ? Yao ‘79, “Some complexity questions related to distributive computing”

  3. Communication Complexity • Applications: • Circuit complexity • Streaming algorithms • Data structures • Distributed computing • Property testing • …

  4. Deterministic Protocols • A protocol specifies, at each point: • Which player speaks next • What should the player say • When to halt and what to output • Formally, what we’ve said so far who speaks next: Alice, Bob, = halt what to say/output

  5. Randomized Protocols • Can use randomness to decide what to say • Private randomness: each player has a separate source of random bits • Public randomness: both players can use the same random bits • Goal: for any compute correctly with probability • Communication complexity: worst-case length of transcript in any execution

  6. Randomness Can Help a Lot • Example: Equality • Input: • Output: is ? • Trivial protocol: Alice sends to Bob • For deterministic protocols, this is optimal!

  7. Equality Lower Bound #rectangles

  8. Randomized Protocol • Protocol with public randomness: • Select random • Alice sends • Bob accepts iff • If : always accept • If : • Reject with probability non-zero vector

  9. Set Disjointness • Input: • Output: ? • Theorem [Kalyanasundaran, Schnitger ‘92, Razborov ‘92]: randomized CC = • Easy to see for deterministic protocols • Today we’ll see a proof by Bar-Yossef, Jayram, Kumar, Srinivasan ‘04

  10. Application: Streaming Lower Bounds • Streaming algorithm: • Example: how many distinct items in the data? • Reduction from Disjointness [Alon, Matias, Szegedy ’99] How much space is required to approximate f(data)? algorithm data

  11. Reduction from Disjointness: • Fix a streaming algorithm for Distinct Elements with space , universe size • Construct a protocol for Disj.with elements: algorithm ⇔ #distinct elements in is State of the algorithm and (#bits = )

  12. Application 2: KW Games • Circuit depth lower bounds: • How deep does the circuit need to be?

  13. Application 2: KW Games • Karchmer-Wigderson’93,Karchmer-Raz-Wigderson’94: find such that

  14. Application 2: KW Games • Claim: if has deterministic CC , then requires circuit depth . • Circuit with depth protocol with length

  15. Information-Theoretic Lower Bound on Set Disjointness

  16. Some Basic Concepts from Info Theory • Entropy of a random variable: • Important properties: • is deterministic • = expected # bits needed to encode

  17. Some Basic Concepts from Info Theory • Conditional entropy: • Important properties: • are independent • Example: • If then , if 1then

  18. Some Basic Concepts from Info Theory • Mutual information: • Conditional mutual information: • Important properties: • are independent

  19. Some Basic Concepts from Info Theory • Chain rule for mutual information: • More generally,

  20. Information Cost of Protocols • Fix an input distribution on • Given a protocol , let also denote the distribution of ’s transcript • Information cost of : • Information cost of a function :

  21. Information Cost of Protocols • Important property: • Proof: by induction. Let . • : what we know after r rounds what we knew after r-1 rounds what we learn in round r, given what we already know

  22. Information vs. Communication • Want: • Suppose is sent by Alice. • What does Alice learn? • is a function of and so • What does Bob learn?

  23. Information vs. Communication • Important property: • Lower bound on information cost ⇒ lower bound on communication complexity • In fact, IC lower bounds are the most powerful technique we know

  24. Information Complexity of Disj. • Disjointness: is ? • Strategy: for some “hard distribution” , • Direct sum: • Prove that .

  25. Hard Distribution for Disjointness • For each coordinate :

  26. Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set • Sample and run • For each ,

  27. Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set • Bad idea: publicly sample Suppose in , Alice sends . In , Bob learns one bit in he should learn bit But if is public Bob learns 1 bit about !

  28. Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set • Another bad idea: publicly sample , Bob privately samples given • But the players can’t sample , independently…

  29. Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set Publicly sample Privately sample Privately sample Publicly sample

  30. Direct Sum Theorem • Transcript of • Need to show:

  31. Information Complexity of Disj. • Disjointness: is ? • Strategy: for some “hard distribution” , • Direct sum: • Prove that . 

  32. Hardness of And transcript on should be “very different”

  33. Hellinger Distance • Examples: • If have disjoint support,

  34. Hellinger Distance • Hellinger distance is a metric • , with equality iff • Triangle inequality:

  35. Hellinger Distance • If for some we have then

  36. Hellinger Distance vs. Mutual Info • Let be two distributions • Select by choosing , then drawing • Then

  37. Hardness of And Same for Bob until Alice acts differently Same for Alice until Bob acts differently

  38. “Cut-n-Paste Lemma” • Recall: • Enough to show: we can write

  39. “Cut-n-Paste Lemma” • We can write • Proof: • induces a distribution on “partial transcripts” of each length : probability that first bits are • By induction: • Base case: • Set

  40. “Cut-n-Paste Lemma” • Step: • Suppose after it is Alice’s turn to speak • What Alice says depends on: • Her input • Her private randomness • The transcript so far, • So • Set

  41. Hardness of And

  42. Multi-Player Communication Complexity

  43. The Coordinator Model sites • bits …

  44. Multi-Party Set Disjointness • Input: • Output: is ? • Braverman,Ellen,O.,Pitassi,Vaikuntanathan’13: lower bound of bits

  45. Reduction from Disjtograph connectivity • Given we want to • Choose vertices • Design inputs such that is connected iff

  46. Reduction from Disjtograph connectivity (Players) (Elements) input graph connected

  47. Other Stuff • Distributed computing

  48. Other Stuff • Compressing down to information cost • Number-on-forehead lower bounds • Open questions in communication complexity

More Related