Sublinear Algorithms

1. 1 0 3 1 2 0 1 3 … Sublinear Algorithms Lecture 23: April 20

2. Matrix Multiplication Matrix multiplication: Given two nxn matrices A,B, compute AB. Straightfoward: O(n3) algorithm. Sophisticated: O(n2.376) algorithm Matrix multiplication verification: Given A,B,C, check if AB=C. Would it be easier?

3. Matrix Multiplication Matrix multiplication verification: Given A,B,C, check if AB=C. Freivald’s technique: there is an O(n2) time randomized algorithm This technique assumes that the entries are field elements, say, rational numbers. Idea:check some random places! Can we just take some row of A and some column of B?

4. Matrix Multiplication Can we just take some row of A and some column of B? Each operation just takes linear time, i.e. O(n) time. 1 1 But the probability of catching it is too low, i.e. O(1/n2).

5. Matrix Multiplication Idea: Need to do something more global, i.e. every entry is involved. Freivald’s technique: pick a random vector Then, compute x=Br, y=Ax=ABr, z=Cr. Check if y=z. This procedure can be done in O(n2) time. Obviously, if AB=C, then y=z.

6. Matrix Multiplication Theorem: if AB≠C, then Pr[ABr=Cr] ≤ 1/2. Let D=AB-C; so D is not the all-zeroes matrix. Proof: We wish to bound the probability that y=z, or equivalently, the probability that Dr=0. Let say the first row D(1) of D has a non-zero entry. And let D(1) = {d1,d2,d3,…,dk,0,0,…,0}. What is the probability that D(1)·r is non-zero? This probability would give a lower bound that we catch y≠z.

7. Matrix Multiplication What is the probability that D(1)·r is non-zero? Recall that D(1) = {d1,d2,d3,…,dk,0,0,…,0}. This is non-zero if and only if: if and only if: r1 are uniformly chosen from two values (0 or 1), the probability that D(1)·r is non-zero is at least 1/2. Hence, we have at least 1/2 probability to catch it if AB≠C.

8. Verifying Polynomial Identities Given three polynomials P1(x), P2(x), and P3(x), verify that P1 P2 = P3. Fast Fourier Transform: O(n log(n)) time if P1,P2 are of degree n. Can we test it in linear time? Idea: just pick a random point y and test if P1(y) P2(y) = P3(y). Consider Q(x)=P1(x)P2(x) – P3(x). It is a degree 2n polynomial, and has at most 2n roots. If we choose randomly from, say, 4n elements and test it, then with probability at least 1/2 we can catch it.

9. Verifying Perfect Matching Given a bipartite graph B=(U,V,E), verify if B has a perfect matching. Consider the following matrix: if there is an edge between u(i) and v(j) = otherwise Theorem: B has a perfect matching if and only if det(A) ≠ 0.

10. Verifying Perfect Matching Theorem: B has a perfect matching if and only if det(A) ≠ 0. Since each variable x(i,j) occurs at most once in A, there can be no cancellation of the terms in the summation. Therefore the determinant is not identically zero if and only if there is a permutation πfor which the corresponding term is non-zero. That is, each entry is non-zero, and hence a perfect matching!

11. Verifying Perfect Matching Theorem: B has a perfect matching if and only if det(A) ≠ 0. So, now we “only” need to check if det(A) ≠ 0. Observation: det(A) is a (multi-variable) polynomial Idea: Substitute random points and evaluate the polynomial. Once the values are substituted, the problem reduces to compute a determinant, which can be done in O(n2.376) time. Deterministic running time = O(m√n) = O(n2.5) time.

12. Verifying Perfect Matching Idea: Substitute random points and evaluate the polynomial. What is the probability that it works? (Schwartz-Zippel Theorem): Let Q(x1,…,xn) in F[x1,…xn] be a multivariate polynomial of total degree d. Fix any finite set S of F, and let r1,…,rn be chosen independently and uniformly at random from S. Then, In our application, d = number of vertices on one side. Choose a field of size at least 2n, say modulo a small prime number, will do.

13. Verifying Equality of Strings Suppose that Alice maintains a large database of information. Bob maintains a second copy of the database. Periodically, they want to compare their databases for consistency. Any deterministic consistency check requires to transmit all n bits. Can we do better? Let the bits of Alice be (a1,…,an) and the bits of Bob be (b1,…,bn). Check random bits? Not efficient enough if there is only one error. Need some more “global” method.

14. Verifying Equality of Strings Idea: create “fingerprints” for the two databases. Interpret the data as n-bit integers a and b, by defining Define the fingerprinting function as where p is a prime number. Algorithm: pick a random prime, test if F(a) = F(b).

15. Verifying Equality of Strings Algorithm: pick a random prime, test if F(a) = F(b). Bits sent: O(log(p)) instead of n. To be efficient, want p to be small. What is the probability of making mistake? Consider c = |a-b|. How many primes can divide c? Since c ≤ 2n, the number of distinct prime divisor of c is at most n, why?

16. Verifying Equality of Strings There are at most n primes which divide c = |a-b|. Bits sent: O(log(p)) instead of n. To be efficient, want p to be small. Now we want to choose a number z, large enough so that there are enough primes smaller than z, small enough so that the communication is efficient. Given z, by the Prime Number Theorem, there are about z/ln(z) primes smaller than z.

17. Verifying Equality of Strings Fix z. Probability of making mistake ≤ n/(z/ln(z)) = nln(z)/z. Bits need to be sent ≤ O(log(z)) A good choice: set z = tn log(tn) for a large t. Probability of making mistake ≤ O(1/t). Bits need to be sent ≤ O(log(t) + log(n)) Set t=n would be good.

18. Pattern Matching Given a text X = x1 x2 … xn, and a pattern Y = y1 y2 … ym, check if there is a j so that xj … x(j+m-1) = y1 … ym. There is a straightforward O(mn) algorithm. There is an optimal O(m+n) deterministic algorithm. We now show a very simple O(m+n) randomized algorithm. Idea: fingerprinting.

19. Pattern Matching Write X(j) for xj … x(j+m-1). Goal: to check if Y=X(j) for some j. Idea: fingerprinting. Algorithm: compare F(Y) and F(X(j)), say “match” if they are equal Fix a number z, choose a random prime p smaller than z. There could be false match, but this probability is at most

20. Pattern Matching There are n positions, so the probability that we make a false match Choose This is at most O(1/n). What is the running time? So, the fingerprints can be updated in O(1) time, and hence O(n+m) time.

21. Interactive Proof System 3-SAT: Given a boolean formula, check if there is a satisfying assignment. Counting 3-SAT: Given a boolean formula, check if the number of satisfying assignments is equal to k. Reversi: Given a broad configuration, check if the black player has a winning strategy. All have an interactive proof system. Implication: even being foolish doesn’t mean someone can fool you easily!

22. Interactive Proof System Counting 3-SAT: Given a boolean formula, check if the number of satisfying assignments is equal to k. Key technique: arithmetization And then use the previous technique or picking random element to evaluate…

23. Probabilistic Checkable Proofs 3-SAT: Given a boolean formula, check if this formula is satisfiable. If this is satisfiable, you can convince me by showing a truth assignment, then I can check in linear time whether it indeed satisfies the formula. Can we do it in sublinear time? PCP theorem: There is a proof protocol with the following property: I only need to read 3 bits of the proof and if you are honest, then I always trust you; if you are lying, then I can catch you lying with probability at least ½!! Has deep implication in hardness of approximation!

24. about this course Focus: analysis Exact algorithm: running time Approximation algorithm: performance guarantee Randomized algorithm: probability • Techniques: • Combinatorial approaches • Linear programming • Probabilities • Algebraic technique? • Topological method?????

25. Feedback?