The Complexity of Linear Dependence Problems in Vector Spaces

1. The Complexity of Linear Dependence Problems in Vector Spaces David Woodruff IBM Almaden Joint work with Arnab Bhattacharyya, Piotr Indyk, and Ning Xie from MIT

2. The 3-SUM Problem • Given a set S containing r real numbers, are there: a, b, c 2 S with a+b+c = 0? • Solve in O(r2) time • Interview question • Conjectured to require ~(r2) time • Useful for hardness results in P. Many problems are “3-SUM Hard”

3. Generalizations • We study generalizations of this problem: • Replace 3 summands with k summands • Replace real field R with a finite field • Replace sum of field elements with sum of vectors • Replace sum with a fixed linear combination • Replace sum with any linear combination • Require vectors be minimally linearly dependent • Replace target 0 with an arbitrary vector • and so on…

4. Applications • Maximum Likelihood Decoding - Given x1, …, xr in Fqn and z in Fqn, do there exist xi1, …, xik that contain z in their span? - xi are the columns of a parity-check matrix - z is the syndrome - there is a codeword corrupted in at most k positions with syndrome z iff the k-span contains z • Weight Distribution Problem • Let A be an n x r matrix over F2 • Define the code C = {x | Ax = 0} • C has a codeword of weight k iff k columns of A sum to 0

5. Formal Definitions • In this talk, we focus on two problems: • (k,r)-LinDependence: given r elements x1, …, xr in F2n and z in F2n, do there exist xi1, …, xik that span z? • (k,r)-ZeroSum: given r elements x1, …, xr in F2n, do there exist xi1, …, xik with xi1 + xi2 + … + xik = 0? • We allow k and r to be functions of n • First problem at least as hard as second

6. Results • Assume 3-SAT cannot be solved in time less than 2cn for a constant c > 0 • Then(k,r)-ZeroSumrequires min(rk, 2n) time, up to polynomial factors • So, (k,r)-LinDependencerequires min(rk, 2n) time • Other variants also require this time • Have matching upper bound: • rk is trivial. Can get roughly rk/2 • Can get 2n with the FFT

7. Implications • (k,r)-LinDependencereduces to Maximum Likelihood Decoding, so min(rk, 2n) lower bound • (k,r)-ZeroSumreduces to the Weight Distribution Problem, somin(rk, 2n) lower bound • Results improve previous best rk1/4 lower bounds for these coding theory problems [Downey, Fellows, Vardy, Whittle] • Hold for r and k functions of n

8. 3-SAT formula F with n variables and m clauses [CIP] - s = 2εn. Each Ái has n variables and O(n) clauses Á1 Á2 … Ás • Ái replaced with 1-in-3-SAT formula Ãi Ã1 Ã2 … Ãs Our starting point: [PW] showed an r(k) bound for k-SUM over R assuming 3-SAT on n variables requires 2cn time: This ensures bit complexity of resulting numbers is small - Ãi converted to k-SUM instance Each k-SUM instance on a set of r = 2Θ(n/k) real numbers. If can solve k-SUMin time ro(k), can solve 3-SAT in time ro(k)¢2εn

9. Partition variables into k groups G1, …, Gk of n/k variables Reducing a 1-in-3-SAT formula Ãi on n variables and O(n) clauses to k-SUM on r = 2Θ(n/k) real numbers Ãi is true iff there are k real numbers that sum to 1k + O(n) vi,1 vi,2 vi,3 … vi, 2n/k • In each group Gi, create a real number vi,j for each possible assignment to its n/k variables vi,j: Base-k representation k group indicator digits O(n) clause digits • i-th indicator digit is 1 iff v 2 Gi • j-th clause digit is 1 iff A(v) sets exactly 1 literal of j-th clause to 1 • All other digits are 0

10. Can we do the same for F2? • A sum of k vectors over F2 can equal 1k+O(n), but just means an odd number of literals in each clause are true • Odd-SAT is easy • Partition variables into k groups G1, …, Gk of n/k variables vi,1 vi,2 vi,3 … vi, 2n/k • In each group Gi, create a vector vi,j for each possible assignment to its n/k variables • In each group Gi, create a real number vi,j for each possible assignment to its n/k variables vi,j: k + O(n) coordinates Base-k representation k group indicator digits k group coordinates O(n) clause coordinates O(n) clause digits • i-th indicator coordinate is 1 iff v 2 Gi • j-th clause coordinate is 1 iff A(v) sets exactly 1 literal of j-th clause to 1 • All other coordinates are 0 • i-th indicator digit is 1 iff v 2 Gi • j-th clause digit is 1 iff A(v) sets exactly 1 literal of j-th clause to 1 • All other digits are 0

11. - Before this was used for bit complexity. - Now it determines the number of dimensions 3-SAT formula F with n variables and m clauses [CIP] - s = 2εn. Each Ái has n variables and O(n) clauses Á1 Á2 … Ás • Ái replaced with NAE-SAT • Formula Ãi Ã1 Ã2 … Ãs Our Modifications • A NAE-SAT formula Ãi is 1 if for each clause, at least one but not all literals are true • We need interactionbetween groups - With 1-in-3-SAT over R, variables in different groups independently update the clause digit - Ãi converted to (k,r)-ZeroSum Each (k,r)-ZeroSum instance on a set of r = 2Θ(n/k) vectors. If can solve (k,r)-ZeroSumin time ro(k), solve 3-SAT in time ro(k)¢2εn

12. Interacting Variables • We can replace duplicates of a variable with distinct variables and introduce equality constraints • preserve NAE-SAT and · 3 literals per clause • each variable occurs in a constant number of clauses • For each clause (a Ç b Ç┐c), we introduce pairvairs • 1 variable is [a, b], 1 variable is [b, c], and 1 variable is [c, a] • Partition original n variables into k groups Gi of n/k variables • For a pairvar [a,b], • if original variables a and b occur in the same group Gi, place [a,b] in Gi • else, if a 2 Gi and b 2 Gj, place [a,b] in Gmin(i, j) Gi still has O(n/k) variables

13. New Reduction • In each group Gi, create a vector vi,j for each assignment to its n/k variables as well as variables in Gi’s pairvars 1 pair of consistency coordinates for each pairvar (a,b) vi,j: k+O(n) coordinates k group coordinates O(n) clause coordinates O(n) consistency coordinates • i-th group coordinate is 1, the others are 0 • clause coordinates more complicated • depend on variables and pairvars assigned to the group • consistency coordinates allow for assignments to the same • variable from different groups to be patched together

14. Clause Coordinates • Clause coordinates are set so that for a consistent assignment (i.e., group and consistency coordinates are ok), then for clause with literals a, b, c • v(a) + v(b) + v(c) – v(a) ¢ v(b) – v(b) ¢ v(c) – v(a) ¢ v(c) • v(.) denotes the value assigned • Case analysis • Clause only equals 1 if exactly 1 or 2 literals are true

15. Upper Bounds • Consider functions f: Z2n ! {0,1} • Fourier transform F: Z2n!R is F(x) = 2-n¢y f(y)¢(-1)<x,y> • Fast Fourier Transform computes F from f in O(n¢2n) time • Let f be indicator function of input set of r vectors. Then sum v1+v2 + …vk = 0f(v1) ¢ f(v2)  f(vk) is what we want • This is just 2n times the 0n-Fourier coefficient of fk • So we can get O(n¢2n) time instead of the trivial 2nk

16. Conclusion Assuming 3-SAT cannot be solved in time less than 2cn for a constant c > 0, • (k,r)-LinDependenceand(k,r)-ZeroSumrequiremin(rk, 2n) time (up to polynomial factors) • Same bound holds for many similar problems • Almost matching upper bounds • New way to prove hardness in coding theory • Optimal hardness of basic problems in coding theory