Seminar in Foundations of Privacy

1. Seminar in Foundations of Privacy Adding Consistency to Differential Privacy Attacks on Anonymized Social Networks Inbal Talgam March 2008

2. 1. Adding Consistency to Differential Privacy

3. Differential Privacy • 1977 Dalenius - The risk to one’s privacy is the same with or without access to the DB. • 2006 Dwork & Naor – Impossibe (auxiliary info). • 2006 Dwork et al – The risk is the same with or without participating in the DB. Plus: Strong mechanism of Calibrated Noiseto achieve DP while maintaining accuracy. • 2007 Barak et al - Adding consistency.

4. Contingency Table # # … 0…0 0…1 … 2k attribute settings Marginals 8 3 … 0 9 … j << k 2j attribute settings 2i attribute settings Setting – Contingency Table and Marginals 0 1 0 0 1 1 1 0 0 0 1 0 1 0 … n participants DB k binary attributes Terminology: Contingency table (private), marginals (public).

5. Contingency Table NaN -0.5 … Marginals 2 0 … Noise Main Contribution • Solve following consistency problem: • At low accuracy cost +

6. Outline • Discussion of: • Privacy • Accuracy & Consistency • Key method - Fourier basis • The algorithm • Part I • Part II

7. Privacy – Definition • Intuition: The risk is the same with or without participating in the DB • Definition: A randomized function K gives ε-differential privacy if for all DB1, DB2 differing on at most 1 element DB1 DB2 Differing on 1 element

8. Pls let me know f(DB) DB Laplace noise: Pr[K(DB)=a] exp (||f(DB) - a||1 / σ) Noise Noise Noise Goal: Privacy - Mechanism K(DB) = f(DB)+

9. For f : D→ Rd, the L1-sensitivity of f is for all DB1, DB2 differing on at most 1 element The Calibrated Noise Mechanismfor DP • Main idea: Amount of noise to add to f(DB) is calibrated according to the sensitivity of f, denoted Δf. • Definition: • All useful functions should be insensitive… (e.g. marginals)

10. The Calibrated Noise Mechanism – How Much Noise • Main result: To ensure ε-differential privacy for a query of sensitivity Δf, add Laplace noise with σ = Δf/ε. • Why does it work? Remember: Laplace: Definition: Pr[K(DB)=a] exp (||f(DB) - a||1 / σ)

11. Contingency Table Contingency Table New Table + NaN 8 8 3 3 -0.5 … … … Marginals Marginals 3 2 2 0 … … Noise Noise • Compromise accuracy • Non-calibrated, binomial noise Var=Θ(2k) Accuracy & Consistency So smoking is one of the leading causes of statistics? + • Compromise consistency • May lead to technical problems and confusion

12. Contingency Table 8 3 … Marginals 2 0 … Noise Key Approach Small number of coefficients of the Fourier basis • Non-redundant representation • Specific for required marginals + + Consistency: Any set of Fourier coefficients correspond to a (fractional and possibly negative) contingency table. Linear Programming + Rounding Accuracy: Few Fourier coefficients are needed for low-order marginals, so low sensitivity and small error.

13. DB Accuracy – What is Guaranteed • Let C be a set of original marginals, each on ≤ j attributes. • Let C’ be the result marginals. • With probability 1-δ, : • Remark: Advantage of working in the interactive model.

14. Outline • Discussion of: • Privacy • Accuracy & Consistency • Key method - Fourier basis • The algorithm • Part I • Part II

15. Contingency Table xα where # # … Marginal 2 0 … Cβ(x) : Notation & Preliminaries • ||x||1 = ? • We say α ≤ β if β has all α’s attributes (and more) e.g. 0110 ≤ 0111 but not 0110 ≤ 0101 • Introduce the linearmarginal operatorCβ β determines attributes • Remember: xα, α ≤ β, Cβ(x), Cβ(x)γ x0…0 x0…1

16. … The Fourier Basis • Orthonormal basis for space of contingency tables x (R2k). • Motivation: Any marginal Cβ(x) can be written as a combination of few fα’s. • How few? Depends on order of marginal. • fα:

17. Write x in Fourier basis Marginal of x with attributes β Linearity Proof. For any coordinate By definition of marginal operator and Fourier vector Writing marginals in Fourier Basis • Theorem:

18. Outline • Discussion of: • Privacy • Accuracy & Consistency • Key method - Fourier basis • The algorithm • Part I – adding calibrated noise • Part II – non-negativity by linear programming

19. Algorithm – Part I INPUT: Required marginals {Cβ} • {fα} = Fourier vectors needed to write marginals • Releasing marginals {Cβ(x)} = releasing coeffs <fα,x> OUTPUT: Noisy coeffs {Φα} METHOD: Add calibrated noise • Sensitivity depends on |{α}| on order of Cβ’s

20. minimize b subject to: x'γ ≥ 0 |Φα - <fα,x'>| ≤ b Part II – Non-negativity by LP INPUT: Noisy coeffs {Φα} OUTPUT: Non-negative contingency table x' METHOD: Minimize difference between Fourier coefficients • Most entries x'γ in a vertex solution are 0 • Rounding adds small error

21. Part I Part II Algorithm Summary Input: Contingency table x, required marginals {Cβ} Output: Marginals {Cβ} of new contingency table x'' • {fα} = Fourier vectors needed to write marginals • Compute noisy Fourier coefficients {Φα} • Find non-negative x' with nearly the correct Fourier coefficients • Round to x''

22. Accuracy Guarantee - Revisited • With probability 1-δ, #Coefficients

23. Summary & Open Questions • Algorithm for marginals release • Guarantees privacy, accuracy & consistency • Consistency: can reconstruct a synthetic, consistent table • Accuracy: error increases smoothly with order of marginals • Open questions: • Improving efficiency • Effect of noise on marginals’ statistical properties

24. Any Questions?