1 / 6

Weak Learning DNF under uniform distribution

Weak Learning DNF under uniform distribution. A parity function weakly approximates f Find this function KM algorithm Form a tree, pruning a node if there are no large coefficients starting with that substring. Strong learning: Boosting. Recall Freund ’s algo

oded
Download Presentation

Weak Learning DNF under uniform distribution

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Weak Learning DNF under uniform distribution • A parity function weakly approximates f • Find this function • KM algorithm • Form a tree, pruning a node if there are no large coefficients starting with that substring

  2. Strong learning: Boosting • Recall Freund’s algo • Construct weak hypotheses h1,h2,h3… • At step i: Distribution Di: more weight to x on which hi,…,hi-1 were wrong Form hypo hi on Di • Combine hypotheses using some rule • Does parity approximate f on Di? • Yes.. Answer on next slide • Needs a distribution-independent weak learner • We don’t have one for DNF

  3. Strong learning DNF • Let f be a DNF having s-terms • Lemma: f has a fourier coefficient of value at least 1/(2s+1)

  4. Can we tweak KM? • Cool fact: KM works for real-valued functions as well • Idea: Construct function g such that • Depends on L(g) in running time

  5. Converting (f,D) to (g,U) • Notice that on dist D is same as on uniform • We can learn on U!! • Need MQ oracle for g => MQ oracle for D • L(g) should not be too large “D close to uniform”

  6. An appropriate Boosting algorithm • Final hypothesis – majority of his • Di = D(x)i(x)  has to be normalized i(x)  prob that hypotheses are almost equally divided over x • Stop if Dis become too small • Notice: easily computable • Close to uniform – within a small factor

More Related