1 / 22

Experiments with STAGE

Experiments with STAGE. Wei Wei. Introduction. STAGE- Developed by Boyan Use value function approximation to automatically analyze sample trajectories. Speed up many local search methods. Diagram of STAGE. Produces new training data. Run p to Optimize Obj. Hillclimb to Optimize V.

libby-ramos
Download Presentation

Experiments with STAGE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiments with STAGE Wei Wei

  2. Introduction • STAGE- Developed by Boyan • Use value function approximation to automatically analyze sample trajectories. • Speed up many local search methods

  3. Diagram of STAGE Produces new training data Run p to Optimize Obj Hillclimb to Optimize V Produces good start states

  4. Apply it to SAT • The base algorithm is WalkSAT (modified) • Got results better than pure WalkSAT

  5. Overview • We need to deal with four aspects of the problem: WalkSAT, STAGE, features, and to make the algorithm Markovian. • Hard to tune; not every combination works. stage WalkSAT features Marko-vianize

  6. Features • %clauses unsatisfied (-) • %clauses satisfied by 1 variable (+) • %clauses satisfied by 2 variables (-) • %critical variables (-) • %variables set to naïve setting (~)

  7. Markovianize • S/W1 : patience based, not Markovian • S/W2 : best-so-far • S/W3 : epsilon cutoff

  8. Parameter tuning • Noise 0.25 seems good • Patience 10,000 • Cutoff 1,000,000 • Epsilon .0001

  9. Function approximator V-bar-pi • Quadratic regression • Linear regression • Linear functions perform 25% better, and faster. • Linear functions are coarse approximators.

  10. results

  11. Results – Hemming Distance traveled by the V step

  12. results

  13. Feature 1 and 2 only

  14. Added feature: %variables set to true

  15. Discussion(1) • Linear regression is very bad approximation is this case, yet it gives better results than quadratic regression. Why? • Hit bottom very often • Lead to long more WalkSAT moves

  16. Discussion(2) • Features – coefficients vary a lot among instances. But relatively stable within one instance. • The signs are relatively stable

  17. Discussion(3) • Time vs evaluation • When # of evaluation is fixed, STAGE performs 3 times better, but time spent is doubled • When time is fixed, the result is 40% better than WalkSAT

  18. Discussion(4) • Can it hit the finish line? • It does vaguely(?) learn some concepts, which hopefully can direct WalkSAT to a good place. • Par-? Is a good set of problems to solve?

  19. One feature No improvement over WalkSAT.

  20. Random restart • 176 Random flips – Worse than S/W3, still better than WalkSAT • 1000 Random flips – Worse than one-run WalkSAT • Complete new start points – similar to the case above. • Parameters: cutoff – 10,000. Restart – 100.

  21. Hanoi • Parameters not yet carefully tuned • It would be interesting to see whether Hanoi4 can be solved by carefully tuned S/W3. I ran WalkSAT for 50,000,000 flips, but failed to solve it.

  22. Hanoi problems

More Related