1 / 25

Assignment 2 Observations

Assignment 2 Observations. Mausam. The SAT Encoding. Induced Subgraph Isomorphism NP complete problem Applications: similarities between chemical compounds; protein-interaction networks, social networks, circuit design.. Variables

samson
Download Presentation

Assignment 2 Observations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assignment 2 Observations Mausam

  2. The SAT Encoding • Induced Subgraph Isomorphism • NP complete problem • Applications: similarities between chemical compounds; protein-interaction networks, social networks, circuit design.. • Variables • X(i,j) denotes that ith node (in smaller graph) mapped to jth node (in larger graph) • Self edge: If node i has a self-edge and node j doesn’t • remove X(i,j) • Self-edge: If node j has a self-edge and node i doesn’t • remove X(i,j)

  3. The SAT encoding • Each node maps to exactly one other node • X(i,1)  X(i,2)  X(i,3)…  X(i,n) • Pairs of  X(i,1)   X(i,2);  X(i,1)   X(i,3) … • Each node in larger graph maps to at most one node • Pairs of X(1,j)X(2,j); X(1,j)X(3,j); … • Disallowed pairs of mappings • If edge(i1,i2) and no edge(j1,j2): X(i1,j1)X(i2,j2); • If no edge(i1,i2) and edge(j1,j2): X(i1,j1)X(i2,j2); • If edge(i1,i2) and edge(j2,j1): X(i1,j1)X(i2,j2); • Overall – O(nm) vars, O(n2m2) clauses

  4. Fun optimizations U S Yes. A mapping is: M(A) = S, M(B) = Q, M(C) = R The edges from P to other nodes don‘t matter since no node in G got mapped to P. • Degree trick: Can B ever map to P/S/R? • No: Indegree of B is greater than indegree of P/S/R • Remove X(B,P), X(B,S), X(B,R) • Common neighbor constraint: Can A-U and C-T? • They both don’t have an edge • They both have outdegrees 1 • No: because AC have common neighbor 1; UT 0 • X(A,U)X(C,T) • … A C P R T B Q

  5. Observations • People solved ~100+80 node problems… • For an NP hard problem that’s not too bad, is it? • Paresh’s observations • adding redundant clauses helps miniSAT for tough problems.

  6. Best Competitors • Kim Wallmark • Alan Ludwig • HaroonBarri

  7. Assignment 3 Discussion Mausam

  8. BlackJack: State Space • MinSum: the minimum sum of the player's hand so far. • NumAces: the number of Aces in the player's hand so far. • dMinSum: the minimum sum of the dealer's hand so far. • dNumAces: the number of Aces in the dealer's hand so far. • isTwoCards: a Boolean that represents that the player's hand has just two cards so far. • isBlackJack: a Boolean that represents that the player got a BlackJack. • pair: has value between 0 and 10. Zero indicates that the player doesn't have a pair. Any other value i indicates that the player has two cards of value i. • turn: a Boolean to indicate whether it is player's turn or dealer's.

  9. BlackJack: Action Space • Stand: Turn = 0 • Hit: get a new card with prob… update minSum, numAces, isTwoCard, pair • Double: get a new card, update minSum, numAces, etc.; • Turn = 1. • Allowed when isTwoCard = 1 • Default: run default dealer policy. • Allowed with turn = 1 • Split: tricky! • Allowed with pair > 0; isTwoCard = 1

  10. Equations

  11. BlackJack Split (say only three cards) • Q((c,c), SPLIT) • 2p1p1V(c,1) • p1p2V(c,1) + p1p2V(c,2) • p1p3V(c,1) + p1p3V(c,3) • p2p1V(c,1) + p2p1V(c,2) • 2p2p2V(c,2) • p2p3V(c,3) + p2p3V(c,3) • p1p3V(c,1) + p1p3V(c,3) • p2p3V(c,3) + p2p3V(c,3) • 2p3p3V(c,3) • (c,c) • [(c,1) & (c,1)] p1p1 • [(c,1) & (c,2)] p1p2 • [(c,1) & (c,3)] p1p3 • [(c,2) & (c,1)] p2p1 • [(c,2) & (c,2)] p2p2 • [(c,2) & (c,3)] p2p3 • [(c,3) & (c,1)] p3p1 • [(c,3) & (c,2)] p3p2 • [(c,3) & (c,3)] p3p3 2p1(p1+p2+p3) = 2p1

  12. Equations

  13. Value of Terminal States • If isTwoCards = 1, minSum=11, numAces = 1 • Return 1.5 • If minSum>21 • Return -1 • If minSum < 21 and dMinSum > 21 • Return 1 • … • …

  14. Can you solve it with Expectimax? • No • Because there is possibility of infinite loop • Values are well-formed • Loop of size n becomes less probable as n increases • (see beginning example from lecture on MDPs) • EXCEPTION: Q(1010, split) for p> 0.5 • Solution: use Value Iteration/Policy Iteration

  15. Optimizations • Dealer’s policy is fixed. So, run the Markov chain ahead of time to generate a table • Pr(dealer gets 17|first card=c) • Pr(dealer gets 18|first card=c) • Pr(dealer gets 19|first card=c) • Pr(dealer gets 20|first card=c) • Pr(dealer gets 21|first card=c) • Pr(dealer busts|first card=c) • Remove dMinSum, dNumAces, turn. • Add dFirstCard • Remove Default action • Change the equations for double/stand based on computed probabilities

  16. Comments • numAces – not required. Convert to hasAce • Only first ace is relevant. After that each ace = 1 • Handle infinite splits by allowing say, 10 splits • Should work fine in practice • Modification: have the bet amount in state space • Needn’t multiply by 2 in equations for doubling • Need to use this info when computing terminal rewards

  17. Difference in solutions? • In real BJ: Dealer and player BJ is a push • In our BJ: player wins • ??? • In real BJ: there is a 4/6/8 card deck • In our BJ: infinite cards • You can easily modify code to handle 1st case • 2nd case: Monte carlo sampling!

  18. Should you play our BlackJack? • Expected reward before game starts • SUM[Pr(game)*expected reward of the game] • Deal three cards… and use V(s) to compute exp reward • My expected reward: $0.058 • Yes! We should play the game

  19. Discussion Mausam

  20. Followup Courses • 546 Machine Learning/Data Mining • 515 Probabilistic Graphical Models • 517 Natural Language Processing • ??? AI/Statistics for Big Data • 571 Robotics • 579 Intelligent control • 574 Special Topics in AI • 528 Computational Neuroscience • 510 Human computer Interaction

  21. Final Exam • 50% objective • True/false • Multiple choice • Fill in the blanks • Match the following… • 25% short answer questions • 1 word – 5 line responses… • opinions/strengths/weaknesses/etc… • 25% Problem solving • Problem modeling; short calculations…

  22. Why do an exam? • Goal: revise and analyze the whole course • Expected preparation time • ~1 day • Go over lecture slides • Read parts of the notes that are unclear…

  23. What worked • Programming Assignments • You hated them • You loved them • You learned through them • Written Assignments(?) • Forced us to be regular • Classes • I absolutely enjoyed the interaction! • Emails/bulletin boards • Videos(?)

  24. What didn’t work • Programming environment • People didn’t know unix • People didn’t know Java! • Packed • Once someone got behind, it got tough to recover • Workload

  25. Thanks • To all of you • who contributed to helping each other through newsgroup and other questions • who asked terrific questions in class • who gave regular feedback on class, assignments, grading, organization

More Related