330 likes | 460 Views
CS B553 : Algorithms for Optimization and Learning. Global optimization. Agenda: Global Optimization. Local search, optimization Branch and bound search Online search. f. Global Optimization. min f ( x ), x in S R n Want guarantees on a global optimum. x 2. S. x 1.
E N D
CS B553: Algorithms for Optimization and Learning Global optimization
Agenda: Global Optimization • Local search, optimization • Branch and bound search • Online search
f Global Optimization • min f(x), x in S Rn • Want guarantees on a global optimum x2 S x1
Lower-bound function • LowerBound(A): for any set A S, returns a lower bound on f(x) for all xA f x2 S A fL(A) x1
1 2 4 2.5 3 Values of LowerBound(A) f(x) = 3.2 Pruning the Search Tree
What order? Branch-and-bound Algorithm • Let f* be the best value seen so far • Init: f* = f(point in S0) • Q = {S0} • While Q not empty, repeat: • S = remove an item from Q • If LowerBound(S) f* or |S|<e, then discard S • f* = min(f*,f(point in S)) • Split S and add subregions to FRINGE Pruning step
Performance • Works well when • When LowerBoundis relatively tight • When n isn’t too large • Methods for generating LowerBound • Interval arithmetic • Solving “relaxed” versions of f • Problem-specific ways
Example: 3D Needle Steering Feedback Controller Closed-loop feedback rule: Sense current position/orientation of needle Twist at the speed such that the predicted helix path minimizes the distance to target
Find small initial domain Finding closest point Use cylindrical lower bound BnB takes < 1ms on average
Collision Checking • Check whether objects overlap
Hierarchical Collision Checking • Enclose objects into bounding volumes (spheres or boxes) • Check the bounding volumes
Hierarchical Collision Checking • Enclose objects into bounding volumes (spheres or boxes) • Check the bounding volumes first • Decompose an object into two
Hierarchical Collision Checking • Enclose objects into bounding volumes (spheres or boxes) • Check the bounding volumes first • Decompose an object into two • Proceed hierarchically
Hierarchical Collision Checking • Enclose objects into bounding volumes (spheres or boxes) • Check the bounding volumes first • Decompose an object into two • Proceed hierarchically
Bounding Volume Hierarchy (BVH) A BVH (~ balanced binary tree) is pre-computed for each object (obstacle, robot link)
A A C C B B Collision Checking Between Two Objects BVH of object 1 BVH of object 2 [Usually, the two trees have different sizes] Search for a collision
pruning Search for a Collision Search tree AA A A
Search for a Collision Search tree AA Heuristic: Break the largest BV A A
BA CA B C Search for a Collision Search tree AA Heuristic: Break the largest BV A
BA CA CB CC C C B Search for a Collision Search tree AA
C B BA CA CB CC C B Search for a Collision Search tree AA If two leaves of the BVH’s overlap(here, C and B) check their content for collision
rY d rX Y X Search Strategy • If there is no collision, all paths must eventually be followed down to pruning or a leaf node • But if there is collision, one may try to detect it as quickly as possible • Greedy best-first search strategy with f(N) = h(N) = d/(rX+rY) [Expand the node XY with largest relative overlap (most likely to contain a collision)]
Performance • On average, over 10,000 collision checks per second for two 3-D objects each described by 500,000 triangles, on a contemporary PC • Checks are much faster when the objects are either neatly separated ( early pruning) or neatly overlapping ( quick detection of collision)
Review of Optimization Unit • Descent vs root finding • Gradient descent, Newton’s method, Quasi-newton methods • Constraints: Lagrange multipliers and KKT conditions • Convex optimization • LP, QP • Interior point methods • Metaheuristic methods
Putting Optimization into Practice • Being able to formulate optimization problems is often just as important as choosing the right algorithm • Well conditioned functions • Constraints • Auxiliary variables • Transformations • Analysis vs. computation
What else is out there? • Special techniques for sparse problems • Special problem formulations • e.g., SDP, SOCP • Mixed-integer programming • Semi-infinite programming • In-depth analysis suitable for “industrial strength” optimization
Modern Trends • Learning with big data • Combinatorial / hybrid problems • Online or distributed optimization • Specific function classes: • Stochastic functions • Robot/biomechanical motions • Feedback control policies
Readings for Next Class • K&F, Chapter 2