1 / 10

MDP and Queues: Optimization, Probability, and Review

Explore Markov Decision Processes (MDP) and Queues through optimization, probability, and review of inventory models. Learn how to analyze and improve system performance measures like waiting time and customer arrivals.

eliasn
Download Presentation

MDP and Queues: Optimization, Probability, and Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Previously • Optimization • Probability Review • Inventory Models • Markov Decision Processes

  2. Agenda • Hwk • Projects • Markov Decision Processes • Queues

  3. Markov Decision Processes (MDP) • States i=1,…,n • Possible actions in each state • Reward R(i,k) of doing action k in state i • Law of motion: P(j | i,k)probability of moving ij after doing action k

  4. MDP f(i) = largest expected current + future profitif currently in state i f(i,k) = largest expected current+future profitif currently in state i, will do action k f(i) = maxk f(i,k) f(i,k) = R(i,k) + ∑j P(j|i,k) f(j) f(i) = maxk [R(i,k) + ∑j P(j|i,k) f(j)]

  5. MDP as LP f(i) = maxk [R(i,k) + ∑j P(j|i,k) f(j)] Idea: f(i) decision variables piecewise linear function min ∑j f(i) s.t. f(i) ≥ R(i,k) + ∑j P(j|i,k) f(j) for all i,k

  6. MDP Examples • Breast cancer screening • Stock options • Airline ticket pricing • Elevator scheduling • Reservoir management

  7. the “system” Queues (Ch 14) • Queue = waiting line image from http://staff.um.edu.mt/jskl1/simweb/intro.htm

  8. Examples • Airport security • Customer service line • Checkout • Doctor’s office • ER • Canada: scheduling operations • Elevators

  9. T time in system Tq waiting time (time in queue) N #customers in system Nq #customers in queue W = E[T] Wq= E[Tq] L = E[N] Lq= E[Nq]  fraction of time servers are busy (utilization) departures arrivals queue servers system Performance Measures

  10. Randomness is Key waiting time customer # arrivals every 15 min (not random) processing times random with mean of 13 min (exponential random variable)

More Related