Flows and Networks (158052)

Introduction to theorie of flows in complex networks: both stochastic and deterministic apects Size 5 ECTS 16 lectures : 8 by R.J. Boucherie focusing on stochastic networks 8 by W. Kern focusing on deterministic networks Common problem How to optimize resource allocation so as to maximize flow of items through the nodes of a complex network Material: handouts / downloads Exam: exercises / (take home) exam References: see website Flows and Networks (158052) Richard Boucherie Stochastische Operations Research -- TW wwwhome.math.utwente.nl/~boucherierj/onderwijs/158052/158052.html

Motivation Production / storage system C:\Flexsim Demo\tutorial\Tutorial 3.fsm Internet Thomas Bonalds's animation of TCP.htm (www-sop.inria.fr/mistral/personnel/Thomas.Bonald/tcp_eng.html)http://www.warriorsofthe.net/trailer Main questions How to allocate servers / capacity to nodes or how to route jobs through the system to maximize system performance, such as throughput, sojourn time, utilization QUESTIONS Motivation and main question

Consider an open Jackson networkwith transition rates Assume that the service rates and arrival rates are given Let the costs per time unit for a job residing at queue j be Let the costs for routing a job from station i to station j be (i) Formulate the design problem (allocation of routing probabilities) as an optimisation problem. (ii) Provide the solution to this problem Aim: Optimal design of Jackson network

Contents • Introduction; Markov chains • Birth-death processes; Poisson process, simple queue;reversibility; detailed balance • Output of simple queue; Tandem network; equilibrium distribution • Jackson networks;Partial balance • Sojourn time simple queue and tandem network • Performance measures for Jackson networks:throughput, mean sojourn time, blocking • Application: service rate allocation for throughput optimisationApplication: optimal routing Flows and network: stochastic networks

Today: • Introduction / motivation course • Discrete-time Markov chain • Continuous-time Markov chain • Next • Exercises

AEX • Continuous, per minute, per day • Random process: reason increase / decrease ? • Probability level 300 of 400 dec 2004 ? • Given level 350 : buy or sell ? • Markov chain : random walk

Gambler’s ruin • Gambling game: on any turn • Win €1 w.p. p=0.4 • Lose €1 w.p. 1-p=0.6 • Continue to play until €N • If fortune reaches €0 you must stop • Xn= amount after n plays • For • Xn has the Markov property: conditional probability that given the entire history depends only on • Xn is a discrete time Markov chain

Markov chain • Xn is time-homogeneous • Transition probability • State space : all possible states • For gambler’s ruin • For N=5: transition matrix • Property

Markov chain : equilibrium distribution • n-step transition probability • Evaluate: • Chapman-Kolmogorov equation • n-step transition matrix • Initial distribution • Distribution at time n • Matrix form

Markov chain: classification of states • j reachable from i if there exists a path from i to j • i and j communicate when j reachable from i and i reachable from j • State i absorbing if p(i,i)=1 • State i transient if there exists j such that j reachable from i and i not reachable from j • Recurrent state i process returns to i infinitely often = non transient state • State i periodic with period k>1 if k is smallest number such that all paths from i to i have length that is multiple of k • Aperiodic state: recurrent state that is not periodic • Ergodic Markov chain: alle states communicate, are recurrent and aperiodic (irreducible, aperiodic)

Markov chain : equilibrium distribution • Assume: Markov chain ergodic • Equilibrium distributionindependent initial statestationary distribution • normalisinginterpretation probability flux

Discrete time Markov chain: summary • stochastic process X(t) countable or finite state space SMarkov propertytime homogeneous independent tirreducible: each state in S reachable from any other state in Stransition probabilities Assume ergodic (irreducible, aperiodic) global balance equations (equilibrium eqns) solution that can be normalised is equilibrium distributionif equilibrium distribution exists, then it is unique and is limiting distribution

Random walk http://www.math.uah.edu/stat/ • Gambling game over infinite time horizon: on any turn • Win €1 w.p. p • Lose €1 w.p. 1-p • Continue to play • Xn= amount after n plays • State space S = {…,-2,-1,0,1,2,…} • Time homogeneous Markov chain • For each finite time n: • But equilibrium?

Continuous time Markov chain • stochastic process X(t) countable or finite state space SMarkov propertytransition probabilityirreducible: each state in S reachable from any other state in SChapman-Kolmogorov equationtransition rates or jump rates

Continuous time Markov chain • Chapman-Kolmogorov equationtransition rates or jump rates • Kolmogorov forward equations: (REGULAR)Global balance equations

Continuous time Markov chain: summary • stochastic process X(t) countable or finite state space SMarkov propertytransition rates independent tirreducible: each state in S reachable from any other state in SAssume ergodic and regular global balance equations (equilibrium eqns) π is stationary distribution solution that can be normalised is equilibrium distributionif equilibrium distribution exists, then it is unique and is limiting distribution

Next time: • [R+SN] section 1.1 – 1.3 • Continuous – time Markov chains: Birth-death processes; Poisson process, simple queue;reversibility; detailed balance;

Exercises: • [R+SN] 1.1.2, 1.1.4, 1.1.5 • Give proof of Chapman-Kolmogorov equation • For random walk, letDetermine the possible states for N=10, and compute for all feasible j • Consider the random walk with reflecting boundary, that has transition probabilities similar to random walk, except in state 0. When the process attempts to jump to the left in state 0, it stays at 0. The transition probabilities are Show that a solution of the global balance equations is For which values of p is this an equilibrium distribution?

Flows and Networks (158052)