1 / 93

Stochastic Process

Stochastic Process. Formal definition A Stochastic Process is a family of random variables {X(t) | t  T } defined on a probability space, indexed by the parameter t, where t varies over an index set T You are supposed to associated T with time Examples

moesha
Download Presentation

Stochastic Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stochastic Process • Formal definition • A Stochastic Process is a family of random variables {X(t) | t  T} defined on a probability space, indexed by the parameter t, where t varies over an index set T • You are supposed to associated T with time • Examples • Frog jumping from one lily pad to another 　 X(t) = lily pad occupied by the frog at time t • Flip a coin and take one step north if heads and one step south if tails（discrete time , discrete state） • 股票, 天氣

  2. arrivals Server departures Queue on waiting room Queuing Systems as stochastic processes • Single server model • Arrival process • Queue scheduling discipline • Service time distribution System

  3. Some random processes associated with a queuing system Continuous time random processes • X(t) ＝ # of arrivals in （0, t） • N(t) ＝ # of customs(jobs) in system at time t • X(tn) ＝ # of customs(jobs) in system when nth arrival occurs • U(t) ＝ amount of unfinished work in the system at time t Discrete time random process Continuous time random process

  4. Markov ??? M/M/1/N • M： Arrival：Poisson Service：Exponential • G：General • D：Deterministic Arrival process # of servers Service time distribution Finite waiting room Memoryless

  5. Fig 2.2 Cn-1 Cn Cn+1 Cn+2 • Wn = nth customer’s waiting time(inside the queue) • Xn = nth customer’s service time(inside the server) • System time = waiting time + service time Xn+1 Xn+2 Xn idle Server Cn Cn+1 Cn+2 Wn Wn+1 Wn+2=0 Queue Cn Cn+1 Cn+2

  6. Little’s ResultλT=n • Def: • λ：mean long term arrival rate (no assumption about Poisson or anything else -not even i.i.d. interarrival times) • T：mean time a customer spends in the black box • n：mean number of customers in the box Black box arrivals departures

  7. Proof customers Err T5 T4 T3 T2 T1 time 0 t Err

  8. Let • n(t) = # of cistomers in the system at time t • A(t) = # of arrivals in (0, t) • # of customers in the system= area

  9. ‾= n ‾= n t→∞, 只要 steady state, Err(t) won’t grow up ∴ , as t →∞

  10. Stochastic Processes • Notation: F(x0;t0) = FX(t0)(x0) = P[X(t0) ≦x0] • Describing stochastic processes • F(x1, x2,…, xn; t1, t2,…, tn) = F(x; t) =P[X(t1) ≦ x1, X(t2) ≦ x2,…, X(tn) ≦ xn] Sample path Distrib of initial state τ τ 0 t1 t2 t

  11. Special case • Stationary • F(x;t)=F(x; t + τ), where t + τ = t1 + τ, t2 + τ, …, tn + τ • Independent • F(x; t)=Fx(t1)(x1)Fx(t2)(x2)…Fx(tn)(xn)

  12. Markov Processes • Future behavior does not depend on all of past history • Only depends on current state • P[X(t) = x|X(t1) = x1,…, X(tn) = xn] =P[X(t) = x| X(tn) = xn], t1<t2<…<tn<t • Concerned mostly with discrete state Markov Chains • Discrete and continuous time • Time homogeneous Markov processes • P[X(t) ≦ x| X(tn) ≦xn]=P[X(t + τ) ≦ x| X(tn + τ) ≦xn], for all τ

  13. Markov Chins --discrete state space • May be continuous time or discrete time p12 2 p24 p23 p14 1 4 p44 p13 p31 p43 3

  14. Markov Chain • For discrete parameter • We only consider the sequence of states and not the time spent in each state • For continuous time M.C. • The “holding time” in each state is exponential distributed • Must be if future behavior only depends on current state • Need the memoryless property of the exponential

  15. Discrete time M.C. • P = transition probability Matrix • P[i, j] = p[X(t+1) = j | X(t) = i] • Continuous time M.C. • Rate of transmission between states

  16. p1 m1 q1 Interconnect p2 m2 q2 qm pn mm How to select state? • Ex: • Model of a shared memory multiprocessors

  17. m1 ?? q1 ?? m2 q2 qm ?? mm How to select state? (cont.) • An abstraction

  18. < (p1, p2), (　 ) > 1 < ( p1 ), ( p2 ) > 2 < ( p2 ), ( p1 ) > 3 < (p2,p1), ( ) > 4 < ( ), (p1, p2) > 5 < ( ), (p2,p1) > 6 How to select state? (cont.) • Parameter case • 2 processors • 2 memory modules ?????? ??????

  19. Discrete Parameter Markov Chains • Discrete state space • Finite or countable infinite • Pjk(m, n) = P[Xn = k | Xm = j], 0 ≦ m ≦ n • For time homogeneous Markov Chains • Pjk(m, n) is only a function of n-m • In particular,Pjk = Pjk(m, m+1)

  20. Transition probability matrix • P = P[j, k] (stochastic matrix) • All elements ≧ 0 and row sums = 1 • π(0) = initial state probability vector • πi(0)= ith component of π(0) = prob. system starts in state i

  21. 12 12 12 12 0 0 0 12 12 12 j i Xn-1 Xn State-transition diagram • Example: Random walk • N-step Transition Probabilities • Def: n-step transition prob. matrix

  22. 1 transient Type of states & types of Markov Chains • Transient states • “no way to go back” • “1” is a transient state • Pji(n) → 0, as n →∞

  23. Recurrent states • 相反於 transient • State i is recurrent iff starting from any state j, state i is visited eventually with probability 1. Pji(n) → 1, as n →∞ • Expectation is ∞ • Recurrent-null • Mean time between visits is ∞ • Recurrent-non-null • Mean time between visits is < ∞ • Periodic • di=g.c.d of all n such that Pii(n) ≠ 0 • aperiodic if di = 1

  24. Absorbing states • “trap states“ • “4” is an absorbing state, so is “5” p 4 7 5 1-p

  25. Irreducible Markov Chain • Any state is reachable from any other state in a finite number of transition, • i.e. vi, j, Pij(n) > 0 for some finite n • For an irreducible Markov chain • All states are aperiodic or all are periodic with the same period • All states are transient or none are • All states are recurrent or none are

  26. Let π(0) = vector of initial state probabilities • i.e. πi(0) = prob. start in state i 　　πi(0) = prob. in state i at time = 1

  27. Thm. • For any aperiodic chain, the limit • Thm. • For an irreducible, aperiodic M.C., the limit of πexists and is independent of the initial state probability distribution • Thm. • For an irreducible, aperiodic M.C. with all states recurrent non-null, πis the unique stationary state probability vector

  28. 1 A B 1 1 C Example • Limit does not exist for a periodic chain: • π(0) = (1, 0, 0) • π(3k) = (1, 0, 0) • π(3k+1) = (0, 1, 0) • π(3k+2) = (0, 0, 1) no limit

  29. A B C D A B C D Example • Reducible M.C: • Solution to π= πP is not independent of initial state prob. Vector • π(0) = (1, 0, 0, 0) Or π(0) = (0, 0, 1, 0) B D A C

  30. q1 p1 m1 Interconnect q2 p2 m2 q1 (2, 0) (1, 1) (0, 2) (2, 0) q2 q12 (2, 0) (1, 1) 1-q12-q22 q22 (1, 1) q1 (0, 2) (0, 2) Recall: problem before --2 memory modules case and 2 processors • Step 1: select state • (2, 0) (1, 1) (0, 2) • Step 2: transition P => q1 P1p2 m1 q2 m2 q2

  31. Step 3: 求π • π= πP • Σ πi = 1 令π=(π1, π2, π3) • (π1, π2, π3) = (π1, π2, π3)

  32. Performance measure • Usually expressed as “reward functions” • E.q. memory bandwidth: • i.e.expected # of references completed per cycle = π(2, 0)‧1 + π(0, 2)‧1 + π(1, 1)‧2

  33. a0 a1 a2 b0 b1 b2 0 1 2 d1 d2 d3 Discrete Parameter Birth-Death Process • bi = prob. of a birth in state i • di = prob. of a death in state i • ai = prob. of a neither in state I • π= πP • Straight form above eqn

  34. Substitute imploes.

  35. Messy with all the b.d.a. arbitrary

  36. Σ pik = 1 pji πj j pik i πi j πj • 另解 • π= πP（by balance equation）

  37. πjPji= • ‘rate’ at which state i is entered from state j • ∴ Σj πjPji= total ‘rate’ into state i • πj = fraction of visits to state i = ‘rate’ at which state i is exited • πi = Σj πjPji（π= πP）

  38. a0 a1 a2 ai ai+1 b0 b1 b2 bi bi+1 0 1 2 i i+1 d1 d2 d3 di+1 di+2 • Another way of solving • --sometimes were convenient • πi bi = πi+1 di+1 πi+1 = （ bi∕ di+1）πi

  39. Examples • An infinite stack • A finite stack • Two stacks growing toward each other

  40. Some properties of Poisson processes • Alternate Def. • N(0) = 0 • Independence of non-overlapping intervals • Time homogeneity • P[1 event in (t, t+h)] = λh + o(h) • P[0 event in (t, t+h)] = 1- λh +o(h) • P[ > 1 event in (t, t+h)] = o(h)

  41. t+h 0 t t+h • Maybe • 0 event • 1 event • 2 events (p → 0)

  42. Poisson λA What distribution? Still Poisson? λz Poisson • Merging independent Poisson processes

  43. ? A λ What distribution? Still Poisson? Poisson Z ? • Splitting a Poisson process

More Related