CMPE 252A: Computer Networks Review Set:

1. CMPE 252A: Computer NetworksReview Set: Quick Review of Probability Theory and Random Processes

2. Why Probability Theory? • Information is exchanged in a computer network in a random way, and events that modify the behavior of links and nodes in the network are also random • We need a way to reason in quantitative ways about the likelihood of events in a network, and to predict the behavior of network components. • Example 1: • Measure the time between two packet arrivals into the cable of a local area network. • Determine how likely it is that the interarrival time between any two packets is less than T sec.

3. Probability Theory • A mathematical model used to quantify the likelihood of events taking place in an experiment in which events are random. • It consists of: • A sample space: The set of all possible outcomes of a random experiment. • The set of events: Subsets of the sample space. • The probability measure: Defined according to a probability law for all the events of the sample space.

4. Random Experiment • A random experiment is specified by stating an experimental procedure and a set of measurements and observations. • Example 2: We count the number of packets received correctly at a base station in a wireless LAN during a period of time of T sec. We want to know how likely it is that the next packet received after T sec is correct.

5. Our Modeling Problem ... • We want to reason in quantitative terms about the likelihood of events that can be observed. • This requires: • Describing all the events in which we are interested for the experiment. • Combining events into sets of events that are interesting (e.g., all packets arriving with 5 sec. latency) • Assigning a number to each of those events reflecting the likelihood that they occur.

6. Probability Law • Probability of an event: A number assigned to the event reflecting the likelihood with which it can occur or has been observed to occur. • Probability Law: A rule that assigns probabilities to events in a way that reflects our intuition of how likely the events are. What we need then is a formal way of assigning these numbers to events and any combination of events that makes sense in our experiments!

7. Probability Law • LetE be a random experiment (r.e.) • Let Abe an event in E • The probability of Ais denoted by P(A) • A probability law forE is a rule that assigns P(A) to A in a way that the following conditions, taken from our daily experience, are satisfied: • A may or may not take place; it has some likelihood (which may be 0 if it never occurs). • Something must occur in our experiment. • If one event negates another, then the likelihood that either occurs is the likelihood that one occurs plus the likelihood that the other occurs.

8. Probability Law • More formally, we state the same as follows. • A probability law for E is a rule that assigns a number P(A) to each event A in E satisfying the following axioms: • AI: • AII: • AIII: Everything else is derived from these axioms!

9. Important Corollaries 1. 2. 3. 4. 5.

10. S Probability Law • All probabilities must be in [0, 1] • The sum of probabilities must be at most 1 A x

11. time time Conditional Probability Events of interest occur within some context, and that context changes their likelihood. packet collisions interarrival times We are interested in events occurring given that others take place!

12. Conditional Probability • The likelihood of an event A occurring given that another event B occurs is smaller than the likelihood that A occurs at all. • We define the conditional probability of A given B as follows: We require P(B) > 0 because we know B occurs!

13. time Theorem of Total Probability • Purpose is to divide and conquer • We can describe how likely an event is by partitioning it into mutually exclusive pieces. busy period idle period busy period success failure

14. A ... Theorem of Total Probability Intersections of A with B’s are mutually exclusive

15. Independence of Events • In many cases, an event does not depend on prior events, or we want that to be the case. • Example: Our probability model should not have to account for the entire history of a LAN. • Independence of an even with respect to another means that its likelihood does not depend on that other event. • A is independent of B if P(A | B) = P(A) • B is independent of A if P(B | A) = P(B) • So the likelihood of A does not change by knowing about B and viceversa! • This also means:

16. Random Variables • We are not interested in describing the likelihood of every outcome of an experiment explicitly. • We are interested in quantitative properties associated with the outcomes of the experiment. • Example: • What is the probability with which each packet sent in an experiment is received correctly? We don’t really care! • What is the probability of receiving no packets correctly within a period of T sec.? We care! This may make a router delete a neighbor!

17. Random Variables • We implicitly use a measurement that assigns a numerical value to each outcome of the experiment. • The measurement is [in theory] deterministic; based on deterministic rules. • The randomness of the observed values of the measurement is completely determined by the randomness of the experiment itself. • A random variableX is a rule that assigns a numerical value to each outcome of a random experiment Yes, it is really a function!

18. S 0 Random Variables • Definition:A random variable X on a sample space S is a function X: S ->R that assigns a real number X(s) to each sample point s in S. which is called the event space

19. 1 S s 0 Possible values of X Random Variables • Purpose is to simplify the description of the problem. • We will not have to define the sample space!

20. time t 0 next packet Busy period time time? Types of Random Variables • Discrete and continuous: Typically used for counting packets or measuring time intervals. 1 2 3 4

21. Discrete Random Variables • We are interested in the probability that a discrete random variable X (our measurement!) equals a certain value or range of values. • Example: We measure the delay experienced by each packet sent from one host to another over the Internet; say we sent 1M packets (we have 1M delay measurements). We want to know the likelihood with which any one packet experiences a delay of 5 ms or less. • Probability Mass Function (pmf) of a random variable X : The probability that X assumes a given value x

22. Cumulative Distribution Function (cdf) of a random variable X: The probability that X takes on any value in the interval Discrete Random Variables The pdf and pmf of a random variable are just probabilities and obey the same axioms AI to AIII. Therefore,

23. Continuous Random Variables • The probability that a continuous r.v. X assumes a given value is 0. • Therefore, we use the probability that X assumes a range of values and make that length of that range tend to 0. • The probability density function (pdf) of X, if it exists, is defined in terms of the cdf of X as

24. What We Will Use • We are interested in: • Using the definitions of well-known r.v.s to compute probabilities • Computing average values and deviations from those values for well-known r.v.s

25. 2 6 A 7 B 5 1 C 3 4 D VC2 For VC1 use 3 For VC2 use 2 ….. For VCn use 3 VC1 Mean and Variance • What is the average queue length at each router? • What is our worst case queue?

26. queue size too much? mean time Mean and Variance Expected value or mean of X:

27. The variance of a r.v. is defined as the mean squared variation Important relation: Variance Describes how much a r.v. deviates from its average value, i.e., D = X - E(X) We are only interested in the magnitude of the deviation, so we use

28. Properties of Mean and Variance Useful when we discuss amplifying random variables or adding constant biases.

29. Examples of Random Variables • We are interested in those r.v. that permit us to model system behavior based on the present state alone. • We need to • count arrivals in a time interval • count the number of times we need to repeat something to succeed • count the number of successes and failures • measure the time between consecutive arrivals • The trick is to map our performance questions into the above four types of experiments

30. Bernoulli Random Variable • Let A be an event related to the outcomes of a random experiment. • X = 1 if outcome occurs and 0 otherwise • This is the Bernoulli r.v. and has two possible outcomes: success (1) or failure (0) We use it as a building block for other types of counting

31. time Geometric Random Variable • Used to count the number of attempts needed to succeed doing something. • Example: How many times do we have to transmit a packet over a broadcast radio channel before it is sent w/o interference? failure failure failuresuccess! Assume that each attempt is independent of any prior attempt! Assume each attempt has the same probability of success (p)

32. Geometric Random Variable • We want to count the number k of trials needed for the first success in a sequence of Bernoulli trials! Why this is the case is a direct consequence of assuming independent Bernoulli trials, each with the same probability of success: k-1 failures needed before the last successful trial Memoryless property: The probability of having a success in k additional trialshaving experienced n failures is the same as the probability of success in k trials before the n failed trails

33. Binomial Random Variable • X denotes the number of times success occurs in n independent Bernoulli trials.

34. Binomial Random Variable Because each outcome is mutually exclusive of the others:

35. The probability of a new arrival in a given subinterval is • defined to be • is constant, independent of the subinterval. Poisson Random Variable • We use this r.v. in cases where we need to count event occurrences in a time period. • An event occurrence will typically be a packet arrival. • Arrivals are assumed to occur at random over a time interval. • The time interval is (0, t] • We divide the time interval into n small subintervals of length

36. arrival …. time 0 t Poisson Random Variable A sequence of n independent Bernoulli trials; with X being the number of arrivals in (0, t] By assumption, whether or not an event occurs in a subinterval is independent of the outcomes in other subintervals. We have: 1 2 3 k 1 2 3 4 n

37. Poisson Random Variable

38. Poisson Random Variable • We see that the Poisson r.v. is the result of an approximation of i.i.d. arrivals in a time interval. • We will say “arrivals are Poisson” meaning we can use the above formulas to describe packet arrivals • The probability of 0 arrivals in [0, t] plays a key role in our treatment of interarrival times.

39. The parameter is called the arrival rate Important Properties of Poisson Arrivals • The aggregation of Poisson sources is a Poisson source. • If packets from a Poisson source are routed such that a path is chosen independently with probability p, that stream is also Poisson, with rate p times the original rate. • It turns out that the time of a given Poisson arrival is uniformly distributed in a time interval

40. Exponential Random Variable • Let Y be the time interval from a time origin (chosen • arbitrarily) to the first arival after that origin • Consider the Poisson r.v., then • Let then • We can obtain now a c.d.f. for Y as follows: and the p.d.f. of Y is then

41. time time? time >t ? time > t? Memoryless Property of Exponential Random Variable • After waiting h sec. for the first arrival, the probability that it occurs after t sec. equals the prob. that the first arrival occurs after t sec. • Knowing that we have waited any amount of time does not improve our knowledge of how much longer we’ll have to wait!

42. Mean and Variance Results Exponential: You have to memorize these! You should be able to derive any of the above Poisson: Geometric: Binomial:

43. X(t5) is a R.V. x x state space of random process x x x x a state of X(t) x x x x x t Random Processes • Definition: A random process is a family of random variables {X(t) | t is in T }defined on a given sample space, and indexed by a parameter t, where t varies over an index set T.

44. 5 4 3 2 1 t a r.v. a r.v. Poisson Process • We assume that events occur at random instants of time at an average rate of lambda events per second. • Let N(t) be the number of event occurrences (e.g., packet arrivals) in (0,t]. • N(t) is a non-decreasing, integer-valued, continuous time random process.

45. Poisson Process • As we did for the case of the Poisson r.v., we divide the time axis into very small subintervals, such that • The probability of more than one arrival in a subinterval is much smaller than the probability of 0 or 1 arrivals in the subinterval. • Whether or not an arrival occurs in a subinterval is independent of what takes place in any other subinterval. • We end up with the same Binomial counting process and with the length of the subintervals going to 0 we can approximate: Again, inter arrival times are exponentially distributed with parameter lambda

46. Poisson Process • Why do we say that arrivals occur “at random” with a Poisson process? • Suppose that we know that only one arrival occurs in (0, t] and x is the arrival time of the single arrival. • Let N(x) be the number of arrivals up to time x, 0 < x ≤ t • Then: N(t) -N(x) = increment of arrivals in (x, t]

47. Poisson Process • Arrivals in different intervals are independent. The above probability corresponds to the uniform distribution! Hence, in the average case, a packet arrives in the middle of a fixed time interval with Poisson arrivals