260 likes | 427 Views
Review of Probability Theory. [Source: Stanford University]. Random Variable. A random experiment with set of outcomes Random variable is a function from set of outcomes to real numbers. Example. Indicator random variable: A : A subset of is called an event. CDF and PDF.
E N D
Review of Probability Theory [Source: Stanford University]
Random Variable • A random experiment with set of outcomes • Random variable is a function from set of outcomes to real numbers
Example • Indicator random variable: • A : A subset of is called an event
CDF and PDF • Discrete random variable: • The possible values are discrete (countable) • Continuous random variable: • The rv can take a range of values in R • Cumulative Distribution Function (CDF): • PDF and PMF:
Expectation and higher moments • Expectation (mean): • if X>0 : • Variance:
Two or more random variables • Joint CDF: • Covariance:
Independence • For two events A and B: • Two random variables • IID : Independent and Identically Distributed
Bernoulli Distribution • The same as indicator rv: • IID Bernoulli rvs (e.g. sequence of coin flips)
Binomial Distribution • Repeated Trials: • Repeat the same random experiment n times. (Experiments are independent of each other) • Number of times an event A happens among n trials has Binomial distribution • (e.g., number of heads in n coin tosses, number of arrivals in n time slots,…) • Binomial is sum of n IID Bernoulli rvs
Mean of Binomial • Note that:
Binomial - Example n=4 p=0.2 n=10 n=20 n=40
Binomial – Example (ball-bin) • There are B bins, n balls are randomly dropped into bins. • : Probability that a ball goes to bin i • : Number of balls in bin i after n drops
Multinomial Distribution • Generalization of Binomial • Repeated Trails (we are interested in more than just one event A) • A partition of W into A1,A2,…,Al • Xi shows the number of times Ai occurs among n trials.
Poisson Distribution • Used to model number of arrivals
Poisson Graphs l=.5 l=1 l=4 l=10
Poisson as limit of Binomial • Poisson is the limit of Binomial(n,p) as • Let
Poisson and Binomial n=5,p=4/5 Poisson(4) n=10,p=.4 n=20, p=.2 n=50,p=.08
Geometric Distribution • Repeated Trials: Number of trials till some event occurs
Exponential Distribution • Continuous random variable • Models lifetime, inter-arrivals,…
Minimum of Independent Exponential rvs • : Independent Exponentials
Memoryless property • True for Geometric and Exponential Dist.: • The coin does not remember that it came up tails l times • Root cause of Markov Property.
Characteristic Function • Moment Generating Function (MGF) • For continuous rvs (similar to Laplace transform) • For Discrete rvs (similar to Z-transform):
Characteristic Function • Can be used to compute mean or higher moments: • If X and Y are independent and T=X+Y
Useful CFs • Bernoulli(p) : • Binomial(n,p) : • Multinomial: • Poisson: