Probability

1. Probability • Sample Space (S) • Collection of all possible outcomes of a random experiment • Sample Point • Each outcome of the experiment (or) element in the sample space • Events are Collection of sample points • Ex: Rolling a die (six sample points), Odd number thrown in a die (three sample point – a subset), tossing a coin (two sample points: head,tail) Review of Random Process

2. Probability • Null Event (No Sample Point) • Union (of A and B) • Event which contains all points in A and B • Intersection (of A and B) • Event that contains points common to A and B • Law of Large Numbers N – number of times the random experiment is repeated NA- number of times event A occurred Review of Random Process

3. Probability • Properties Review of Random Process

4. Probability • Conditional Probability • Probability of B conditioned by the fact that A has occurred • The two events are statistically independent if Review of Random Process

5. Probability • Bernoulli’s Trials • Same experiment repeated n times to find the probability of a particular event occurring exactly k times Review of Random Process

6. Random Signals • Associated with certain amount of uncertainty and unpredictability. Higher the uncertainty about a signal, higher the information content. • For example, temperature or rainfall in a city • thermal noise • Information is quantified statistically (in terms of average (mean), variance, etc.) • Generation • Toss a coin 6 times and count the number of heads • x(n) is the signal whose value is the number of heads on the nth trial Review of Random Process

7. Random Signals • Mean • Median: Middle or most central item in an ordered set of numbers • Mode = Max{xi} • Variance • Standard Deviation measure of spread or deviation from the mean Review of Random Process

8. Random Variables • Probability is a numerical measure of the outcome of the random experiment • Random variable is a numerical description of the outcome of a random experiment, i.e., arbitrarily assigned real numbers to events or sample points • Can be discrete or continuous • For example: head is assigned +1 tail is assigned –1 or 0 Review of Random Process

9. Random Variables • Cumulative Distribution Function (CDF) • Properties: • Probability Density Function (PDF) • Properties: Review of Random Process

10. Important Distributions • Binary distribution (Bernoulli distribution) • Random variable has a binary distribution • Partitions the sample space into two distinct subsets A and B • All elements in A are mapped into one number say +1 and B to another number say 0. Review of Random Process

11. Important Distributions • Binomial Distribution • Perform binary experiment n times with outcome X1,X2,…Xn, if , then X has binomial distribution Review of Random Process

12. a b Important Distributions • Uniform Distribution • Random variable is equally likely • Equally Weighted pdf Review of Random Process

13. Important Distributions • Poisson Distribution • Random Variable is Poisson distributed with parameter m with • Approximation to binomial with p << 1, and k << 1, then Review of Random Process

14. Important Distributions • Gaussian Distribution • Normalized Gaussian pdf - N(0,1) • Zero mean, Unit Variance Review of Random Process

15. Important Distributions • Normalized Gaussian pdf Review of Random Process

16. Joint and Conditional PDFs • For two random variables X and Y Review of Random Process

17. Joint and Conditional PDFs • Marginal pdfs • Conditional pdfs Review of Random Process

18. Expectation and Moments • Centralized Moment • Second centralized moment is variance Review of Random Process

19. Expectations and Moments • (i,j) joint moment between random variables X and Y Review of Random Process

20. Expectations and Moments • (i,j) joint central moment Review of Random Process

21. Expectations and Moments • Auto-covariance • Characteristic Function (moment generator) Review of Random Process

22. Random Process • If a random variable X is a function of another variable, say time t, x(t) is called random process • Collection of all possible waveforms is called the ensemble • Individual waveform is called a sample function • Outcome of a random experiment is a sample function for random process instead of a single value in the case of random variable Review of Random Process

23. Random Process • Random Process X(.,.) is a function of time variable t and sample point variable s • Each sample point (s) identifies a function of time X(.,s) referred as “sample function” • Each time point (t) identifies a function of sample points X(t,.), i.e., a random variable • Random or Stochastic Processes can be • continuous or discrete time process • continuous or discrete amplitude process Review of Random Process

24. Random Process • Ensemble statistic : Ensemble average at a particular time • Temporal average for a sample function • Random Process Classifications • Stationary Process : Statistical characteristics of the sample function do not change with time (time-invariant) Review of Random Process

25. Random Process • Second Order joint pdf • Autocorrelation is a function of only time difference • Wide Sense (or Weak) Stationary • Independent of time up to second order only • Ergodic Process • Ensemble average = time average Review of Random Process

26. Random Process • Mean • Mean of the random process at time t is the mean of the random variable X(t) • Autocorrelation • Auto-covariance Review of Random Process

27. Random Process • Cross Correlation and covariance • Power Density Spectrum Review of Random Process

28. Random Process • Total Average Power Review of Random Process