1 / 65

Probability Theory: Dealing with Uncertainties

Learn the mathematical framework for handling uncertainties, including probabilistic models, random variables, distributions, expectation, variance, and special random variables.

morrillr
Download Presentation

Probability Theory: Dealing with Uncertainties

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Probability theory provides a mathematical framework for dealing with uncertainties. Appendix A: Probability Theory Many processes inevitably involves uncertainties, which may result from error, noise, imprecision, incompletion, distortion, etc.. A.1 Probabilistic Models -- A probabilistic model is a mathematical description of a process under an uncertain situation (i.e., random process). 1

  2. Elements of probability model Random process : whose outcome o is not predictable Sample space S: the set of all possible outcomes Event E: any subset of S Probability law P : the collective probability of the outcomes of an event,e.g. 2

  3. Probability :frequency of occurring o, or degree of belief in o (e.g., war) Example: A coin is tossed 3 times. Let the probability of a head on an individual toss be p. Sample space: Probability Law:

  4. Properties of probability laws: Let A, B, C: events

  5. Axioms of probability Conditional Probability The proportion of F in E 5

  6. Bayes’ Formula • Total Probability Theorem: Partition: 6

  7. Bayes’ formula becomes Multiplication Rule: 7

  8. 8

  9. “Disjoint” simplifies “Independent” simplifies 9 9

  10. Example: Two successive rolls of a 4-sided die in which all 16 possible outcomes have probability 1/16. 10 10

  11. ((1,2), (2,1), (2,2)) ((4,2), (2,4), (3,2), (2,3), (2,2)) 11 11

  12. A.2 Random Variables -- A random variable is a function, which assigns to each outcome a number that may represent a label or a quantity. • Example (label): Coin tossing Define random variable • Example (quantity): A coin is tossed 3 times (head: +$1; tail: -$1) Define random variable X as the net earned. 12

  13. A.2.1 Cumulative Distribution Function (CDF), Probability Density Function (PDF) and Probability Mass Function (PMF) The cumulative distribution function (CDF)F of a random variable X for a real number a is Continuous: probability density function (PDF) Discrete: probability mass function (PMF) 13

  14. Example: A coin is tossed 3 times. PMF: 14 14

  15. CDF: 15 15

  16. A.2.2 Joint CDF and Joint PDF -- involving multiple random variables Example: A coin is tossed 3 times. RVs: X (head: +$1; tail: -$1); Y (head: +$2; tail: $0) 16 16

  17. Joint CDF: : joint PDF Marginal CDF: Marginal PDF: • Marginal functions are to eliminate the influences of some random variables. 17

  18. Conditional distribution: Given joint PDF , the marginal PDF can be calculated and in turn the conditional distribution can be computed. Multiplication rule: 18

  19. Proof: 19 19

  20. Bayes’ rule: Proof: • If X and Y are independent, 20

  21. * What is the relationship between events and random variables (rv)? -- rvs are a special case of events, i.e., events containing only one outcome (singleton events). * What is the difference between ordinary variables (ov) and random variables (rv)? -- rv’s are associated with probability distributions while ovs are not. 21

  22. A.2.5 Expectation Expectation: Example: 22 22

  23. Properties of expectation: Proof: (Assignment) 23 23

  24. A.2.6 Variance Variance: : standard deviation • Properties of variance: 24 24

  25. Covariance: 25 25

  26. Properties of covariance: 26 26

  27. (Assignment) vi) If are independent, the covariance of any two rv’s is zero, Correlation: 27 27

  28. A.3 Special Random Variables A.3.1 Discrete Distributions Bernoulli distribution • Random variable X takes 0/1, e.g., coin tossing. Let p be the probability that X = 1. • Bernoulli probability: 28 28

  29. Binomial distribution -- N identical independent Bernoulli trials • Random variable X represents the number of 1s. • Binomial probability: Multinomial distribution -- N iid trials, each takes one of K states 29 29

  30. Multinomial probability: 30 30

  31. Geometric distribution -- rv X represents #Bernoulli tosses needed for a head to come up for the first time Geometric probability: Poisson distribution -- rv X represents, e.g., i) #typos in a book with a total of n words, ii) #cars involved in accidents in a city Poisson probability: 31 31

  32. A.3.2 Continuous Distributions Uniform distribution: -- X takes values equably over the interval [a, b] Uniform density: 32 32

  33. 33

  34. Normal (Gaussian) distribution: Normal density: 34 34

  35. Exponential distribution: Exponential density: Rayleigh distribution: Rayleigh density: 35

  36. A.3.3 Hybrid Distributions Theorem 1:Let f be a differentiable strictly increasing or strictly decreasing function defined on I. Let X be a continuous random variable having density . Let having density . Then, Proof: Let the CDFs ofrandom variables X and Y. 36

  37. (a) f strictly increasing 34

  38. (b) f strictly decreasing 34

  39. Example:Z-normalization Proof: 39 39

  40. 40 40

  41. Theorem 2: Let be independent random variables having the respective normal densities . Then, has the normal density Proof: Assume that , then Since are independent, 41

  42. 42

  43. Example: 43

  44. Theorem 3: Central Limit Theorem Example: 44

  45. Chi-square distribution Chi-square density: Properties: 45 45

  46. Gamma density: Chi-square density 46 46

  47. Erlang density: Beta density: Dirichlet density: 47 47

  48. t density: 48 48

  49. F density: 49 49

  50. Summary • Probability Model • Properties of probability laws: 50

More Related