Probability

Key Probability Concepts

  • Probability: A mathematical method of quantifying uncertainty. It assigns numerical values, typically between 0 and 1, to uncertain events.
  • Random Experiment: An action or process that leads to one of several possible outcomes, where the result is uncertain until it happens.
  • Sample Space, often denoted by S: The set of all possible outcomes of a random experiment.
  • Event: A subset of the sample space. It represents one or several outcomes of the experiment.

Probability Rules/Principles

  • The Addition Rule: If events A and B are disjoint (i.e., they can’t both occur simultaneously), then the probability of A or B occurring is P(A)+P(B).
  • The Principle of Inclusion and Exclusion: If events A and B can both occur simultaneously, then the probability of A or B occurring is P(A)+P(B)-P(A∩B).
  • The Multiplication Rule: For any two events A and B, the probability that both A and B occur is given by P(A∩B)=P(A)P(B A), where P(B A) is the probability of event B given event A has occurred.

Types of Events

  • Independent Events: Two events, A and B are independent if the occurrence of A does not affect the probability of B, and vice versa.
  • Mutually Exclusive Events (Disjoint Events): A and B are mutually exclusive if they cannot happen at the same time.
  • Compound Events: Compound events refers to the combination of two or more simple events.

Conditional Probability

  • Conditional Probability, denoted by P(B A): The probability of event B given that event A has occurred.
  • Theorem of Total Probability: If A1, A2,…, An are mutually exclusive events that form a partition of the sample space, then for any other event B, P(B)=Σ[P(B Ai)P(Ai)] for all i.
  • Bayes’ Theorem: A fundamental theorem that calculates the conditional probability of an event based on prior knowledge of conditions that might be related to the event.

Expectation and Variance

  • Expected Value (E(X)): The long-term average or mean value of random variables.
  • Variance (Var(X)): A measure of the spread of random variables around its expected value.
  • Standard Deviation: The square root of the variance. Giving a measure of dispersion of the distribution in the units of the random variable.

Remember, understanding and mastering probability forms a foundation of many other areas such as Statistics, Machine Learning and Inference. Retain these concepts clearly to set solid footing.