Hypothesis Testing

Understanding Hypothesis Testing

  • Hypothesis testing involves making assumptions about a population, based on sample data.
  • The initial assumption made is known as the Null Hypothesis (H0).
  • An alternative to this initial assumption is called the Alternative Hypothesis (H1).
  • These hypotheses are mutually exclusive meaning if one is true, the other must be false.

Steps in Hypothesis Testing

  • Step 1: Define the Null Hypothesis (H0) and Alternative Hypothesis (H1).
  • Step 2: Choose a significance level (commonly 0.05), which determines the probability of rejecting the null hypothesis if it is true.
  • Step 3: Collect and summarise the sample data through a test statistic.
  • Step 4: Calculate the probability of the observed test statistic under the null hypothesis (the p-value).
  • Step 5: Compare the p-value to the significance level to decide whether to reject the null hypothesis.

Types of Hypotheses

  • The Null Hypothesis (H0) states that there is no effect or difference – this is the hypothesis that the test seeks evidence against.
  • The Alternative Hypothesis (H1) is the statement of what a hypothesis test is seeking to prove.

Types of Errors in Hypothesis Testing

  • A Type I error occurs when the null hypothesis is true, but is falsely rejected. It is equivalent to a false positive finding.
  • A Type II error is the opposite, occurring when an incorrect null hypothesis is not rejected. It equates to a false negative finding.

One-tailed and Two-tailed Tests

  • A one-tailed (or one-sided) test is used when the direction of the effect being tested is known (greater than or less than).
  • A two-tailed (or two-sided) test is used when the direction of the effect is not known or is not relevant to the hypothesis.

Understanding p-values

  • The p-value is the probability, under the null hypothesis, of obtaining a result equal to or more extreme than what was actually observed.
  • If the p-value is small (typically ≤ 0.05), it is possible to reject the null hypothesis in favour of the alternative hypothesis.
  • The p-value alone does not determine the significance of a test result. Other factors, such as the science being tested and the consequences of a wrong conclusion, should be considered.

Power and Sample Size

  • Statistical power is the probability of correctly rejecting a false null hypothesis. It’s affected by the sample size, variance, significance level, and effect size.
  • An appropriately large sample size increases the power of a statistical test.