Hypothesis Testing

Understanding Hypothesis Testing

• Hypothesis testing involves making assumptions about a population, based on sample data.
• The initial assumption made is known as the Null Hypothesis (H0).
• An alternative to this initial assumption is called the Alternative Hypothesis (H1).
• These hypotheses are mutually exclusive meaning if one is true, the other must be false.

Steps in Hypothesis Testing

• Step 1: Define the Null Hypothesis (H0) and Alternative Hypothesis (H1).
• Step 2: Choose a significance level (commonly 0.05), which determines the probability of rejecting the null hypothesis if it is true.
• Step 3: Collect and summarise the sample data through a test statistic.
• Step 4: Calculate the probability of the observed test statistic under the null hypothesis (the p-value).
• Step 5: Compare the p-value to the significance level to decide whether to reject the null hypothesis.

Types of Hypotheses

• The Null Hypothesis (H0) states that there is no effect or difference – this is the hypothesis that the test seeks evidence against.
• The Alternative Hypothesis (H1) is the statement of what a hypothesis test is seeking to prove.

Types of Errors in Hypothesis Testing

• A Type I error occurs when the null hypothesis is true, but is falsely rejected. It is equivalent to a false positive finding.
• A Type II error is the opposite, occurring when an incorrect null hypothesis is not rejected. It equates to a false negative finding.

One-tailed and Two-tailed Tests

• A one-tailed (or one-sided) test is used when the direction of the effect being tested is known (greater than or less than).
• A two-tailed (or two-sided) test is used when the direction of the effect is not known or is not relevant to the hypothesis.

Understanding p-values

• The p-value is the probability, under the null hypothesis, of obtaining a result equal to or more extreme than what was actually observed.
• If the p-value is small (typically ≤ 0.05), it is possible to reject the null hypothesis in favour of the alternative hypothesis.
• The p-value alone does not determine the significance of a test result. Other factors, such as the science being tested and the consequences of a wrong conclusion, should be considered.

Power and Sample Size

• Statistical power is the probability of correctly rejecting a false null hypothesis. It’s affected by the sample size, variance, significance level, and effect size.
• An appropriately large sample size increases the power of a statistical test.