Understanding Hypothesis Testing: Core Concepts, Steps, and Common Errors

1 Concept of Hypothesis Testing

Hypothesis testing (hypothesis testing), also called statistical hypothesis testing, is a statistical inference method used to determine whether differences between samples or between a sample and a population are caused by sampling error or by inherent differences.

Significance testing is the most commonly used method in hypothesis testing and a basic form of statistical inference. Its principle is to first make an assumption about a population characteristic, then use sampling inference to decide whether the hypothesis should be rejected or accepted. Common methods include Z‑test, t‑test, chi‑square test, and F‑test.

2 Basic Idea

The basic idea of hypothesis testing is the small‑probability event principle, a probabilistic form of proof by contradiction.

The small‑probability idea states that a small‑probability event essentially never occurs in a single trial. The proof‑by‑contradiction approach first proposes a null hypothesis H0, then uses appropriate statistical methods and the small‑probability principle to determine whether H0 holds. If the observed sample leads to a “small‑probability event”, H0 is rejected; otherwise, H0 is accepted.

In hypothesis testing, the so‑called “small‑probability event” is not an absolute logical contradiction but a practical principle: the event’s probability is so low that it is considered practically impossible in a single experiment. The smaller this probability (denoted α, 0 < α < 1), the stronger the evidence against the null hypothesis. This probability is called the significance level.

Different problems may use different significance levels; commonly used thresholds are 0.1, 0.05, or 0.01.

3 Test Statistic

A test statistic is a sample statistic calculated from the observed data that is used to make a decision about the null hypothesis and the alternative hypothesis.

The test statistic is essentially a point estimate of a population parameter, but it must be standardized before it can be used to measure the discrepancy between the estimated value and the hypothesized value. The standardization is based on (1) assuming the null hypothesis H0 is true and (2) the sampling distribution of the point estimate.

4 Alternative Hypothesis and Rejection Region

The rejection region (also called the denial region) is the set of values of the test statistic for which the null hypothesis H0 is rejected, determined by the chosen significance level α.

The complementary acceptance region contains the values for which H0 is not rejected. The boundary between the two regions is the critical value.

The size of the rejection region depends on the pre‑selected significance level; a smaller α yields a smaller rejection region and requires a more extreme test‑statistic value to reject H0.

The location of the rejection region depends on whether the test is two‑sided or one‑sided.

In a two‑sided test, the rejection region lies in both tails of the sampling distribution.

In a one‑sided test, if the alternative hypothesis is “<”, the rejection region is in the left tail (left‑side test); if the alternative hypothesis is “>”, the rejection region is in the right tail (right‑side test).

5 Basic Steps

1. Formulate the null hypothesis (H0) and the alternative hypothesis (H1 or Ha). H0: The observed difference is due to sampling error. H1: A genuine difference exists; the test level is usually set at 0.05, with α = 0.05 or 0.01.

2. Choose a statistical method (e.g., Z‑test, t‑test, rank‑sum test, chi‑square test) and compute the test statistic from the sample data.

3. Make a decision using one of two approaches: 3.1 P‑value method : Compare the P‑value with α. If P > α, do not reject H0; if P ≤ α, reject H0 and accept H1. 3.2 Critical‑value method : Compare the test statistic with the critical value(s). Two‑sided: reject if |statistic| > critical value. Left‑side: reject if statistic < ‑critical value. Right‑side: reject if statistic > critical value.

6 Two Types of Errors

In hypothesis testing, a Type I error occurs when a true null hypothesis is incorrectly rejected, while a Type II error occurs when a false null hypothesis is incorrectly accepted.

7 Conclusion

This article briefly introduced the basic concepts, ideas, and procedures of hypothesis testing, providing a foundation for applying specific tests such as large‑sample mean tests.