Probability Testing: Practical Experience and Guidelines for Game QA

Game players often hear vague or mystical claims about luck and probability, which reflect a desire for favorable outcomes and the mysterious nature of random systems. As QA engineers, we must uncover the full picture of probabilities, verify them with facts, and provide solid evidence to ensure quality.

1. Importance of Boundary Values

Using a critical‑hit feature as an example, a typical Python implementation might look like:

if random.uniform(0, 1) <= critRate:
    do critical dmg
else:
    do dmg

Because numpy.random.uniform(0,1) returns values in the half‑open interval [0, 1) , a configuration with critRate == 0 could still trigger a critical hit when the random value is exactly 0, violating the expectation of a 0% chance. Reviewers must pay attention to the random function’s range and the comparison logic. During testing, strict boundary‑value cases (probability set to 0 and 1) should be exercised with extensive simulations to confirm no bugs appear.

2. Do Not Overlook Misleading Probability Placements

Players often complain about low‑yield activities, assuming the displayed probabilities are the true expectations. However, bugs or hidden mechanics can distort these values. QA should verify the actual value of fixed rewards (e.g., daily experience) and compare them against design expectations.

For random drops, the expected value must match the designer’s anchor. For example, a chest A that yields B with 50% probability and C with 50% probability should satisfy:

If the actual values of B and C do not align with the anchored value of chest A, the configuration is erroneous. In more complex economies, hidden probabilities may be adjusted to favor certain items, which can be detected by analyzing extreme cases.

3. How to Communicate Probabilities to Players

Unclear probability presentation can lead to public backlash. A notable incident involved a game that advertised a 1.9% drop rate for a rare item while also guaranteeing the item after 60 draws. Players assumed the 1.9% applied to each draw without the guarantee, leading to accusations of probability fraud. The developer later clarified the mechanics, but the damage to trust had already occurred.

Clear disclosure of when guarantees or pity systems apply is essential to avoid misleading expectations and potential reputation risks.

Summary Checklist

1. Understand the random strategy, its purpose, and expected outcomes.

2. Verify proper use of random seeds.

3. Ensure simulation results match statistical expectations (both distribution and aggregate).

4. Check reset mechanisms for pseudo‑random interventions.

5. View the feature from the player’s perspective to spot ambiguities or unreasonable designs.