Testing programs

Testing is the process of running a program with carefully chosen data to verify it works correctly and find errors. OCR J277 Paper 2 tests three types of test data and two approaches to testing — you must be able to choose and justify test data for a given scenario.

Why is testing necessary?

• Programs must work correctly for all valid inputs and handle invalid inputs gracefully.
• Errors (bugs) must be found and fixed before software is deployed.
• Untested software in safety-critical systems (aircraft, medical devices, banking) can cause injury or financial loss.
• Testing cannot prove a program is completely error-free — it can only demonstrate that tested cases work.

Two approaches to testing

Iterative testing

• Testing is performed throughout development, after each module or feature is completed.
• Errors are found and fixed early — when they are cheapest to correct.
• Each iteration tests new code AND regression-tests previous code to ensure changes didn't break anything.
• Advantage: bugs found early; easier to isolate which change caused a bug; continuous feedback.

Final (terminal) testing

• Testing is performed after the entire program is complete.
• The finished product is tested against requirements.
• Advantage: tests the system as a whole (integration testing).
• Disadvantage: bugs found late are expensive to fix; may require redesign; deadlines may limit thorough testing.

Three types of test data

1. Normal (valid) data

• Data that is within the expected valid range.
• The program should accept and process this data correctly.
• Example: for an age input (1–120): test data = 25, 50, 100.

2. Boundary data

• Data at the extreme edges of the valid range.
• Must test the boundary itself AND just outside it (both sides).
• The program should accept data at the boundary but reject data just beyond it.
• Example: for age 1–120:
  • Boundary accepted: 1, 120.
  • Boundary rejected: 0, 121.

Why boundary data? Programs often have "off-by-one" errors (e.g. > 120 instead of >= 120). Boundary data specifically catches these.

3. Invalid / erroneous data

• Data that is outside the valid range, wrong type, or in the wrong format.
• The program should reject this data (not crash, not accept silently).
• Example for age 1–120:
  • Out of range: -5, 0, 999, 200.
  • Wrong type: "hello", "abc".
  • Empty input: "" (blank).

Summary table

Type	Example (age 1–120)	Expected outcome
Normal	25, 70	Accepted, processed
Boundary	1, 120, 0, 121	1 and 120 accepted; 0 and 121 rejected
Invalid/erroneous	-5, 999, "hello"	Rejected with error message

Trace tables

A trace table is used to manually trace the execution of an algorithm step by step, tracking variable values at each stage.

• Used to verify algorithm logic before coding.
• Used to find where a program produces incorrect output.

Step	i	total	Output
Start	-	0
i=0	0	5
i=1	1	12
i=2	2	18	18

Refining solutions

After testing, bugs are fixed (debugging), and the program may need refinement to:

• Handle edge cases not originally considered.
• Improve error messages.
• Optimise performance.
• Improve code readability.

Common OCR exam mistakes

1. Confusing boundary with invalid data — boundary data IS valid (at the edge); invalid data is OUTSIDE the range.
2. Only giving one boundary value — always test BOTH edges of the range AND the values just outside.
3. Saying "testing proves the program is correct" — testing can only show the program works for the cases tested; it cannot prove correctness for all possible inputs.
4. Confusing iterative testing with iteration (loops) — iterative testing means testing throughout development; it has nothing to do with programming loops.