Lesson 12 of 20 · 6 min
Ten thousand runs is not a prediction
A simulation reports the consequences of its assumptions with great precision and no additional knowledge.
Running more paths makes a simulation's output more stable. It does not make it more true. Those are separate properties and conflating them is the central error in how simulation results get used.
What more runs buys is reduction of sampling noise. With 100 runs the 5th percentile jumps around between repetitions; with 10,000 it is steady to within a percent or so. That is a real improvement in the precision of the estimate of what the assumptions imply.
What more runs cannot buy is any correction to the assumptions. If the volatility input is 20 percent and the true figure is 30, ten million runs will produce an exquisitely precise answer to the wrong question. If the random-number generator draws from a normal distribution, no number of runs will produce the fat tails that lesson 3 documented — the extreme paths simply are not in the pool being sampled from.
The sensitivity table is the honest way to present simulation output for this reason. In the standard setup, the probability of touching the trigger is 4.6 percent at 15 percent volatility, 14.3 percent at 20 percent, and 49.6 percent at 35 percent. The assumption moved by twenty points and the answer moved by a factor of eleven. Reporting only the middle figure hides the fact that the input dominates the output.
Report a range across assumptions, not a number from one. The spread across inputs is almost always larger than the noise across runs.
Key terms
- Sampling noise
- Variation in results from run to run purely because a finite number of paths was drawn. More runs reduce it.
- Assumption error
- Error from the inputs being wrong. Unaffected by the number of runs, and usually far larger.
- Sensitivity table
- Simulation output across a range of inputs rather than at one point. Shows which assumption is driving the answer.
Worked example
Precision against accuracy
Standard setup, varying only the volatility assumption. Probability of touching the trigger value.
- 15% volatility
- 4.6%
- 20% volatility
- 14.3%
- 35% volatility
- 49.6%
- Range of answers
- 4.6% to 49.6%
- Run-to-run noise at 10,000 runs
- Well under one point
- Which uncertainty dominates
- The assumption, by a very wide margin
An eleven-fold spread.
Sampling noise is under a point and assumption uncertainty spans forty-five. Adding runs addresses the smaller problem.
Check yourself
A simulation with 10,000 runs says the probability of touching your trigger is 14.3%. What does running 100,000 paths improve, and what does it not?
Saved in this browser only. No account, nothing sent anywhere.
Educational only. Nothing in this lesson is investment, legal or tax advice, or a recommendation to enter into any transaction. Figures are illustrative and computed from the inputs stated in the lesson.