How to measure a security awareness programme

Keep the measurement question narrow

Ask whether people are practising the intended action, whether reports reach the right team and whether follow-up happens. A completed course is evidence of participation, not proof that incidents cannot occur.

Avoid combining several percentages into a risk score unless you can explain the model, inputs and limitations. Keep operational measurements visible alongside any summary score.

Use a consistent definition for each measure

The definitions below are a suggested reporting template. Confirm how your own tools count delivery, interaction and reporting before comparing their exports.

• Completion rate: people who completed the assigned activity divided by people assigned, for the same reporting window.
• Simulation interaction rate: unique recipients who performed the measured action divided by successfully delivered simulation messages, with one message per recipient in that exercise.
• Simulation report rate: unique recipients who reported the simulation divided by successfully delivered messages in the same exercise.
• Time to report: elapsed time from delivery to the first report for each reporting recipient. State the median and the number of reporters; non-reporters are excluded.
• Follow-up completion: agreed follow-up actions completed divided by actions due in the period.

Show what changed between exercises

Record the audience, language, scenario, difficulty and time window. NIST’s Phish Scale provides a method for assessing human phishing detection difficulty. These factors help explain why click rates vary.

Exclude automated mail-security interactions where your tools can identify them, and document any uncertainty. Do not present a change in simulation results as a measured reduction in real-world incidents.

Use a short review record

For each reporting period, record the metric definition, numerator, denominator, exclusions, comparison period and owner. Add one sentence explaining the next action.

For example, a hypothetical exercise with 100 delivered messages and 12 unique reporters has a 12% report rate. This example is arithmetic, not a Moxso benchmark or customer result.