Every security team I know produces one number the board loves and the program should be a little embarrassed by: the annual training completion rate. Ninety-something percent, green, delivered on time. It is the cleanest figure in the deck, and it is measuring the wrong thing. Completion tells you people clicked through a module. It tells you nothing about whether anyone behaves differently on the Tuesday afternoon a spoofed vendor emails accounts payable to "update our wire instructions before the two o'clock cutoff."
The tell is that the incidents keep arriving from the same places. The same team falls for the same style of lure. The same engineers push the same kind of secret into the same repositories. The same handful of executives get spearphished, because they are worth spearphishing. A 99 percent completion rate sits on top of all of it, unchanged, because it was never connected to any of it. We ran awareness. We did not move risk.
So here is the reframe I have been building toward, and it is a change of category, not a change of vendor. Stop running security awareness as an annual campaign and start running human risk as a program — measured, scored, segmented by team, with interventions matched to where the loss actually concentrates. Gartner has been pushing the industry toward the same shift under the label Security Behavior and Culture Program, and the renaming matters, because it moves the objective from attendance to behavior. In a regulated business, that distinction is the entire game.
Human risk is a portfolio, not a headcount
Awareness treats the workforce as one undifferentiated population, trained on one schedule with one curriculum. But loss exposure is not distributed evenly across a headcount, and pretending it is guarantees you spend the most effort where it matters least. The FAIR model — Factor Analysis of Information Risk, the one quantitative loss-exposure framework most auditors will actually recognize — decomposes risk into the frequency of a loss event and the magnitude when it lands. Apply that lens to people and the uniform-population assumption falls apart immediately. Both terms vary wildly by role.
The person in accounts payable with authority to change wire instructions carries a different loss magnitude than a warehouse associate, full stop. An engineer with standing production access and a key to the secrets manager carries a different frequency-times-magnitude than a recruiter. Treating those three people as one training cohort is not fair, egalitarian, or thorough. It is imprecise, and imprecision in a risk portfolio is just money and attention spent in the wrong column. You would never manage a control portfolio this way — flat coverage, no weighting by exposure — yet that is precisely how the standard awareness program treats the human layer.
I already have this instinct for machines. When we built AgentOS, our governed internal agent platform, scoring non-human identities by their blast radius was table stakes — an agent that can move money is not the same risk object as one that summarizes a wiki, and it does not get the same controls. The human layer deserves the identical discipline. Score the exposure per team first. Everything downstream depends on getting that segmentation honest.
Score the behavior, not the completion
Once you accept that the goal is behavior change, you have to measure behavior, which means retiring completion as your headline metric and instrumenting the signals that actually precede loss. Most of these you already collect; you have simply never assembled them into a human-risk score. The SANS Security Awareness Maturity Model puts measurable behavior and culture change at the top of its ladder for exactly this reason — the mature program is the one that can prove behavior moved, not that seats were filled.
The signals worth scoring, in rough order of how much they tell you:
- Reporting rate and reporting speed. The most important culture metric is not whether people click the bad link — some always will — but whether they report it, and how fast. A workforce that reports a live phish in four minutes is a sensor network. One that stays quiet out of embarrassment is blind spots wearing lanyards. Measure the report, and measure the dwell time before it.
- Real-world susceptibility as a trend, not a gotcha. Simulation click and credential-submit rates matter, but only as a trend line per segment over time. Run them as a scored campaign against named people and you buy short-term fear and long-term evasion. Run them as an anonymized trend and you get a behavior curve you can actually manage.
- Credential and factor hygiene. Adoption of phishing-resistant authentication, reuse flagged by tooling, dormant privileged sessions. These are behaviors, they already sit in your logs, and they move when you intervene.
- Sensitive-data movement. The DLP and CASB signals you already generate — risky external sends, unsanctioned SaaS, data leaving into personal accounts — are behavior telemetry, not just alerts to chase.
- Privileged-action hygiene. For the engineering population specifically: secrets committed, production changes made outside change control, standing access that should have been just-in-time. This is where a single behavior becomes a very large magnitude very quickly.
None of these is a survey. All of them are things people actually did, which is the only evidence that awareness became behavior.
Segment the way the attacker already has
Here is the uncomfortable part: the adversary segmented your workforce before you did. Proofpoint popularized the term Very Attacked People for a reason — targeting in the real world is not uniform, it is concentrated on the individuals whose access or authority makes them worth the effort. If the attacker is running a targeted portfolio and you are running a flat curriculum, you have conceded the initiative before the first email lands. The program's job is to find the same high-exposure clusters the attacker is already working, and to get there first.
In a fintech the clusters are predictable, and I would map them roughly like this:
- Payments, AP, and treasury. Anyone who can originate, approve, or redirect a payment. This is where business email compromise concentrates, and where a single successful lure clears the largest dollar amount.
- Engineering and platform. Standing production access, credentials, and the ability to change what runs. High frequency of exposure, high magnitude, and the loss often looks like a leaked secret rather than a clicked link.
- Executives and their assistants. Authority plus a public profile plus a calendar full of plausible urgency. The assistant is frequently the actual target, and almost never the person the awareness program was built around.
- Customer servicing and support. Account access at scale, plus a service culture that rewards saying yes. Social engineering aims straight here, because the whole team is trained to be helpful under pressure.
- The recently onboarded and recently reorganized. No muscle memory, unclear reporting lines, and a strong incentive not to question a message that appears to come from a new boss they have not met.
Segment first. The interventions only make sense once you know which portfolio you are managing.
Interventions are a menu, not a mandate
The awareness era's cardinal error was believing the intervention is always "more training." Most of the time it is not. Once the population is segmented, the right response to each segment is usually specific, and often it is engineering rather than education.
For payments and treasury, change the procedure, not the person. The durable control against a redirected-wire lure is not a smarter employee; it is callback verification against a number of record and a dual-control threshold no single person can clear alone. Drill the procedure until it is reflexive. The behavior you are building is "verify out of band," and it should be cheaper to follow than to skip.
For engineers, pave the road. Nobody commits a secret because they skipped a module. They commit it because the .env file was the path of least resistance at 6 p.m. The intervention is a pre-commit hook, a secrets manager that is genuinely easier than the shortcut, and a paved path where the secure choice is the default choice. Culture follows friction; reduce the friction on the safe route.
For executives, run concierge. High-exposure, low-availability people do not belong in the general curriculum. They get a short, direct, personalized briefing on the specific ways their authority is being impersonated, and their assistants get the same one — because the assistant is the real perimeter.
For everyone, make reporting the product. The highest-leverage cultural investment is a report button that is one click, a response that thanks rather than scolds, and a visible signal that reports lead somewhere. A workforce that reports is a control. Awareness content is how you get there; the report is what you are actually buying.
Report the trend, not the attendance
The compliance floor is real and I am not waving it away. FFIEC examination guidance and the GLBA Safeguards Rule both require security awareness training, SOC 2 expects it, and at a company serving 1,500+ financial institutions the examiner will ask for the completion evidence. Produce it. But produce it as the floor it is, and do not confuse satisfying the requirement with reducing the risk. The completion report answers "did you train them." The program answers "did their behavior change," and only the second one shows up in a loss curve.
Standing the program up is a sequence, not a purchase:
- Inventory human-risk exposure by team, using the FAIR decomposition — frequency and magnitude — so the segmentation is defensible rather than intuitive.
- Instrument the behavior signals you already emit: reporting, susceptibility, credential hygiene, data movement, privileged action. Assemble them into a per-segment score.
- Baseline every segment, then rank them by exposure so effort follows loss rather than headcount.
- Target interventions to the segment — procedure, paved road, concierge, or nudge — and change one variable at a time so you can attribute the movement.
- Re-measure on a fixed cadence and report the delta, per segment, as risk reduced.
When I take this to the board, the slide is not a completion percentage. It is a set of trend lines: reporting rate up and to the right, dwell time before a report falling, high-exposure-segment susceptibility declining quarter over quarter. That is a picture of risk being managed, and it is the only version of this story a director should accept — because it is the only version tied to loss instead of attendance.
Run the program, not the campaign
Score exposure by team, not by headcount. Measure behavior, not completion. Match the intervention to the segment the attacker already picked. And report the trend, because the trend is the only part that was ever about risk.
I want to hear which behavior signal actually earned its place for you — the one that moved before a real incident, or the one you scored for a year and then cut because it predicted nothing. That gap, between the metrics that feel rigorous and the ones that actually forecast loss, is where this work gets hard. Tell me in the comments.
