How to Measure Security Culture: Beyond Completion Rates

Most security awareness programs report a single number to leadership: percentage of employees who completed the assigned training module. It is easy to collect, easy to chart, and almost entirely disconnected from whether anyone behaves differently the next time a convincing email lands in their inbox. A 98 percent completion rate says employees clicked through slides or watched a video; it says nothing about whether they can spot a spoofed OJK circular or a fake HR salary notification. Regulators and boards are increasingly aware of this gap, and conflating completion with capability is starting to draw scrutiny rather than reassurance.

The problem compounds when completion becomes the only KPI tied to a program's success, because it creates an incentive to optimize for compliance rather than behavior change. Training teams push reminders until the number goes up, employees learn to click through as fast as possible, and the actual skill (recognizing manipulation, verifying senders, reporting suspicious messages) never gets measured at all. This is the training equivalent of teaching to the test: the metric improves while the underlying risk stays flat or worsens.

A mature security culture program treats completion as a hygiene floor, not a headline number. It is necessary, since regulators including OJK expect documented training records, but it is the starting point for measurement, not the destination. The rest of this article covers the metrics that actually correlate with reduced organizational risk, and how to turn them into something a board or an OJK examiner can trust.

Reporting rate, the percentage of phishing simulations and real phishing attempts that employees report through the correct channel, is the single strongest proxy for a healthy security culture. It captures something completion rate cannot: whether employees are actively participating in the organization's defense rather than passively avoiding punishment. A rising reporting rate, especially one that rises faster than the click rate falls, signals that people are internalizing the behavior you actually want: pause, verify, report.

Repeat-clicker reduction matters more than aggregate click rate, because a small group of habitual clickers usually drives a disproportionate share of organizational risk. Track the percentage of employees who click on two or more simulations within a rolling twelve-month window, and watch that cohort shrink over time as targeted interventions take effect. Aggregate click rate can look good while a repeat-clicker population remains stubbornly present and dangerous; segmenting the data is what reveals the truth.

Dwell time, the interval between a phishing email landing and the first click or report, tells you how quickly employees are processing suspicious signals. A shortening dwell time to report, meaning people act fast to flag something, paired with a lengthening dwell time to click, meaning people hesitate before engaging, shows genuine behavioral caution developing, not just awareness of the concept. None of these numbers mean much as a single snapshot; the value is in the trend line across quarters, correlated against campaign difficulty and department.

Behavioral data tells you what people do; a validated security culture survey tells you why, and closes the gap between what leadership assumes and what employees actually believe. A short, anonymous instrument covering perceived responsibility, confidence in reporting without repercussion, and awareness of policy, run twice a year, surfaces problems that click-through data alone will miss, such as a department that avoids reporting because they fear it reflects poorly on them.

Keep the survey short, ten to fifteen items, keep it anonymous at the individual level but segmentable by department and tenure, and repeat it on a fixed cadence so trends are comparable. In the Indonesian context, run it in Bahasa Indonesia as the default language with English available, since a survey administered only in English will systematically under-report concerns from frontline branch staff who are not comfortable expressing nuance in a second language.

Cross-reference survey sentiment against behavioral metrics quarterly. A department that self-reports high confidence in reporting phishing but shows a low actual reporting rate has a friction problem, not an awareness problem, likely an unclear reporting channel or a fear that reporting will be logged against them personally. That is a very different fix than a department that is genuinely unaware simulations exist.

The most useful benchmark is your own organization over time, segmented by department, role seniority, tenure, and branch or head office location. A bank's teller population in a regional branch faces a different attack surface and a different baseline risk tolerance than head-office finance staff who routinely handle wire transfer approvals, and treating them as one undifferentiated population hides where the real risk concentrates.

External benchmarking is harder in Indonesia than in mature markets because sector-wide phishing metrics are rarely published, but directional context still matters: OJK's expectations under POJK 11/2022 for financial sector cybersecurity resilience assume an organization can demonstrate ongoing risk-based awareness activity, not a single annual event. Framing your internal trend data against that expectation, rather than against an external number that may not exist, is usually the more defensible approach in an examination.

Where sector data does exist, from industry associations, BSSN advisories, or aggregated platform data across a vendor's customer base, use it as a sanity check rather than a target. The goal is not to be average; it is to show a credible, improving trend that a board member or examiner can look at and understand within thirty seconds, without needing a data science background to interpret it.

Consolidate reporting rate, repeat-clicker population, dwell time trend, and survey sentiment into a single quarterly scorecard with a short narrative, not just a dashboard screenshot. Boards and OJK examiners respond to a clear story: what changed, why, and what you are doing about the parts that have not improved, more than they respond to a wall of charts.

Claro's analytics and risk scoring modules are built around this principle: campaign results, reporting-rate trends, and repeat-clicker segmentation roll up automatically into the same views used for compliance reporting, so the metrics that satisfy a board conversation are the same ones that satisfy an OJK POJK 11/2022 evidence request. That consistency matters because a security culture story that changes shape depending on the audience undermines its own credibility.

Review the scorecard at a fixed cadence, quarterly at minimum, and treat a flat or worsening trend on any single metric as a trigger for a targeted intervention rather than a footnote. A security culture program that only reports good news is not measuring hard enough; the value of this scorecard is precisely that it will occasionally show you something uncomfortable early enough to act on it.