Why 'We Ran a Phishing Test' Is Not Evidence
Most Indonesian financial institutions and enterprises already run some form of phishing simulation and security awareness training. But when an OJK examiner, a UU PDP compliance reviewer, or an ISO 27001 lead auditor asks for evidence of an effective awareness program, a summary slide showing an 82% pass rate rarely satisfies them. Auditors want traceability: which specific employees were tested, on what date, with what simulated attack, what they did, and what happened next. Without that granularity, a program that is genuinely running well can still fail an audit simply because it cannot prove itself.
This gap is common because simulation platforms and training systems are often treated as operational tools, not evidentiary systems. Security teams optimize for click-through rate and completion percentage, the numbers that matter for internal reporting to management. Auditors, on the other hand, work backwards from a control statement, for example that the institution shall periodically test staff awareness of social engineering risks and remediate gaps, and expect a chain of evidence that ties a named individual, a dated event, and a documented remediation action together.
The fix is not more testing. It is treating every phishing simulation, every training assignment, and every reported email as a compliance artifact from the moment it is created, not retrofitted into a report two weeks before an audit. That means capturing tenant-level, user-level, and time-stamped data by design, and exporting it in a form an auditor who has never used Claro can still read and verify without help.
The Four Evidence Pillars Auditors Actually Ask For
In practice, a defensible awareness program evidence pack rests on four pillars. The first is simulation results: who received which simulated phishing, vishing, smishing, or quishing scenario, on what date, and what action they took, whether they opened, clicked, submitted credentials, or reported it. The second is training completion: who was assigned which module or learning path, when, whether they completed it, and their quiz score against the pass threshold.
The third pillar is behavioral signal beyond pass or fail: phish-reporting rate, the percentage of employees who actually report simulated and real phishing through the button or portal rather than simply failing to click, and JIT (just-in-time) awareness engagement, showing that a user who clicked was immediately shown corrective content and can be shown to have engaged with it. The fourth pillar is the risk score trend: an aggregated, defensible per-user and per-department risk score that moves over time, demonstrating the program changes behavior rather than just measuring it once.
Each pillar on its own is a metric. Together, tied to named individuals and dated events, they become a narrative an auditor can follow: this person clicked in March, was retrained within 48 hours, reported the next simulated attempt in April, and their individual risk score dropped from high to medium by June. That narrative, repeated across a representative sample of the workforce, is what turns a training program into audit evidence.
Mapping Evidence to OJK POJK 11/2022, UU PDP, and ISO 27001
OJK's POJK 11/2022 on cyber resilience and security for commercial banks, and its counterparts for other regulated financial institutions, expects demonstrable operational risk controls around people, not just technology. Examiners look for evidence that staff, particularly those with access to customer funds or data, are tested against realistic social engineering scenarios on a recurring cadence, and that failures trigger a documented remediation path rather than being logged and ignored. Your simulation results and training completion records, tied to job role and department, are the direct evidence for this control.
UU PDP, Indonesia's Personal Data Protection Law, does not mention phishing simulations by name, but its accountability principle requires data controllers to demonstrate organizational measures that protect personal data, which includes evidence that staff handling personal data understand social engineering risks that could lead to a data breach. A record showing that employees with data access completed awareness training, understood the reporting channel, and were retested after a lapse, supports the appropriate technical and organizational measures standard that UU PDP enforcement and DPO audits increasingly ask about.
ISO 27001:2022 Annex A control 6.3, information security awareness, education, and training, is the most explicit of the three: it requires evidence of a training program, evidence of periodic review of its effectiveness, and evidence that non-conformities, meaning people who fail, are addressed. Certification body auditors will typically sample individual employee records during a surveillance audit, so having per-user completion and simulation history readily exportable, not buried in a dashboard, is the difference between a smooth surveillance audit and a finding.
Building the Evidence Pack: A Practical Checklist
Start with scope and sampling: define the population the evidence pack must cover for this audit cycle, whether that is all staff, a specific business unit, or all users with data access, and be ready to produce records for a statistically reasonable sample the auditor selects on the spot, not just the aggregate. Next, pull the four pillars for that population and period: simulation participation and outcomes, training assignment and completion, phish-report rate, and risk score movement, each exportable to CSV or PDF with names, dates, and departments intact, redacted only if the audit context specifically requires anonymization.
Layer in the narrative documents auditors expect alongside the raw data: the assignment rule or policy that determined who gets tested and how often, the remediation workflow definition, meaning what happens automatically when someone clicks and what a manager does when someone fails repeatedly, and a one-page executive summary translating the numbers into a risk story a board member or examiner without a technical background can follow. Compliance report generation that produces a Claro-branded PDF per framework, OJK, ISO 27001, PDPA, or board summary, with the underlying control coverage table already mapped is exactly this layer, and should require no manual reformatting the night before submission.
Finally, include a short methodology note: what a click, a report, and a completion mean operationally in your platform, how risk scores are calculated, and how long records are retained. Auditors trust evidence more when the methodology behind it is explicit and consistent release over release, rather than something that has to be explained verbally in the room. A one-page methodology appendix, updated whenever scoring logic changes, removes most follow-up questions before they are asked.
Keeping the Evidence Pack Audit-Ready Year-Round
The single biggest mistake is treating evidence-gathering as a pre-audit sprint. By the time an OJK examination notice or ISO surveillance audit date is confirmed, it is too late to fix six months of inconsistent assignment rules or unexplained gaps in reporting-rate data. Instead, generate and file the quarterly evidence pack as a standing operational habit, the same way financial statements are closed monthly regardless of when the external audit lands.
Automate what can be automated: a scheduled compliance report run, not a manually triggered one remembered only when someone thinks of it, that produces the current control coverage view, the latest risk score trend, and completion status every quarter, stored in the same evidence archive the previous quarters live in. This turns producing evidence for the last twelve months from a scramble into an export of twelve already-generated snapshots.
Finally, review the evidence pack itself for gaps before the auditor does. Are there departments with no simulation history because they were added to the tenant late. Is there a training assignment rule that quietly stopped firing after a reorganization. Does the risk score show a plateau that a good auditor will ask about. Closing those gaps proactively, quarter over quarter, is what separates a security team that merely runs a program from one that can prove, on demand, that the program works.
Key takeaways
- Evidence auditors want is people-level and time-stamped, not a summary slide.
- Four pillars: simulation results, training completion, phish-report and JIT metrics, and risk score movement over time.
- Map each pillar to the control it satisfies: OJK operational risk controls, the UU PDP accountability principle, and ISO 27001 Annex A 6.3.
- Build the evidence pack continuously, on a quarterly cadence, not two weeks before the audit.
- Keep raw exportable data (CSV or PDF) alongside narrative summaries; both matter to a defensible audit.