False positives, and what they cost

What a wrong alert actually costs

The obvious cost is time: somebody investigates something that turns out to be a project deadline.

The larger cost is credibility. A queue that is mostly noise gets processed less carefully each week, and the mechanism by which a real signal is missed is not carelessness but rational adaptation to a system that has been wrong two hundred times.

The third cost is the worst. An alert acted on too quickly, involving a person who did nothing wrong, damages that person and teaches everybody watching what the programme is for.

Four sources

Baselines built too quickly. A fortnight of observation does not capture month-end, quarter-end or the annual audit.

Rules written for a different organisation. Default detections assume an office. A manufacturer moves large files to machines at odd hours as a matter of routine.

Roles not distinguished. What is unusual for an accounts clerk is Tuesday for a programmer.

Legitimate change. A new customer, a new production line, a new tool. The system correctly reports that things are different.

An activity, not a setting

Observe before alerting, for long enough to see the cycles. Suppress known patterns explicitly and with a reason recorded. Write rules per role rather than globally. And review the alerts that were closed as nothing, monthly, looking for the pattern that produces them.

That review is the work. Systems that are never tuned converge on being ignored, which is the same as not having them.

Precision, not volume

The useful number is the proportion of alerts that turned out to be worth investigating. A system producing four alerts a month of which three merit a look is more valuable than one producing four hundred of which five do.

Vendors report volume because it demonstrates activity. Ask instead what proportion of alerts their customers act on.

Quietly first

Check the obvious explanations before speaking to anybody: is there a project, a deadline, a shift change, a system migration. Ask the person's manager whether the activity fits the work.

Only then, if it still does not fit, does anybody talk to the person, and that conversation is a question rather than an accusation.

This sequence costs half an hour and prevents almost every case where these programmes cause harm.

Including the ones that were nothing

What was alerted, what was checked, what the explanation was, who decided. The record demonstrates a functioning process to an assessor and it is the input to the monthly tuning review.

It also protects the person who was investigated and cleared, which matters if the question ever arises again.

Few alerts, each worth reading

An organisation of a hundred people should expect a small number of signals a month, not a dashboard. If the system produces more, it is untuned rather than the organisation being unusually eventful.

Saying so is against the interest of anybody selling by the alert, and it is what makes the capability worth having.

Deploy in observation mode first

Collect without alerting for a full business cycle, then look at what would have fired. Almost always the answer is far too much, and tuning against real data before anybody is asked to act on an alert is the cheapest possible version of this work.

Elsewhere in insider risk

• What the three levels are, and which appliesThe level is set by your contract, not by your size. Most suppliers handling controlled unclassified information are at the second.
• Reading the clause that binds youReading the clause that binds you. What the requirement says, what it means in practice, and what an assessor will ask.
• Which systems are in scopeWhich systems are in scope. What the requirement says, what it means in practice, and what an assessor will ask.
• Self-assessment, and why the scores driftedSelf-assessment, and why the scores drifted. What the requirement says, what it means in practice, and what an assessor will ask.
• Who assesses you, and how each kind worksWho assesses you, and how each kind works. What the requirement says, what it means in practice, and what an assessor will ask.
• What the standard does not requireWhat the standard does not require. What the requirement says, what it means in practice, and what an assessor will ask.
• Machines older than the requirementsA control written for a server, applied to a lathe with a shared login, a decade-old operating system and a USB port.
• Drawings, job packets and paperDrawings, job packets and paper. What the requirement says, what it means in practice, and what an assessor will ask.
• Removable media, and why bans failRemovable media, and why bans fail. What the requirement says, what it means in practice, and what an assessor will ask.
• Vendor remote accessVendor remote access. What the requirement says, what it means in practice, and what an assessor will ask.
• Technical data packages and what they containTechnical data packages and what they contain. What the requirement says, what it means in practice, and what an assessor will ask.
• Segmentation, and what it does not solveSegmentation, and what it does not solve. What the requirement says, what it means in practice, and what an assessor will ask.
• What insider risk actually coversWhat insider risk actually covers. What the requirement says, what it means in practice, and what an assessor will ask.
• What detection can and cannot establishWhat detection can and cannot establish. What the requirement says, what it means in practice, and what an assessor will ask.
• Monitoring, and where the law constrains itMonitoring, and where the law constrains it. What the requirement says, what it means in practice, and what an assessor will ask.
• The fortnight around a departureThe fortnight around a departure. What the requirement says, what it means in practice, and what an assessor will ask.
• Privileged access, and who reviews itPrivileged access, and who reviews it. What the requirement says, what it means in practice, and what an assessor will ask.
• The documents an assessor reads firstThe documents an assessor reads first. What the requirement says, what it means in practice, and what an assessor will ask.
• What counts as evidenceWhat counts as evidence. What the requirement says, what it means in practice, and what an assessor will ask.
• The findings that recurThe findings that recur. What the requirement says, what it means in practice, and what an assessor will ask.
• Plans of action, and their limitsPlans of action, and their limits. What the requirement says, what it means in practice, and what an assessor will ask.
• What the assessment week is likeWhat the assessment week is like. What the requirement says, what it means in practice, and what an assessor will ask.
• The certificate, and what it coversThe certificate, and what it covers. What the requirement says, what it means in practice, and what an assessor will ask.