Time to Close vs. Root Cause – Are we measuring the wrong thing (again)?

“Human beings adjust behavior based on the metrics they’re held against. Anything you measure will impel a person to optimize his score on that metric. What you measure is what you’ll get. Period.” – Dan Ariely, Duke University behavioral economist in Harvard Business Review  

When the Verizon Data Breach Investigation Report started reporting “time to” metrics around 2013 (time to detect, time to contain, time to remediate), most security operations managers started to monitor their own team’s performance against these stats. That’s not a bad thing – I’ve certainly touted these numbers in my posts before. They help assess workloads and justify investment.

However, as managers, we need to add another lens to emphasize efficiency AND effectiveness.

Closing cases (time to contain, time to remediate) without getting to root cause is like chopping off the arm of the starfish – the arm will likely grow back and may come back bigger and nastier.

Why care about root cause?

Root cause is the secret to returning to a healthy state. Getting to root cause means you identify how the attacker got in, which systems provided cover, which credentials were abused, and how they manipulated system, countermeasure, and application software to hide their tracks. When you push investigations to the point of root cause analysis, you are more likely to fully scope the attacker’s activities and excise them from your estate. If you don’t get to root cause, an attacker may retain a foothold, ready to reactivate after you have reimaged the host or blocked an IP address and claimed “case closed.” That lingering presence means you still risk damage, as well as repeated cleanup costs.

In Disrupting the Disruptors, Art or Science?, we researched threat hunting practices in security operations centers. Time to close is an important stat, and the most mature orgs are closing faster than anyone else, by a huge margin. Mature orgs were 2 times more likely to close cases within a day than the merely innovative, and closer to three times more likely to close within a day than the SOCs just getting started. (For details on the maturity definitions and other findings, download the free report.)

Leaders close, with higher confidence the incident won’t recur

But – there’s another very important metric that clearly isn’t being rewarded as aggressively, or the numbers would be better, per the behavioral psychologists who say you get what you measure. The most advanced threat hunting organizations are winning on time to close AND aggressively uncovering root cause. Hunters at the minimal level typically determine the cause of just 20-30% of attacks, compared to leading hunters’ digging in to find 70% or more.

Net net: the leading SOCs are closing more cases faster AND getting to root cause most of the time – performing far better than their peer groups. As an industry, let’s start to measure both of these goals to increase overall cybersecurity health.

For insights on how leading SOCs are achieving these results, such as advanced use of automation and sandboxing, read the report.

Leave a Comment

seventeen − sixteen =