Why traditional metrics are giving CISOs a false sense of security is attracting attention across the tech world. Analysts, enthusiasts, and industry observers are watching closely to see how this story develops.
This update adds another signal to a fast-moving sector where product decisions, platform changes, and competition can quickly shape the market.
Traditional security metrics mislead CISOs, masking real cyber risk exposure
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Get full access to premium articles, exclusive features and a growing list of member rewards.
The rising threat of cyberattacks has cranked up the pressure for CISOs right at the heart of business resilience. But their job has become all the more difficult.
Our research found that 50% of organizations now carry critical security debt, meaning they have software vulnerabilities that have been left unresolved for longer than a year.
That’s an open invitation for cyber criminals and requires a comprehensive, long-term application risk management strategy to fix it. Yet most organizations still equate more scans with better security.
This assumption is creating serious security gaps, especially across software supply chains and CI/CD pipelines.
The fact is, not only do traditional security KPIs not measure real security efficacy—they also create a false sense of progress. Recent pipeline and dependency compromises, like the Shai-Hulud supply chain wormware campaign, are a good example of why high scan volume alone does little to prevent breaches.
CISOs need to refocus. The most significant metrics measure vulnerability backlogs, undetected attacker dwell time, and existing security controls with proven ability to mitigate real-world threat, not just theoretical risk. Ultimately, depth and validation matter far more than breadth.
Measuring against volume-based KPIs, like the number of scans run, vulnerabilities found and alerts generated, only tracks the effort taken to increase security—not the actual outcome. These traditional KPIs tell you how needed security measures are, but not whether they are stopping anything meaningful.
for instance, a scan finding 10,000 low impact issues might look productive on a dashboard, but at the same time a single exploitable dependency might have been untouched for months, presenting a critical, unresolved security risk.
Board members and the C-suite see rising KPI numbers and automatically assume strengthened protection when, in fact, it could be quite the opposite. This blurred measurement line skews the reality of how security teams are tackling security risk.
These industry wide tropes are inadvertently rewarding security teams for generating noise but not reducing actual risk. And with the average fix time for security flaws rising from 171 days to 252 days over the past five years, the delay to remediation quietly backlogs security risks.
Those vulnerabilities hidden in the depths of the supply chain and pipeline are a ticking time bomb.
With security teams already stretched and struggling to find the capacity for finding and fixing vulnerabilities, these outdated metrics encourage a culture where security teams and CISOs look “on top of it”, right up until an old, known flaw gets exploited – at which point, it could be too late.
With the rapid pace of technological advancement and the apparent rise in successful cyberattacks, point-in-time scanning is now inadequate. It overlooks critical time factors—such as the mean time to remediate or the duration an attacker can operate undetected—which are precisely what attackers exploit.
Modern attacks happen in the gap between scans, with security snapshots unable to catch moving targets. For CI/CD pipelines, they are obsolete. Code changes multiple times a day and dependencies update automatically.
And nowadays, an attacker doesn’t even need to evade a scan. They just wait for the next build, commit, or dependency pull and, by the time the scan report is read, the environment it assessed no longer exists.
Scanners traditionally inspect source or binaries, but not the inner workings of the build process, meaning a malicious build step can inject code after a scan has passed.
This happened with the infamous SolarWinds Orion attack, which compromised thousands of organizations (including US government agencies) back in 2020, injecting malicious code into software updates that were then distributed to the unsuspecting customers.
If the build is already poisoned, then the scan is irrelevant.
As cyber risk increases and hackers become more sophisticated, balancing the challenges associated with assessing risk and proving the value of application security is becoming more of a minefield for CISOs. They need metrics and that security teams can prioritize to better reflect real application and supply-chain security risk.
These include the backlog reduction of exploitable flaws, the time it takes to fix critical issues in production, and evidence that the fixes actually work, rather than just a scan. The shift isn’t from less measurement to more measurement. It’s from counting security activity to measuring true exposure and business resilience.
Ultimately, security metrics should tell leadership how much risk has been removed and how quickly platforms are back to normal—not how hard the security team worked to find it. This change in positioning will help us all become better equipped to properly defend against risk.
This article was produced as part of TechRadarPro’s Expert Insights channel where we feature the best and brightest minds in the tech innovation industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro
Please logout and then login again, you will then be prompted to enter your display name.
Why This Matters
This development may influence user expectations, future product strategy, and the competitive balance inside the broader technology industry.
Companies in adjacent segments often react quickly to similar moves, which is why stories like this tend to matter beyond a single announcement.
Looking Ahead
The full impact will become clearer over time, but the story already highlights how quickly the modern tech landscape can evolve.
Observers will continue tracking the next steps and how they affect products, users, and the wider market.
