As security environments become increasingly complex, threat actors wreak more havoc faster than ever, causing severe financial implications for the unprepared. With 34% of breaches caused by an external attack, taking a hacker’s perspective of an organization can significantly inform security strategies and policies.1 External attack surface management (EASM) tools provide vital insights to de-risk and mitigate exposures stemming from vulnerable external assets.
As part of a unified offensive security platform, IBM Security Randori provides a comprehensive view of an organization’s external attack surface. The IBM Security Randori platform consists of:
- IBM Security Randori Recon. The foundational external attack surface management tool, Recon uses continuous monitoring to help security, operations (SecOps) teams detect and prioritize external cyber risks, shadow IT, software misconfigurations, and other vulnerabilities.
- IBM Security Randori Attack. Augmenting the Randori Recon foundation, Attack delivers continuous automated red teaming, and complements, or replaces, penetration testing.2
IBM commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study and examine the potential return on investment (ROI) enterprises may realize by deploying Randori. The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of Randori on their organizations.
90%
Less vulnerability scanning
To better understand the benefits, costs, and risks associated with the investment, Forrester interviewed four representatives with experience using Randori. Forrester aggregated the interviewees’ experiences into a single composite organization with 10,000 knowledge workers, $10 billion in revenue, and 15,000 assets.
Prior to using Randori, interviewees cited experiences with complex and ephemeral external attack surfaces. The persistence of shadow IT led to a proliferation of “unknown unknowns” in the face of faster and increasingly hostile cyber foes. The interviewees’ organizations were early adopters of attack surface management with Randori the tool at the center of their exposure management programs.3
By leaning into a more mature security culture in the cloud, interviewed security leaders eliminated rote, manual processes with automation. Enhanced support for threat hunting from Randori helped interviewees’ organizations increase visibility and control over their externally-facing cyber assets, increasing confidence in security investments while streamlining expenditures. These improvements helped interviewees mitigate risk exposure, better prioritize risk response decisions, and act faster.
Consulting Team: Courtenay O’Connor, Nahida Nisa
Key Statistics
-
Return on investment (ROI):
-
Benefits PV:
-
Net present value (NPV):
-
Payback (months):
7
Registration Required To Continue
COOKIE ACCEPTANCE IS REQUIRED TO REGISTER FOR ACCESS TO DIGITAL ASSET
Key Findings
Quantified benefits. Three-year, risk-adjusted present value (PV) quantified benefits for the composite organization include:
- 85% reduction in losses due to an external attack totaling $1.5 million. Randori helps the composite locate a small but important number of previously undetected assets left exposed. By reducing the amount of time an exposed asset is left “in the wild,” the composite avoids financial and brand impacts from an attack.
For , avoided losses from an external attack could total .
- 30% reduction in time to triage exposures for remediation totaling over $522,000. The composite organization reduces the time it takes SecOps resources to detect, analyze, and action exposure-related external attacks by over 13,500 hours. The composite organization reallocates time savings to threat hunting and purple teaming exercises and evaluating the security tech stack.
For , SecOps rapid response efficiencies could total .
- Up to 75% labor savings from augmented red team activities totaling $214,000. With an upgrade to Randori Attack in Year 2, the composite organization’s SecOps resources reattribute time formerly spent detecting and triaging exposures, and helps the composite avoid fees for penetration testing.
For , savings from augmented red team activities could total .
- 90% reduction in exposure analysis effort valued at $103,000. With Randori, the composite organization avoids 2,600 hours of regular scanning for and triaging exposures.
For , exposure analysis efficiencies could total .
- 85% reduction in exposure-related help desk tickets worth $175,000. With Randori’s effect on reducing false positives and mitigating true issues, the composite avoids issuing nearly 8,000 exposure-related tickets.
For , a reduction in exposure-related help desk tickets could be worth .
- 15% reduction in vulnerability scanning license fees totaling $41,000. By leveraging Randori capabilities, the composite avoids fees for vulnerability scanning licenses.
For , avoided vulnerability scanning license fees could total .
- 20% reduction in cyber insurance totaling $27,000. Randori helps maintain valuable cyber insurance with lower premiums.
For , savings on cybersecurity insurance could total .
Unquantified benefits. Benefits that provide value for the composite organization but are not quantified in this study include:
- Cleaner audits and streamlined mergers and acquisitions (M&A).
- Further time savings and quality improvements from automation.
- More collaborative, informed approach to risk.
- Upskilling for newer hires.
- Security tool validation and optimized tech stack.
Costs. Three-year, risk-adjusted PV costs for the composite organization include:
- Randori subscription fees totaling $621,000.The composite deploys Randori Recon for all three years, adding Randori Attack in Years 2 and 3. Subscription fees are assessed as a function of the average number of knowledge workers, which hovers around 10,000 annually.
For , Randori subscription fees could total .
- Internal deployment and administration labor costs of $13,000. During the one-month initial period, the composite organization dedicates security resources to technical deployment and training on Randori Recon. The composite increases the level of internal deployment and administration effort during Years 2 and 3 when it upgrades to include Randori Attack.
For , deployment and administration labor costs could total .
The representative interviews and financial analysis found that a composite organization experiences benefits of $2.55 million over three years versus costs of $633,000, adding up to a net present value (NPV) of $1.92 million and an ROI of 303%.
could experience benefits of over three years versus costs of , adding up to a net present value (NPV) of and an ROI of .
“As we use more cloud services online, we still find traveling sites. IBM Security Randori is continuously scanning, so each problem I may have has a smaller window for someone to attack.”
Security engineer, entertainment
Benefits (Three-Year)
Avoided losses from an external attack SecOps rapid response efficiencies Savings from augmented red team activities Exposure analysis efficiencies Reduction in exposure-related help desk tickets Avoided vulnerability scanning license fees Savings on cybersecurity insuranceAttacks will happen, and they will sometimes go unnoticed. Randori mitigates the impact of inevitable intrusions.
Daily investment in hardening the attack surface is vital to staying ahead of today’s worrisome cyber adversaries.
Before an attack can even occur, the composite organization fully achieves a positive return on the Randori investment.
This initial yield is achieved through multiple internal and external efficiencies in both capex and opex.
TEI Framework And Methodology
From the information provided in the interviews, Forrester constructed a Total Economic Impact™ framework for those organizations considering an investment Randori.
The objective of the framework is to identify the cost, benefit, flexibility, and risk factors that affect the investment decision. Forrester took a multistep approach to evaluate the impact that Randori can have on an organization.
-
Due Diligence
Interviewed IBM stakeholders and Forrester analysts to gather data relative to Randori.
-
Interviews
Interviewed four representatives at organizations using Randori to obtain data about costs, benefits, and risks.
-
Composite Organization
Designed a composite organization based on characteristics of the interviewees’ organizations.
-
Financial Model Framework
Constructed a financial model representative of the interviews using the TEI methodology and risk-adjusted the financial model based on issues and concerns of the interviewees.
-
Case Study
Employed four fundamental elements of TEI in modeling the investment impact: benefits, costs, flexibility, and risks. Given the increasing sophistication of ROI analyses related to IT investments, Forrester’s TEI methodology provides a complete picture of the total economic impact of purchase decisions. Please see Appendix A for additional information on the TEI methodology.
Disclosures
Readers should be aware of the following:
This study is commissioned by IBM and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.
Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the study to determine the appropriateness of an investment in Randori.
IBM reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.
IBM provided the customer names for the interviews but did not participate in the interviews.
Drivers leading to the Randori investment
Interviews
| Role | Industry | Region | Number Of External Assets |
|---|---|---|---|
| Chief information security officer (CISO) | Grocery store | US HQ, US reach | 100,000 |
| Security engineer | Entertainment | North America HQ, global reach | 50,000 |
| Head of cybersecurity | Professional services | North America HQ, global reach | 12,000 |
| Information security manager | Manufacturing | North America HQ, global reach | 10,000 |
Key Challenges
Prior to their investment in Randori, interviewees shared that their legacy security environments could no longer rely on traditional vulnerability management regimes due to pressures such as:
Lack of visibility. Interviewees described complex and ephemeral data estates. Shadow IT led to an increased level of unknowns in interviewees’ dynamic, rapidly-changing attack surfaces.
- The CISO in the grocery store industry described a lack of visibility prior to Randori: “The attack surface is so dynamic; it changes every day. It encompasses on-premises, public cloud, private cloud, and [software-as-a-service] SaaS offerings. We have remote, hybrid, and office workers. That all constitutes the attack surface, and we had to fundamentally rethink how we manage that.”
- The security engineer in the entertainment industry said, “We really didn’t have a lot of visibility.”
- The senior security engineer in the entertainment industry shared: “The developers’ main goal is to send the code out, get the software up, and they don’t always have the time to patch their own things. They’re not security specialists. They are system admins, it’s not their day-to-day stuff.”
“Before Randori, we had limited visibility. There were probably dozens of instances where an application went to end of life and we removed it because we have a new application; but then, for some reason, that old application was not properly taken down.”
Information security manager, manufacturing
Reactive shifts to the cloud. Rapid digital transformation further exacerbated shadow IT in interviewees’ organizations’ prior environments. This lent unnecessary complexity to the security teams’ efforts to harden their organizations’ dynamic external attack surface.
- The security engineer in the entertainment industry pointed out cloud visibility challenges. They said: “An asset could have been outdated for a year. Users are on the cloud nowadays, so they can just bring up a site without letting my team know.”
- The information security manager in the manufacturing industry described their organization’s journey to the cloud, saying: “The cloud space is such a fast-moving environment with lots of innovation happening there. Oftentimes, it’s the necessity of the business that drives the adoption of those cutting-edge technologies and the cloud.”
“The speed at which the business wants to adopt the cloud can be challenging for information security because we don’t want to be playing catch-up, we want to be out in front.”
Information security manager, manufacturing
Inadequate tools for the job. Interviewees described manual processes to scan for cyber asset exposures which left their organizations vulnerable to an increasingly daunting threat landscape. Whether part of routine scanning, or in rapid response to an exposure “in the wild,” legacy technologies and the processes that appended them stymied SecOps resources’ efforts.
- The CISO in the grocery store industry described how insufficient tooling had a cascading impact on costs: “We’d spend a lot of time taking threat intel data and layering it on top of vulnerability scans and then prioritizing, because I can’t just send a spreadsheet of a thousand vulnerabilities over. [Furthermore], in the last couple of years, more and more carriers are just not providing cyber insurance coverage, and if you get coverage, it’s very difficult.”
- The information security manager in the manufacturing industry pointed out that legacy tools were facing an ever-mounting remit: “If you just look at the statistics and indicators, you can see the number of vulnerabilities reported in the last five years is exponentially higher, and that the time to exploit a vulnerability is shrinking more and more since 2017. It used to be months, then weeks, now its days.”
Supply chain risk. Two interviewees also described further information security challenges related to their physical supply chains:
- The CISO in the grocery store industry noted: “We have to comply with the Customs and Border Patrol (CBP) compliance program because we import so many goods from overseas. [CBP] stood up an information security requirement for organizations with containers coming into the country, making sure that you have control over that entire supply chain in the end. Homeland Security is very concerned with smuggling potential products that can be used to do harm, and with human trafficking. You have to control containers from the supplier all the way to our distribution centers.”
- The information security manager in the manufacturing industry shared: “Supply chain is always a concern for a manufacturing company. It’s a tougher nut to crack because you’re talking about physical security as well as logical.”
“Our biggest pain point was improving our time to detect and time to respond as a security organization.”
Security engineer, entertainment
“All the indicators in the industry are saying that you can’t just sit back and not improve. You have to continue to keep pace and improve your defenses.”
Information security manager, manufacturing
Market Overview
Today, digital transformation has more companies thinking about when and how to move to the cloud, not if they should do so (see Figure 1). This has led to a rapid rise of cloud adoption, while leaving unprepared security organizations vulnerable to leading threats.
According to Forrester, increased cloud complexity is changing the way security teams defend and the emerging threats they face.4 External attacks can come from a number of vectors with software vulnerability exploitation at the top of the list (see Figure 2).
Cloud Adoption
Figure 1. In 2022, a majority of technology decision-makers planned to or were in the process of adopting a cloud deployment model.
Note: Numbers may not total 100% due to rounding.
Top Five Causes Of External Attacks
Figure 2. Top five causes of external attacks in 2021 can mitigated through effective EASM approaches.
Source: “Top Security Threats In 2022,” Forrester Research, Inc., April 8, 2022.
“Our goal was to become much more efficient in our operations, so that we could spend more time doing proactive security and threat hunting.”
CISO, grocery store
Investment Objectives
The interviewees searched for a solution that could enable a more proactive, mature security posture beyond vulnerability management to cover the ASM spectrum (see Figure 4).
- The information security manager in the manufacturing industry shared: “Randori was selected to help us gain an external picture of what our environment looked like to the general public or to the adversary. We’re a big company with lots of moves and changes when it comes to servers and applications. That attacker view then also helps us to see things that we may not have known otherwise.”
- The head of cybersecurity in the professional services industry pointed to increasing pressures from compliance regimes to augment their security posture: “Our Trust department includes cybersecurity, privacy and GDPR, [California Consumer Privacy Act of 2018] (CCPA). The attack surface point of view was where we needed to be.”
Attack surface management definitions
| Cyber asset | Granular, software-defined entities that include IP addresses, users, devices, code commits, security controls, applications, access policies, cloud configurations, and more. |
| Attack surface management (ASM) | The process of continuously discovering, identifying, inventorying, and assessing the exposures of an entity’s IT asset estate. |
| External attack surface management (EASM) | Tools or functionalities that continually scan for, discover, and enumerate internet-facing assets, establish the unique fingerprints of discovered assets, and identify exposures on both known and unknown assets. |
| Cyber asset attack surface management (CAASM) | A tool or capability that delivers unified visibility across all known assets (internal, external, cloud, on-premises) for better identification of vulnerabilities and insufficient security controls. |
Figure 4. Attack surface management definitions per Forrester’s February 2022 report, “Tame The Asset Management Beast.”
Composite Organization
Based on the interviews, Forrester constructed a TEI framework, a composite company, and an ROI analysis that illustrates the areas financially affected. The composite organization is representative of the four representatives that Forrester interviewed and is used to present the aggregate financial analysis in the next section. The composite organization has the following characteristics:
Description of composite. With headquarters in North America, the globally distributed company has a large infrastructure and a strong online and offline presence with an average of 15,000 external assets monitored on a regular basis. Operating in a highly regulated space, the composite organization has 10,000 knowledge workers and generates $10 billion in revenue annually.
Prior to Randori, the composite had a moderately mature security program, relying heavily on vulnerability management tools coupled with manual processes to research, contextualize, prioritize, and issue an average of 300 exposure-related tickets per month. The company outsourced a small number of penetration tests with limited in-house red/blue/purple team functions.
Deployment characteristics. With Randori, the composite organization evolves its security maturity to a more advanced, ASM-focused posture during the investment period.
- Year 1: Randori Recon is deployed in Year 1 to support the development of the composite organization’s ASM team.
- Years 2 and 3: Randori Attack is added in, further augmenting red team capabilities.
Key Assumptions
- $10 billion in revenue
- 15,000 external assets
- 10,000 knowledge workers
- 300 exposure-related tickets monthly
Quantified benefit data as applied to the composite
Total Benefits
| Ref. | Benefit | Year 1 | Year 2 | Year 3 | Total | Present Value |
|---|---|---|---|---|---|---|
| Atr | Avoided losses from an external attack | |||||
| Btr | SecOps rapid response efficiencies | |||||
| Ctr | Savings from augmented red team activities | |||||
| Dtr | Exposure analysis efficiencies | |||||
| Etr | Reduction in exposure-related help desk tickets | |||||
| Ftr | Avoided vulnerability scanning license fees | |||||
| Gtr | Savings on cybersecurity insurance | |||||
| Total benefits (risk-adjusted) |
“Part of Randori’s value is to mature our attack surface management by helping us to identify our shadow IT.”
Information security manager, manufacturing
Avoided Losses From An External Attack
Evidence and data. Randori helped interviewees locate a small but important number of undetected assets previously left exposed. Interviewees reported a decline in exposures that could lead to significant financial and brand impacts for their organizations.
- The CISO in the grocery store industry described how a small number of such assets left their organization at risk: “Without Randori, we would miss critical vulnerabilities. Threat intel evolves every single minute, and my team is not just sitting there constantly looking at that information. In the last year with Randori, we found eight exposed assets out of 230. For example, we were recently notified of a very old server that was fired up, connected, and exposed to the internet. As old as that device was, the vulnerabilities would have been exploited and totally compromised in less than 24 hours, and we’ve had eight of those types of assets discovered in the last year.”
- The security engineer in the entertainment industry shared how Randori better positioned their organization: “It helped us see things like shadow IT, where assets came online without following the normal business process. Our most critical online assets are our reputation and IP. At any given time, 10,000 assets are accessible from the internet that Randori can see. That’s our core environments, our subsidiaries, and a bit of ephemeral environment that spins up for projects and then spins down in the cloud.”
- The head of cybersecurity in the professional services industry pointed to the value that Randori provided in protecting their organization against the brand and customer aftermath of costly exposures: “Randori is building a level of confidence in our reputation. That confidence helps us from a dollars and cents perspective.”
“Randori helps us find our [outdated] sites and services that are no longer needed. It also shows how we look from the lens of the attacker.”
Security engineer, entertainment
Modeling and assumptions. The composite avoids losses from an external attack as follows:
-
In the prior environment, the composite organization experienced the following costs related to an external attack caused by an exposed externally facing cyber asset:
- Loss of business due to system downtime totaling $60,000.
- Efficiency losses from end-user downtime totaling $57,000.5
The information provided suggests that may experience the following costs related to an external attack caused by an exposed externally facing cyber asset: loss of business due to system downtime totaling and efficiency losses from end-user downtime totaling .
- Upon making the Randori Recon investment, the initial scan for exposed external assets finds a small but important number of cyber assets that make the composite organization vulnerable to costly disruptions.
The information provided suggests that might have cyber assets vulnerable to costly disruptions.
-
Over the three-year investment period, the composite organization:
- Deploys Randori Recon, which cuts the potential window during which an exposed asset could fall prey to an external attacker in half.
- Upgrades to include Randori Attack capabilities. The window during which an exposed asset is vulnerable to an external attack further shortens by 70% in Year 2 and by 85% in Year 3.
Based on the Randori deployment schedule of and the findings of the TEI case study, the window during which an exposed asset is vulnerable to an external attack could fall by in Year 1, in Year 2, and in Year 3.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:
- An organization’s deployment configuration of Randori Recon and Attack. Extended use of the full platform may amplify results, compared to only deploying Randori Recon.
- An organization’s security context, including its security maturity, and the likelihood, number, cost; severity of exposure-related security incidents in the prior environment; and other resources, processes, and technology deployed to mitigate exposures.
- The number of vulnerable external assets detected and protected with Randori. Organizations will have varying levels of impact to revenue-generating and employee-facing assets.
Results. To account for these risks, Forrester adjusted this benefit downward by 25%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1.5 million.
For , this benefit may have a three-year, risk-adjusted total PV of .
Avoided Losses From An External Attack
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| A1 | Number of vulnerable assets found with Randori | ||||
| A2 | Loss of business due to system downtime during exposure-related external attack | ||||
| A3 | End-user downtime during exposure-related external attack | ||||
| A4 | Reduction potential window of an exposure-related external attacks with Randori | ||||
| At | Avoided losses from an external attack | A1*(A2+A3)*A4 | |||
| Risk adjustment | ↓25% | ||||
| Atr | Avoided losses from an external attack (risk-adjusted) | ||||
| Three-year total: | Three-year present value: | ||||
“Randori prioritizes down into the medium-level threats, which can be chained together to cause breaches.”
CISO, grocery store
30%
Faster SecOps rapid response
SecOps Rapid Response Efficiencies
Evidence and data. With increasing use of Randori, interviewees reduced the time it took rapid response teams to triage exposure-related issues.
- The CISO in the grocery store industry reported the impact of deploying Randori as part of the engine of their new ASM team: “We’ve more than doubled in our efficiency and operations and a large part of that is due to Randori. My team prior was so in the weeds that I was adding headcount. Of the 15 people in [SecOps], I had 12 FTEs working on threat and vulnerability alerts. That team has now shrunk down to five. The rest of them are working on threat hunting and purple teaming exercises, as well as evaluating our tech stack.”
- The head of cybersecurity in the professional services industry shared: “[Randori] gives you a lot more contextual data and that single pane of glass. Instead of my team having to log into 12 different solutions to get that full visibility, they can just say, ‘What do I know about asset or that hostname or that IP address?’”
- The information security manager in the manufacturing industry noted the value of Randori’s attacker’s perspective: “We have several dozen [SecOps] personnel looking at threats and indications of compromise. Randori has allowed us to focus on the medium-tier-severity risks. Previously, most of our effort went towards high or critical severity, which were easier to see with vulnerability scanning. As you get down from the tip of the pyramid where there’s a discrete number of issues, there become many more medium- or low-severity risks. Oftentimes, you have to trade off and only focus on the most severe. But Randori has helped us slice through those medium-severity items and then attack them in sectors.”
Modeling and assumptions. The composite organization improves its rapid response to exposure-related incidents as follows:
- In the prior environment, the composite organization dedicated 10 SecOps resources to identifying and triaging exposures.
According to the information provided, state dedicates inSecOps SecOps resources to identifying and triaging exposures.
- With automations from Randori Recon deployed in Year 1, SecOps resources reduce the time spent on exposure response by 10%.
- By adding Randori Attack in Years 2 and 3, SecOps resources improve prioritization and situational understanding. These added security capabilities further decrease the time SecOps resources spend on exposure response by 25% and 30% in Years 2 and 3, respectively. The composite organization reallocates time savings to threat hunting, purple team exercises, and evaluating the tech stack.
Based on the Randori deployment schedule of state and the findings of the TEI case study, the time SecOps spends on exposure response could fall by Benefit2B2CFYear1 in Year 1, Benefit2B2CFYear2 in Year 2, Benefit2B2CFYear2 in Year 3.
- The average fully burdened hourly rate per security resource is $60.
- Deployment configuration of Randori Recon and Attack. Extended use of the full platform may amplify results, compared to only deploying Randori Recon.
- An organization’s security context, including its security maturity, and the likelihood, number, cost, and severity of exposure-related security incidents in the prior environment.
- The type of FTEs on the rapid response team and the amount of time and complexity of effort required for remediation per asset. These factors will dictate the number of SecOps resources fully dedicated to triaging exposures and other resources, processes, and technology deployed to mitigate exposures.
- The number of vulnerable external assets detected and protected with Randori.
Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $522,000.
For state, this benefit may have a three-year, risk-adjusted total PV of Benefit2BtrCFPV.
Interviewee Voices: Log4j
On December 9, 2021, cybersecurity professionals learned of a critical vulnerability in the open source Apache code underpinning billions of devices. The CISO in the grocery store industry contemplated how Randori helped their organization respond to the vulnerability in a fraction of the time it would have taken without the platform: “One example that gave me my return on investment with Randori was Log4j. Had we not been able to identify it in time, it could have been the beginning of a major incident for us. Log4j came out on a Thursday night. Without Randori, it would realistically have taken 8 to 12 hours to detect. My team would have to scan our external environment, get all those results, figure out which ones are false positives, and act on them. That time is precious. With Randori, that Friday morning it was telling me we had systems that were vulnerable externally to Log4j. My team was able to pull those systems offline and get them remediated immediately. So, when my CEO called me Friday morning, I already identified the affected, externally facing-assets and informed that we’ve taken them offline and were actively remediating them.”
SecOps Rapid Response Efficiencies
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| B1 | Number of SecOps resources fully dedicated to identifying and triaging exposures for remediation in prior environment | {{Benefit2B1Src}} | {{Benefit2B1CFYear1}} | {{Benefit2B1CFYear2}} | {{Benefit2B1CFYear3}} |
| B2 | Percent reduction in SecOps time spent on exposure response | {{Benefit2B2Src}} | {{Benefit2B2CFYear1}} | {{Benefit2B2CFYear2}} | {{Benefit2B2CFYear3}} |
| B3 | Hours of SecOps response avoided with Randori | {{Benefit2B3Src}} | {{Benefit2B3CFYear1}} | {{Benefit2B3CFYear2}} | {{Benefit2B3CFYear3}} |
| B4 | Average fully burdened hourly rate per security resource | {{Benefit2B4Src}} | {{Benefit2B4CFYear1}} | {{Benefit2B4CFYear2}} | {{Benefit2B4CFYear3}} |
| Bt | SecOps rapid response efficiencies | B3*B4 | {{Benefit2BtCFYear1}} | {{Benefit2BtCFYear2}} | {{Benefit2BtCFYear3}} |
| Risk adjustment | ↓20% | ||||
| Btr | SecOps rapid response efficiencies (risk-adjusted) | {{Benefit2BtrCFYear1}} | {{Benefit2BtrCFYear2}} | {{Benefit2BtrCFYear3}} | |
| Three-year total: {{Benefit2BtrCFTotal}} | Three-year present value: {{Benefit2BtrCFPV}} | ||||
Savings From Augmented Red Team Activities
Evidence and data. Interviewees noted that, particularly when deploying Randori Attack, their SecOps teams were able to reattribute time formerly spent detecting and triaging exposures to more proactive threat hunting, penetration testing, and red/blue/purple team exercises.
- The head of cybersecurity in the professional services industry shared how Randori helped avoid several costs. With the upgrade to Randori Attack, the interviewee avoided $50,000 to $75,000 in annual fees to third-party penetration testing services. Furthermore, the scope of Randori’s penetration testing exceeded the often narrow constraints on traditional tests, “With Randori, they’re just looking for everything.”
- The CISO in the grocery store industry shared that their security team dedicated approximately 25 hours per week spread across two team members to red team activities. With Randori, particularly with the upgrade to Attack, the interviewee reported, “We were able to offload that work to Randori and use those resources to staff a third-party risk management program.”
- The information security manager in the manufacturing industry also noted improved security team collaboration with the Randori investment: “It helps to have a purple team approach, where your red team and your blue team both put on each other’s hats.”
- The security engineer in the entertainment industry shared: “Randori Attack will also scan your internal network, and that is tremendous. Maybe one of our engineers could do that detection work, but because we’re not a red team on a daily basis, it would take me hours and hours to both scan and analyze everything. Instead, we get direct access to their specialist hackers, and that is a huge benefit for us. It’s like having two hackers in your back pocket.”
“I’m a big fan of the purple teaming concept, [and] I’ve been able to run red and blue teams together with Randori.”
Head of cybersecurity, professional services
Modeling and assumptions. The composite organization reaps savings from augmented red team activities as follows:
-
In the prior environment, the composite organization:
- Paid $65,000 in penetration testing fees from a managed service.
The information provided suggests that state may pay Benefit3C1CFYear2 in penetration testing fees from a managed service.
-
- Dedicated 25 hours a week or 1,300 hours annually on red teaming functions. The fully burdened hourly rate of a red team analyst is $100.
According to the information provided, state dedicates inHrsRed hours per week (or Benefit3C2CFYear1 hours annually) to red teaming functions.
-
With Randori:
- The penetration testing fees are avoided altogether when the composite organization upgrades to include Randori Attack.
- Red team resources improve their situational awareness, leading to 10% time savings in Year 1.
- In Years 2 and 3, the composite organization automates mundane and manual aspects of red teaming, driving further time savings of 60% in Year 2 and 75% in Year 3.
Based on the Randori deployment schedule of state and the findings of the TEI case study, state could realize time savings on red teaming of Benefit3C3CFYear1 in Year 1, Benefit3C3CFYear2 in Year 2, and Benefit3C3CFYear3 in Year 3.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:
- Deployment configuration of Randori Recon and Attack. Extended use of the full platform may amplify results, compared to only deploying Randori Recon.
- An organization’s security context, including its security maturity in the prior environment, which will impact the cost of running red teams internally and the amount of penetration testing Randori replaces.
- The number of vulnerable external assets detected and protected with Randori.
- The type of FTEs scanning for vulnerabilities and exposures and the amount of time required per asset for remediation.
Results. To account for these risks, Forrester adjusted this benefit downward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $214,000.
For state, this benefit may have a three-year, risk-adjusted total PV of Benefit3CtrCFPV.
Savings From Augmented Red Team Activities
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| C1 | Avoided penetration testing fees with Randori Attack Targeted | {{Benefit3C1Src}} | {{Benefit3C1CFYear1}} | {{Benefit3C1CFYear2}} | {{Benefit3C1CFYear3}} |
| C2 | Hours dedicated to red teaming functions in prior environment | {{Benefit3C2Src}} | {{Benefit3C2CFYear1}} | {{Benefit3C2CFYear2}} | {{Benefit3C2CFYear3}} |
| C3 | Reduction in manual red-teaming with Randori | {{Benefit3C3Src}} | {{Benefit3C3CFYear1}} | {{Benefit3C3CFYear2}} | {{Benefit3C3CFYear3}} |
| C4 | Fully burdened hourly rate of a red team analyst | {{Benefit3C4Src}} | {{Benefit3C4CFYear1}} | {{Benefit3C4CFYear2}} | {{Benefit3C4CFYear3}} |
| Ct | Savings from augmented red team activities | C1+(C2*C3*C4) | {{Benefit3CtCFYear1}} | {{Benefit3CtCFYear2}} | {{Benefit3CtCFYear3}} |
| Risk adjustment | ↓15% | ||||
| Ctr | Savings from augmented red team activities (risk-adjusted) | {{Benefit3CtrCFYear1}} | {{Benefit3CtrCFYear2}} | {{Benefit3CtrCFYear3}} | |
| Three-year total: {{Benefit3CtrCFTotal}} | Three-year present value: {{Benefit3CtrCFPV}} | ||||
Exposure Analysis Efficiencies
Evidence and data. Compared to their prior environments, interviewees reported between 50% and approximately 95% reductions in manual effort with Randori’s continuous security monitoring.
- The head of cybersecurity in the professional services industry shared that three team members were dedicated to vulnerability scanning: “We were doing traditional vulnerability scanning with tools. It was a very manual process. You scan what you know, and it’s always hard to scan what you don’t know. IPs change all the time, so scanning takes effort. You have to commit to it on a regular basis or the data gets stale. It probably took 20 hours quarterly to run the scan, hold onto certain results, review them, audit them, ask a few people about them. With Randori, the exponential change — from weeks or months to hours — is just a game changer. And showing me what to actually be worried about is a real value-add. Otherwise, I’m making a gut response, running and juggling a thousand other things. Having Randori help prioritize decisions is wildly helpful.”
- The CISO in the grocery store industry discussed the length of time it took to complete vulnerability scanning and prioritization prior to Randori and described how the platform enabled more efficient analyses of exposures. They shared: “Prior to Randori, attack surface management really was our spreadsheets and our vulnerability management tool, including third parties. They would scan our external environment for us, and we would aggregate that data with the results of our internal scans. It would take a four-person team 80 hours to conduct weekly scheduled scans, aggregate the results, identify false positives and true vulnerabilities, and then apply threat intelligence on top of that. With Randori, the team is spending about 50% less time on these efforts.”
- The information security manager in the manufacturing industry shared how their organization approached external attack surface management with Randori: “It’s definitely improved our time to detect things that are not already in our scan configuration. We would try to detect shadow IT using open source intelligence tools. Doing it ourselves with scripts was not as effective, and it took a lot longer. Randori has at least cut the time in half. It actually lets you manipulate and triage the data, so you can investigate a target that looks tempting, assess, and document it in Randori. It allows us to see what we were dismissing because it was not in our general vulnerability scanning. By enabling us, saving us time and finding the targets, we can now spend that extra time peeling back the onion, quantifying the real risk on those assets.”
- The security engineer in the entertainment industry shared: “Randori gives me visibility of what is owned by my organization with a much narrower field to look at. It will constantly monitor targets and make sure that they don’t detect any kind of vulnerability from the external perspective. Even with all the time in the world, I still wouldn’t be able to find this kind of information by myself or from internal teammates. It’s tremendous, the amount of time you can potentially save.”
“We’re no longer just looking at what we know we own. We can now see residual breadcrumbs on the internet of what we potentially own and need to make sure is secure. This freed up time to dig deeper into risks.”
Information security manager, manufacturing
“Randori has dramatically reduced the time it takes to scan our environment. The team spends about 50% less time now because it performs continuous reconnaissance of our external environment for us. Then it prioritizes what we need to care about.”
CISO, grocery store
Modeling and assumptions. The composite improves its exposure analysis effort as follows:
- In the prior environment, SecOps resources conducted 1,040 hours of regular scanning and reporting on exposures.
According to the information provided, SecOps resources at state spend inHrsReg hours per week (or Benefit4D1CFYear1 hours annually) on regular scanning and reporting activities.
- With the Randori platform, the composite organization reduces the number of vulnerability scanning hours needed by 75%, 85%, and 90% in Years 1, 2, and 3, respectively.
Based on the Randori deployment schedule of state and the findings of the TEI case study, state could realize time savings on vulnerability scanning of Benefit4D2CFYear1 in Year 1, Benefit4D2CFYear2 in Year 2, and Benefit4D2CFYear3 in Year 3.
- The average fully burdened hourly rate per security resource is $60.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:
- An organization’s size and security context. including its security maturity, and the complexity of effort dedicated to vulnerability scanning and exposure detection efforts.
- The number of vulnerable external assets detected and protected with Randori.
- The type of FTEs scanning for vulnerabilities and exposures and the amount of time required per asset for remediation.
Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $103,000.
For state, this benefit may have a three-year, risk-adjusted total PV of Benefit4DtrCFPV.
Exposure Analysis Efficiencies
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| D1 | Security operations hours spent on regular scanning and reporting in prior environment | {{Benefit4D1Src}} | {{Benefit4D1CFYear1}} | {{Benefit4D1CFYear2}} | {{Benefit4D1CFYear3}} |
| D2 | Percent reduction in hours of vulnerability scanning per year with Randori | {{Benefit4D2Src}} | {{Benefit4D2CFYear1}} | {{Benefit4D2CFYear2}} | {{Benefit4D2CFYear3}} |
| D3 | Average fully burdened hourly rate per security resource | {{Benefit4D3Src}} | {{Benefit4D3CFYear1}} | {{Benefit4D3CFYear2}} | {{Benefit4D3CFYear3}} |
| Dt | Exposure analysis efficiencies | D1*D2*D3 | {{Benefit4DtCFYear1}} | {{Benefit4DtCFYear2}} | {{Benefit4DtCFYear3}} |
| Risk adjustment | ↓20% | ||||
| Dtr | Exposure analysis efficiencies (risk-adjusted) | {{Benefit4DtrCFYear1}} | {{Benefit4DtrCFYear2}} | {{Benefit4DtrCFYear3}} | |
| Three-year total: {{Benefit4DtrCFTotal}} | Three-year present value: {{Benefit4DtrCFPV}} | ||||
Reduction In Exposure-Related Help Desk Tickets
Evidence and data. With Randori, interviewees had fewer exposures requiring help desk tickets.
- The security engineer in the entertainment industry shared how Randori reduced the volume of exposure-related tickets: “Prior to Randori, we had more tickets. When it first scanned our environment, we dug up a lot of dirt that was buried for years. The number of tickets then became incrementally smaller to a handful per month, because the accuracy of our false positive detection rate got better. Then, going through a vulnerability list of 100 items would take me a couple of days. Randori generates tickets automatically, so I don’t have to do them manually. I just forward them, so my downstream colleague will get to them even quicker.”
- The CISO in the grocery store shared: “Randori is scanning our environment and looking for new assets that have popped up. We try to automate and get alerts when that information is exposed. Randori does a much better job at that than a third-party tool we’ve used before.”
- The information security manager in the manufacturing industry shared: “Randori does a much better job dialing in [false positives] than a third-party tool we’ve used before, so false positives are not really a concern anymore.
“Randori definitely helps a lot with tickets. It’s an order of magnitude better compared to a year ago, whereas I had maybe hundreds, I’m down to tens.”
Security engineer, entertainment
Exposure-related help desk tickets
Before: 10,800
After: 2,880
Modeling and assumptions. With Randori, the composite organization reduces exposure-related help desk tickets as follows:
- In the prior environment, the composite organization processed 300 exposure-related help desk tickets per month, totaling 3,600 exposure-related help desk tickets annually.
According to the information provided, state processes inTickets exposure-related help desk tickets per month (or Benefit5E1CFYear1 tickets annually).
- The composite’s average cost per exposure-related help desk tickets is $30.
- With Randori, the composite organization ramps up to an 85% reduction in exposure-related help desk tickets by the end of the three-year period.
Based on the Randori deployment schedule of state and the findings of the TEI case study, state could realize reductions in exposure-related help desk of Benefit5E2CFYear1 in Year 1, Benefit5E2CFYear2 in Year 2, and Benefit5E2CFYear3 in Year 3.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:
- An organization’s size and security context and maturity, including how an organization’s help desk is organized. Automating efforts with Randori may reduce outsourcing fees or internal labor hours.
- The number of false positives and level of effort required to discern them.
- The total number and cost per exposure-related help desk tickets.
Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $175,000.
For state, this benefit may have a three-year, risk-adjusted total PV of Benefit5EtrCFPV.
Reduction In Exposure-Related Help Desk Tickets
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| E1 | Number of exposure-related IT help desk tickets in prior environment | {{Benefit5E1Src}} | {{Benefit5E1CFYear1}} | {{Benefit5E1CFYear2}} | {{Benefit5E1CFYear3}} |
| E2 | Percent reduction in exposure-related IT help desk tickets with Randori | {{Benefit5E2Src}} | {{Benefit5E2CFYear1}} | {{Benefit5E2CFYear2}} | {{Benefit5E2CFYear3}} |
| E3 | Number of IT tickets avoided with Randori | {{Benefit5E3Src}} | {{Benefit5E3CFYear1}} | {{Benefit5E3CFYear2}} | {{Benefit5E3CFYear3}} |
| E4 | Cost per security-related ticket | {{Benefit5E4Src}} | {{Benefit5E4CFYear1}} | {{Benefit5E4CFYear2}} | {{Benefit5E4CFYear3}} |
| Et | Reduction in exposure-related help desk tickets | E3*E4 | {{Benefit5EtCFYear1}} | {{Benefit5EtCFYear2}} | {{Benefit5EtCFYear3}} |
| Risk adjustment | ↓10% | ||||
| Etr | Reduction in exposure-related help desk tickets (risk-adjusted) | {{Benefit5EtrCFYear1}} | {{Benefit5EtrCFYear2}} | {{Benefit5EtrCFYear3}} | |
| Three-year total: {{Benefit5EtrCFTotal}} | Three-year present value: {{Benefit5EtrCFPV}} | ||||
Avoided Vulnerability Scanning License Fees
Evidence and data. Randori functionalities exceeded the capabilities of their traditional vulnerability scanners. This reduced their reliance on vulnerability scans, and consolidated license costs for legacy vulnerability scanners.
- The CISO in the grocery store industry shared, “We reduced our vulnerability scanner licensing cost by 15% because we use Randori for that same functionality.”
- The head of cybersecurity in the professional services industry shared how Randori helped avoid several legacy license costs: “There is a fee for the [open-source vulnerability scanning] services. It’s a few thousand dollars, so we saved on that front as well.
“Randori sees things that would be lost in the noise of regular vulnerability scanning.”
Information security manager, manufacturing
Modeling and assumptions. The composite organization avoids legacy vulnerability scanning license costs as follows:
- In the prior environment, the composite organization spent $150,000 on vulnerability scanning license fees.
The information provided suggests that state may spend Benefit6F1CFYear1 on vulnerability scanning license fees.
- With Randori, the composite reduces reliance on the legacy vulnerability scanning vendor, decreasing the total cost of vulnerability scanning licenses by 10%, 12% and 15% in Years 1, 2, and 3, respectively.
Based on the findings of the TEI case study, state could realize reductions in vulnerability scanning license fees of Benefit6F2CFYear1 in Year 1, Benefit6F2CFYear2 in Year 2, and Benefit6F2CFYear3 in Year 3.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:
- An organization’s security context, including its security maturity in the prior environment, which will impact the cost of vulnerability scanning license fees.
- The number of vulnerable external assets detected and protected with Randori.
Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $41,000.
For state, this benefit may have a three-year, risk-adjusted total PV of Benefit6FtrCFPV.
Avoided Vulnerability Scanning License Fees
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| F1 | Vulnerability scanning license fees in prior environment | {{Benefit6F1Src}} | {{Benefit6F1CFYear1}} | {{Benefit6F1CFYear2}} | {{Benefit6F1CFYear3}} |
| F2 | Percent reduction in vulnerability scanning license fees with Randori | {{Benefit6F2Src}} | {{Benefit6F2CFYear1}} | {{Benefit6F2CFYear2}} | {{Benefit6F2CFYear3}} |
| Ft | Avoided vulnerability scanning license fees | F1*F2 | {{Benefit6FtCFYear1}} | {{Benefit6FtCFYear2}} | {{Benefit6FtCFYear3}} |
| Risk adjustment | ↓10% | ||||
| Ftr | Avoided vulnerability scanning license fees (risk-adjusted) | {{Benefit6FtrCFYear1}} | {{Benefit6FtrCFYear2}} | {{Benefit6FtrCFYear3}} | |
| Three-year total: {{Benefit6FtrCFTotal}} | Three-year present value: {{Benefit6FtrCFPV}} | ||||
Savings On Cybersecurity Insurance
Evidence and data. The CISO in the grocery store industry reported that their organization reduced cyber insurance premiums with Randori. The interviewees relayed how underpinning their organization’s broader ASM effort program in Randori reduced cyber risk in the eyes of their insurers, lowering their premiums. The interviewee noted, “We just met with our insurance consulting group and received a 22% reduction on our premium, which was unheard of, so they were really impressed.”
“Our attack surface management program with Randori was a big hit with our underwriters. They loved it. It helped us get coverage this year, and another discount from last year.”
CISO, grocery store
Modeling and assumptions. The composite reduces cyber insurance costs as follows:
- In the prior environment, the composite pays $75,000 in annual cyber insurance premiums.
According to the information provided, state pays inPremiums in annual cybersecurity insurance premiums.
-
In Years 2 and 3, the composite organization successfully evolves its vulnerability management focus to an attack surface management approach with the Randori platform. The composite organization’s cyber insurance underwriters recognize and appreciate the value of this advancement in security maturity. This improves its standing with its cyber insurers, contributing to the following savings:
- A 10% decrease in cyber insurance premiums in Year 1.
- A 20% decrease in cyber insurance premiums in Years 2 and 3. The increase is due to the composite organization’s renewed focus on attack surface management, which is further augmented in Years 2 and 3 when the composite organization upgrades to include Attack.
Based on the Randori deployment schedule of state and the findings of the TEI case study, state could realize reductions in cybersecurity insurance premiums of Benefit7G2CFYear1 in Year 1, Benefit7G2CFYear2 in Year 2, and Benefit7G2CFYear3 in Year 3.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:
- An organization’s security context, including its security maturity in the prior environment, which will impact the cost of cyber insurance premiums.
- The number of vulnerable external assets detected and protected with Randori.
Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $27,000.
For state, this benefit may have a three-year, risk-adjusted total PV of Benefit7GtrCFPV.
Savings On Cybersecurity Insurance
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| G1 | Cyber insurance premiums in prior environment | {{Benefit7G1Src}} | {{Benefit7G1CFYear1}} | {{Benefit7G1CFYear2}} | {{Benefit7G1CFYear3}} |
| G2 | Percent reduction in cyber insurance premiums attributed with Randori | {{Benefit7G2Src}} | {{Benefit7G2CFYear1}} | {{Benefit7G2CFYear2}} | {{Benefit7G2CFYear3}} |
| Gt | Savings on cybersecurity insurance | G1*G2 | {{Benefit7GtCFYear1}} | {{Benefit7GtCFYear2}} | {{Benefit7GtCFYear3}} |
| Risk adjustment | ↓10% | ||||
| Gtr | Savings on cybersecurity insurance (risk-adjusted) | {{Benefit7GtrCFYear1}} | {{Benefit7GtrCFYear2}} | {{Benefit7GtrCFYear3}} | |
| Three-year total: {{Benefit7GtrCFTotal}} | Three-year present value: {{Benefit7GtrCFPV}} | ||||
Unquantified Benefits
Interviewees mentioned the following additional benefits that their organizations experienced but were not able to quantify:
Cleaner audits and streamlined mergers and acquisitions (M&A). Randori helped the interviewees gain a better view of their auditing and M&A efforts.
- The information security manager in the manufacturing industry noted: “Randori helps us with cleaner audits. We see things that we might have missed if we didn’t have that broader look that we get from Randori. [Furthermore], when you’re engaged in acquisitions, there’s an auditing element where Randori gives us an additional, predeal view of their external posture to determine the level of investment needed to bring them up to our standard.”
- The CISO in the grocery store industry described complicated compliance requirements: “The way that I manage risk in every product is different. My pharmacy applications are architected completely differently than my point-of-sale solutions. In order to accurately demonstrate how we manage that risk, I need to consider every single part of that product’s unique attack surface. With [payment card industry] (PCI), a significant finding would mean we would be unable to accept credit card transactions in our stores. Our ability to prescribe drugs in our pharmacies would be impacted. Most importantly, any potential security incident would impact our brand and that’s what we can’t have. There’s no larger impact on our work.”
Further time savings and quality improvements from automation. Randori allowed interviewees to improve consistency while saving time through automated policies to accept a certain level of risk for predetermined applications.
- The information security manager in the manufacturing industry shared: “Policy automation just saves us time. The policies tool allows you to take automated action on the data. When we’re implementing things where we know we’re going to accept a certain level of risk, we can have the tool do some automation and accept the risk when it runs into those items. Then, we periodically audit them.”
- The CISO in the grocery store industry noted time savings related to automating port scanning and certificate searches.
More collaborative, informed approach to risk. Improved security team collaboration led some interviewees to observe their organizations assume a stronger and more informed risk posture.
- The security engineer in the entertainment industry shared how Randori helped security professionals encourage developers to factor the development of secure code earlier in the software lifecycle: “Having visibility lets us take a lot of preemptive action. [Prior], we would have to wait until something happened to fix it. With visibility from Randori Recon detecting from the outside and also using Randori Attack scanning software internally, they help us show developers how to be more actionable on security items.”
- The information security manager in the manufacturing industry pointed to an improved risk posture resulting from the Randori investment: “It helps our folks see the full scope of risk in different layers, versus just operating system risk. Things like an old copyright are really an indication of a neglected system. That causes an attacker to spend more time poking at it. The platform has different tiers of privileges with an observer role that we use for directors and managers to get a sense of the attacker’s view. When you have more folks involved in the data, you get different lenses. By opening it up to broader expertise, it’s like you’re crowdsourcing.”
Upskilling for newer hires. Randori’s ability to provide the perspective of an attacker helped some interviewees improve their defensive abilities.
- The information security manager in the manufacturing industry shared: “You need to know your enemy to be able to defend well. Randori has been really helpful upskilling folks that are new to the positions. They may be fresh into the industry and haven’t had to think like an attacker. The whole concept of a target temptation helps you to think differently. Going into Randori and having the data presented to you from the attacker’s view just helps you become a better defender.”
Flexibility
The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement Randori and later realize additional uses and business opportunities, including security tool validation and optimized security stack.
With Randori’s multiple functions, some interviewees could eliminate overlapping software license and maintenance costs while streamlining and validating the other tools in their tech stack.
- The information security manager in the manufacturing industry indicated: “It’s always good to have a second and third source of information to validate whether something is truly an issue or maybe a false positive. We use Randori to validate across our scanning tools. If we believe we have a false positive or negative in one of our tools, Randori helps provide that additional lens, and builds our confidence in the quality of what we’re seeing.”
- The head of cybersecurity in the professional services industry shared similar sentiments: “Sometimes you want to validate your vendors. When my team is doing a security evaluation, we pump data into Randori to see if it thinks there’s a high risk on that device as well. It really gives you that full context. With Randori Attack, we were able to find a device that wasn’t properly managed. That was a good test. I can sell that same value proposition to the CEO and say that we have at least one other data point showing our tools are working as expected.”
- The security engineer in the entertainment industry shared: “Randori Attack detected a vulnerability on one of our internal servers, and they deployed an implant to show it was indeed something that [attackers] could get to, not just a theoretical thing, but an actual, real incident.”
Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Appendix A).
Quantified cost data as applied to the composite
Total Costs
| Ref. | Cost | Initial | Year 1 | Year 2 | Year 3 | Total | Present Value |
|---|---|---|---|---|---|---|---|
| Htr | Randori subscription fees | {{Cost1HtrCFYear0}} | {{Cost1HtrCFYear1}} | {{Cost1HtrCFYear2}} | {{Cost1HtrCFYear3}} | {{Cost1HtrCFTotal}} | {{Cost1HtrCFPV}} |
| Itr | Deployment and administration | {{Cost2ItrCFYear0}} | {{Cost2ItrCFYear1}} | {{Cost2ItrCFYear2}} | {{Cost2ItrCFYear3}} | {{Cost2ItrCFTotal}} | {{Cost2ItrCFPV}} |
| Total costs (risk-adjusted) | {{CostTotalCFYear0}} | {{CostTotalCFYear1}} | {{CostTotalCFYear2}} | {{CostTotalCFYear3}} | {{CostTotalCFTotal}} | {{CostTotalCFPV}} |
Randori Subscription Fees
Evidence and data. All interviewees’ engagement with Randori began with the Recon platform. Three then later upgraded to include Randori Attack, while the fourth interviewee was in the process of considering the Attack platform.
Modeling and assumptions. The composite deploys Recon in Year 1 and includes Attack in Years 2 and 3. Its fees are based on the number of knowledge workers it maintains, averaging 10,000 over the three-year period.
Pricing may vary. Contact Randori for additional details.
The information provided suggests that state might pay Randori subscription fees of Cost1H1CFYear1 in Year 1, Cost1H1CFYear2 in Year 2, and Cost1H1CFYear3 in Year 3. These estimates are automated and do NOT constitute a quote. Pricing may vary. Contact Randori for additional details.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this cost:
- An organization’s subscription model and deployment of one or both Randori Recon and Attack platforms.
- The number of an organization’s knowledge workers, which may fluctuate.
Results. To account for these risks, Forrester adjusted this cost upward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $621,000.
For state, this cost may have a three-year, risk-adjusted total PV of Cost1HtrCFPV .
Randori Subscription Fees
| Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|---|
| H1 | Randori subscription fees | {{Cost1H1Src}} | {{Cost1H1CFYear0}} | {{Cost1H1CFYear1}} | {{Cost1H1CFYear2}} | {{Cost1H1CFYear3}} |
| Ht | Randori subscription fees | H1 | {{Cost1HtCFYear0}} | {{Cost1HtCFYear1}} | {{Cost1HtCFYear2}} | {{Cost1HtCFYear3}} |
| Risk adjustment | ↑10% | |||||
| Htr | Randori subscription fees (risk-adjusted) | {{Cost1HtrCFYear0}} | {{Cost1HtrCFYear1}} | {{Cost1HtrCFYear2}} | {{Cost1HtrCFYear3}} | |
| Three-year total: {{Cost1HtrCFTotal}} | Three-year present value: {{Cost1HtrCFPV}} | |||||
Deployment And Administration
Evidence and data. Interviewees described various experiences deploying and administering Randori.
Deployment. Interviewees shared that the initial deployment process for Randori Recon took approximately one month with Attack coming online in later years for three interviewees.
- The CISO in the grocery store industry estimated that their organization invested more effort deploying the Attack solution compared to Recon: “Recon stood up very quickly without really much on our part. We required more resources for Attack, the pen testing portion.”
- The security engineer in the entertainment industry specified the deployment timeline and steps: “It could be over a course of a month. You basically give Randori your IP address, your domain name, and some other information, then they put it in the system and automate a scanning process. In a day, you learn the tool pretty quickly with an initial meeting and whatever help you need on the spot. Randori is very responsive.”
- The information security manager in the manufacturing industry shared: “Being a SaaS platform, there really wasn’t much we had to do to be initially up and running out of the gates. It took about 8 hours of tuning of the assets to engage with the data. Once you get in it, Randori does a great job of shepherding you into success at the beginning of the engagement which is great. Other tools may take quite a bit longer to get full value out of them.”
Administration. Administrative costs for interviewees related to ongoing system administration and calibration. These costs also increased when the three interviewees added Attack to their Randori Recon configuration.
- The security engineer in the entertainment industry described their organization’s monthly meetings with Randori to maintain the solution: “It’s a monthly, 1-hour meeting. On my team, usually the manager and one of my colleagues would join in, so there is two to three of us on our side. We’d go through two parts: the Recon part and then the Attack report for the month.”
- The CISO in the grocery store industry shared: “There’s not a lot to administering the Recon tool, maybe 3 to 5 hours, [but] there’s more to administering Attack. I have two team members who are responsible for Randori, both Recon, the external attack surface management piece, and also Attack.”
“The ramp up to full value is pretty fast. Within a month, we were confident with how to engage the tool.”
Information security manager, manufacturing
Modeling and assumptions. Forrester assumes the following for the composite organization:
-
The initial period to full deployment of Randori Recon takes one month. During this time, the composite commits:
- 20 hours of security resource time for technical deployment and product tuning.
- 9 security resources to participate in 4 hours of training each.
The information provided suggests that during the initial Randori Recon deployment period, state might commit Cost2I1CFYear0 hours of security resource time for technical deployment and product tuning as well as Cost2I2CFYear0 security resources to participate in 4 hours of training each.
-
The composite devotes the following number of security resource hours to Randori monthly:
- 12 hours to system administration in Year one, doubling to 24 hours in Years 2 and 3 after the Attack upgrade.
- 24 hours to calibration in Year 1, growing to 36 hours in Years 2 and 3 with Attack.
The information provided suggests that state might devote the following number of security resource hours to Randori monthly: system administration time of Cost2I6CFYear1 hours in Year 1, Cost2I6CFYear2 hours in Year 2, and Cost2I6CFYear3 hours in Year 3; calibration time of Cost2I7CFYear1 hours in Year 1, Cost2I7CFYear2 hours in Year 2, and Cost2I7CFYear3 hours in Year 3.
- The average fully burdened hourly rate per security resource involved in Randori deployment and administration is $60.
Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this cost:
- An organization’s subscription model and deployment of one or both Randori Recon and Attack platforms.
- The amount of employee time and effort required for Randori deployment.
- The number and type of employees required to help Randori with deployment.
- The level or effort an organization chooses to invest in product administration, including regular meetings with Randori.
Results. To account for these risks, Forrester adjusted this cost upward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $13,000.
For state, this cost may have a three-year, risk-adjusted total PV of Cost2ItrCFPV.
Deployment And Administration
| Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|---|
| I1 | Total of hours of technical deployment and product tuning | {{Cost2I1Src}} | {{Cost2I1CFYear0}} | {{Cost2I1CFYear1}} | {{Cost2I1CFYear2}} | {{Cost2I1CFYear3}} |
| I2 | Number of deployment security resources involved in deployment | {{Cost2I2Src}} | {{Cost2I2CFYear0}} | {{Cost2I2CFYear1}} | {{Cost2I2CFYear2}} | {{Cost2I2CFYear3}} |
| I3 | Number of hours of training per security resource | {{Cost2I3Src}} | {{Cost2I3CFYear0}} | {{Cost2I3CFYear1}} | {{Cost2I3CFYear2}} | {{Cost2I3CFYear3}} |
| I4 | Average fully burdened hourly rate per security resource | {{Cost2I4Src}} | {{Cost2I4CFYear0}} | {{Cost2I4CFYear1}} | {{Cost2I4CFYear2}} | {{Cost2I4CFYear3}} |
| I5 | Subtotal: Deployment and training | (I1+(I2*I3))*I4 | {{Cost2I5CFYear0}} | {{Cost2I5CFYear1}} | {{Cost2I5CFYear2}} | {{Cost2I5CFYear3}} |
| I6 | Hours of monthly system administration | {{Cost2I6Src}} | {{Cost2I6CFYear0}} | {{Cost2I6CFYear1}} | {{Cost2I6CFYear2}} | {{Cost2I6CFYear3}} |
| I7 | Hours of monthly calibration with Randori | {{Cost2I7Src}} | {{Cost2I7CFYear0}} | {{Cost2I7CFYear1}} | {{Cost2I7CFYear2}} | {{Cost2I7CFYear3}} |
| I8 | Average fully burdened hourly rate per security resource | {{Cost2I8Src}} | {{Cost2I8CFYear0}} | {{Cost2I8CFYear1}} | {{Cost2I8CFYear2}} | {{Cost2I8CFYear3}} |
| I9 | Subtotal: Product administration | (I6+I7)*I8 | {{Cost2I9CFYear0}} | {{Cost2I9CFYear1}} | {{Cost2I9CFYear2}} | {{Cost2I9CFYear3}} |
| It | Deployment and administration | I5+I9 | {{Cost2ItCFYear0}} | {{Cost2ItCFYear1}} | {{Cost2ItCFYear2}} | {{Cost2ItCFYear3}} |
| Risk adjustment | ↑15% | |||||
| Itr | Deployment and administration (risk-adjusted) | {{Cost2ItrCFYear0}} | {{Cost2ItrCFYear1}} | {{Cost2ItrCFYear2}} | {{Cost2ItrCFYear3}} | |
| Three-year total: {{Cost2ItrCFTotal}} | Three-year present value: {{Cost2ItrCFPV}} | |||||
-
The financial results calculated in the Benefits and Costs sections can be used to determine the ROI, NPV, and payback period for the composite organization’s investment. Forrester assumes a yearly discount rate of 10% for this analysis.
These risk-adjusted ROI, NPV, and payback period values are determined by applying risk-adjustment factors to the unadjusted results in each Benefit and Cost section.
Cash Flow Chart
Total costs Total benefits Cumulative net benefitsCash Flow Analysis (Risk-Adjusted Estimates)
Appendix A: Total Economic Impact
Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.
Total Economic Impact Approach
-
Benefits represent the value delivered to the business by the product. The TEI methodology places equal weight on the measure of benefits and the measure of costs, allowing for a full examination of the effect of the technology on the entire organization.
-
Costs consider all expenses necessary to deliver the proposed value, or benefits, of the product. The cost category within TEI captures incremental costs over the existing environment for ongoing costs associated with the solution.
-
Flexibility represents the strategic value that can be obtained for some future additional investment building on top of the initial investment already made. Having the ability to capture that benefit has a PV that can be estimated.
-
Risks measure the uncertainty of benefit and cost estimates given: 1) the likelihood that estimates will meet original projections and 2) the likelihood that estimates will be tracked over time. TEI risk factors are based on “triangular distribution.”
-
Present Value (PV)
The present or current value of (discounted) cost and benefit estimates given at an interest rate (the discount rate). The PV of costs and benefits feed into the total NPV of cash flows.
-
Net Present Value (NPV)
The present or current value of (discounted) future net cash flows given an interest rate (the discount rate). A positive project NPV normally indicates that the investment should be made unless other projects have higher NPVs.
-
Return on investment (ROI)
A project’s expected return in percentage terms. ROI is calculated by dividing net benefits (benefits less costs) by costs.
-
Discount rate
The interest rate used in cash flow analysis to take into account the time value of money. Organizations typically use discount rates between 8% and 16%.
-
Payback period
The breakeven point for an investment. This is the point in time at which net benefits (benefits minus costs) equal initial investment or cost.
The initial investment column contains costs incurred at “time 0” or at the beginning of Year 1 that are not discounted. All other cash flows are discounted using the discount rate at the end of the year. PV calculations are calculated for each total cost and benefit estimate. NPV calculations in the summary tables are the sum of the initial investment and the discounted cash flows in each year. Sums and present value calculations of the Total Benefits, Total Costs, and Cash Flow tables may not exactly add up, as some rounding may occur.
Appendix B: Endnotes
1 Source: Forrester’s Security Survey, 2022.
2 Randori Recon is the main platform included in all subscriptions. Randori Attack can be added onto the Recon platform for additional capabilities in penetration test and internal attack surface reconnaissance.
3 Source: “Find And Cover Your Assets With Attack Surface Management,” Forrester Research, Inc. January 6, 2022
4 Source: “Top Cybersecurity Threats In 2023,” Forrester Research, Inc., April 16, 2023.
5 Ibid.
ABOUT FORRESTER CONSULTING
Forrester provides independent and objective research-based consulting to help leaders deliver key transformation outcomes. Fueled by our customer-obsessed research, Forrester’s seasoned consultants partner with leaders to execute on their priorities using a unique engagement model that tailors to diverse needs and ensures lasting impact. For more information, visit forrester.com/consulting.
© Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies.
PDF
Cookie Preferences
Accept Cookies
Decline
Close
This website uses cookies to deliver functionality and enhance your experience. GDPR requires that we obtain your consent before activating these cookies. Please accept the use of cookies or review your cookie settings now.
A cookie is a small text file that a website saves on your computer or mobile device when you visit the site. It enables the website to remember your actions (data inputs, website navigation), so you don’t have to re-enter data when you come back to the site or browse from one page to another.
Behavioral information collected by our web analytics vendor is used to analyze data pertaining to visitor trends, plan website enhancements, and measure overall website effectiveness. We may also use cookies or web beacons to help us offer you products, programs, or services that may be of interest to you and to deliver relevant advertising. We may use third-party advertising companies to help tailor website content to users or to serve ads on our behalf. These companies may also employ cookies and web beacons to measure advertising effectiveness.
Please accept cookies and the collection of behavioral information to receive full functionality and enhance your experience. If you decline cookies, some features of the website may not function normally.
Please see our
Privacy Policy for more information.