A Forrester Total Economic Impact™ Study Commissioned By Cisco, May 2024
The availability of cloud computing has led to on-prem migration efforts and an exponential explosion of telemetry data across hybrid cloud environments.1 Observability enables organizations to gain valuable insights into a system’s internal states and behaviors, which can empower them to take proactive measures and enhance their understanding of performance. As infrastructure and business-critical applications become increasingly complex, organizations are leveraging AIOps and observability to prevent incidents and drive value within their operations. Through the deployment of Cisco Full-Stack Observability, organizations can reduce costs associated with tool consolidation, improve system performance resulting in fewer outages, decrease mean time to resolve (MTTR) issues, and avert revenue loss.
Cisco Full-Stack Observability (Cisco FSO) is an observability portfolio that includes Cisco AppDynamics, Cisco ThousandEyes, and a multitude of supportive services. As a comprehensive monitoring and observability suite, it is tailored for complex IT environments and designed to provide high-performance IT organizations with complete visibility and insights across their infrastructure stack including applications, networks, servers, and cloud environments. AppDynamics serves as an application performance monitoring (APM) and IT operations analytics (ITOA) platform, which enables proactive resolution of application performance issues across the entire enterprise. Additionally, because ThousandEyes serves as the network intelligence for Cisco FSO, it offers valuable insights into the performance and security of an organization’s digital infrastructure. By leveraging Cisco Full-Stack Observability, organizations can gain a comprehensive understanding of their IT infrastructure with contextual insights, ensuring the seamless functioning of their applications and services.
Cisco commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study and examine the potential return on investment (ROI) enterprises may realize by deploying a Cisco Full-Stack Observability solution.2 The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of Cisco Full-Stack Observability on their organizations.
To better understand the benefits, costs, and risks associated with this investment, Forrester interviewed four representatives with experience using Cisco FSO. For the purposes of this study, Forrester aggregated the interviewees’ experiences and combined the results into a single composite organization that is a retail organization with 50,000 employees and revenue of $20 billion per year. The composite organization has a hybrid infrastructure environment, and it monitors 125 applications with Cisco FSO solutions.
has in annual revenue and 0 apps to be monitored by Cisco FSO solutions. Custom results are based on these inputs and the TEI case study.
Interviewees said that prior to using Cisco FSO, their organizations lacked visibility into siloed people, processes, and technology and that they often could not determine if something was a root cause or a system issue when encountering an incident, including major outages. As a result, when various teams gathered in war-room environments without a means to identify the issue properly and efficiently, the default mode of operation was finger-pointing. Moreover, the interviewees said their organizations relied on customer reports to become aware of service problems and outages, which risked brand reputation and customer retention even more. The migration to the cloud further exacerbated the problem because visibility into on-premises systems was lost. Previous attempts to address these issues had limited success, which left the organizations with complex IT problems, high costs, unsustainable processes, and lost revenue.
The interviewees said that after the investment in Cisco FSO, their organizations experienced significant improvements. They shared that they now have comprehensive visibility into their infrastructure, applications, and services. Moreover, the investment in Cisco FSO yielded positive outcomes, such as consolidating legacy systems, increasing efficiency in the war room, avoiding revenue loss, and increasing team productivity.
Quantified benefits. Three-year, risk-adjusted present value (PV) quantified benefits for the composite organization include:
might save by eliminating tool sprawl and streamlining maintenance processes.
might save from improved incident prevention and decreased MTTR.
might save due to reductions in the likelihood and length of revenue-impacting outages.
might save with staff productivity increases enabled through Cisco certifications.
Unquantified benefits. Benefits that provide value for the composite organization but are not quantified for this study include:
Costs. Three-year, risk-adjusted PV costs for the composite organization include:
For , Cisco FSO licensing and hybrid environment costs might be .
For , AppDynamics implementation and ongoing management costs might be .
For , ThousandEyes implementation and ongoing support costs might be .
For , Cisco FSO training costs might be .
The representative interviews and financial analysis found that a composite organization experiences benefits of $56.80 million over three years versus costs of $12.40 million, adding up to a net present value (NPV) of $44.40 million and an ROI of 358%.
might experience benefits of over three years versus costs of , adding up to an NPV of and an ROI of 0%.
Return on investment (ROI)
Benefits PV
Net present value (NPV)
Payback
From the information provided in the interviews, Forrester constructed a Total Economic Impact™ framework for those organizations considering an investment in a Cisco Full-Stack Observability solution.
The objective of the framework is to identify the cost, benefit, flexibility, and risk factors that affect the investment decision. Forrester took a multistep approach to evaluate the impact that Cisco Full-Stack Observability can have on an organization.
Interviewed Cisco stakeholders and Forrester analysts to gather data relative to the Cisco Full-Stack Observability portfolio.
Interviewed four representatives at organizations using Cisco Full-Stack Observability to obtain data about costs, benefits, and risks.
Designed a composite organization based on characteristics of the interviewees’ organizations.
Constructed a financial model representative of the interviews using the TEI methodology and risk-adjusted the financial model based on issues and concerns of the interviewees.
Employed four fundamental elements of TEI in modeling the investment impact: benefits, costs, flexibility, and risks. Given the increasing sophistication of ROI analyses related to IT investments, Forrester’s TEI methodology provides a complete picture of the total economic impact of purchase decisions. Please see Appendix A for additional information on the TEI methodology.
Readers should be aware of the following:
This study is commissioned by Cisco and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.
Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the study to determine the appropriateness of an investment in a Cisco Full-Stack Observability solution. For the interactive functionality using Configure Data/Custom Data, the intent is for the questions to solicit inputs specific to a prospect's business. Forrester believes that this analysis is representative of what companies may achieve with Full-Stack Observability based on the inputs provided and any assumptions made. Forrester does not endorse Cisco or its offerings. Although great care has been taken to ensure the accuracy and completeness of this model, Cisco and Forrester Research are unable to accept any legal responsibility for any actions taken on the basis of the information contained herein. The interactive tool is provided ‘AS IS,’ and Forrester and Cisco make no warranties of any kind.
Cisco reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.
Cisco provided the customer names for the interviews but did not participate in the interviews.
Consulting Team:
Stephanie Slate
Role | Industry | Revenue | Number of employees |
---|---|---|---|
Manager of application performance | Retail | $1B | 5,500 |
Program manager | Government administration | $21B | 35,000 |
Director of infrastructure and operations | Hospitality | $14B | 85,000 |
VP of enterprise command center | Financial services | $2.2B | 5,520 |
While the internet existed long before the COVID-19 pandemic, the crisis solidified the shift for even the most hesitant of consumers to use the internet and trust it for all their needs. Now, the constant expectation for seamless digital experiences has amplified the need for providers to swiftly respond to operational disruptions. If brands cannot meet customers’ demands, these disruptions not only mean operational losses and brand damage, but also potential concerns for public health and safety.3 As more companies push technology to their limit, failure is more complex to understand and troubleshoot as teams try to make sense of the technology and the collective understanding for which employees are needed for triage and resolution.
Interviewees said the high volume of system issues and outages without the proper visibility had a detrimental impact on business continuity at their organizations, which caused teams to work in an unstructured reactive state instead of a proactive stable IT environment.
The interviewees noted how their organizations struggled with common challenges, including:
The interviewees’ organizations searched for a solution that could:
After a request for proposal (RFP) and business case process evaluating multiple vendors, the interviewees’ organizations chose the Cisco Full-Stack Observability portfolio and began deployment. Each organization deployed ThousandEyes and AppDynamics together to gain the full stack observability into infrastructure and applications. And they took multiple steps to implement the services. During the POC stage, a small percentage of the infrastructure and applications integrated with the AppDynamics component of the Cisco FSO solution. Full integration followed in years 2 and 3.
Based on the interviews, Forrester constructed a TEI framework, a composite company, and an ROI analysis that illustrates the areas financially affected. The composite organization is representative of the four interviewees, and it is used to present the aggregate financial analysis in the next section. The composite organization has the following characteristics:
Description of composite. The composite is a global, multibillion-dollar retail organization that provides sales, customer support, and service/warranty support for its consumer products in high volume. The organization has a strong brand, a large customer base, and a strong online and offline presence. It has 50,000 employees, a hybrid IT infrastructure, and generates annual revenue of $20 billion. As a result of the COVID-19 pandemic, more of the composite’s customers rely on the organization’s digital experiences to interact with the brand. As a result, the composite has undertaken a digital transformation to improve the experiences of internal users and customers because they are often the ones who alert the organization when there are issues with the website. Executive leadership demands more visibility into the environment to ensure that all systems are running smoothly and that teams are working as efficiently and productively as possible.
Deployment characteristics. The composite organization opts to complete a proof of concept on AppDynamics and installs five critical applications onto the solution. With the plan to have 125 applications total, it rolls out the applications in phases: 31 applications by the end of in Year 1, 63 by the end of Year 2, and 125 by the end of Year 3. The composite organization prioritizes fully deploying ThousandEyes in Year 1 and covers the full infrastructure during the implementation phase. The goal is to ensure optimal productivity for noncustomer-facing employees.
Ref. | Benefit | Year 1 | Year 2 | Year 3 | Total | Present Value |
---|---|---|---|---|---|---|
Atr | Cost savings from eliminated tool sprawl and maintenance | $6,630,000 $6,630,000 | $7,956,000 $7,956,000 | $8,840,000 $8,840,000 | $23,426,000 $23,426,000 | $19,244,102 $19,244,102 |
Btr | Labor savings from incident prevention and accelerated MTTR | $2,465,190 $2,465,190 | $5,035,928 $5,035,928 | $10,021,883 $10,021,883 | $17,523,000 $17,523,000 | $13,932,594 $13,932,594 |
Ctr | Avoided revenue loss from outage prevention | $5,190,324 $5,190,324 | $9,308,733 $9,308,733 | $13,539,975 $13,539,975 | $28,039,032 $28,039,032 | $22,584,428 $22,584,428 |
Dtr | Internal resource efficiency and productivity savings | $350,280 $350,280 | $437,850 $437,850 | $525,420 $525,420 | $1,313,550 $1,313,550 | $1,075,052 $1,075,052 |
Total benefits (risk-adjusted) | $14,635,794 $14,635,794 | $22,738,510 $22,738,510 | $32,927,278 $32,927,278 | $70,301,582 $70,301,582 | $56,836,176 $56,836,176 | |
Evidence and data. Among the four interviewees, three said their organization relied on legacy tools to monitor its infrastructure and applications. These applications were not functioning properly and, as solutions migrated to the cloud, the organizations lost the visibility they once had on-premises. In light of this, the mix of legacy, cloud, on-premises, and hybrid meant that teams struggled to get a handle on things, which seemed unsustainable. With the implementation of Cisco FSO, the interviewees’ organizations consolidated their disparate legacy systems and eliminated technical debt in their environments.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
has 0 legacy tools for monitoring IT infrastructure and applications.
might retire 0 tools in Year 1, 0 more tools in Year 2 (totaling 0 tools retired), and 0 more tools in Year 3 (totaling 0 tools retired.
For , the average annual licensing cost per tool is $0. might spend another $0 on maintenance for each tool.
Risks. Forrester recognizes that these results may not be representative of all experiences and that the benefit will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this benefit downward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $19.2 million.
For , this benefit might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Year 1 | Year 2 | Year 3 | |
---|---|---|---|---|---|---|
A1 | Tools to observe, monitor, operate, etc. in prior environment | CompositeComposite | 3030 | 3030 | 3030 | |
A2 | Percentage consolidation of tools with Cisco FSO | InterviewsInterviews | 50%50% | 60%60% | 65%65% | |
A3 | Avoided duplicative observe, operate, and monitor applications (tool sprawl) with Cisco FSO (rounded) | A1*A2 | 1515 | 1818 | 2020 | |
A4 | Average annual licensing cost per observe, monitor, operate, etc. applications | CompositeComposite | $500,000 $500,000 | $500,000 $500,000 | $500,000 $500,000 | |
A5 | Subtotal: Tool-sprawl licensing avoided with Cisco FSO | A3*A4 | $7,500,000 $7,500,000 | $9,000,000 $9,000,000 | $10,000,000 $10,000,000 | |
A6 | Average infrastructure and maintenance cost per application | CompositeComposite | $20,000 $20,000 | $20,000 $20,000 | $20,000 $20,000 | |
A7 | Subtotal: Infrastructure savings from eliminated tools with Cisco FSO | A3*A6 | $300,000 $300,000 | $360,000 $360,000 | $400,000 $400,000 | |
At | Cost savings from eliminated tool sprawl and maintenance | A5+A7 | $7,800,000 $7,800,000 | $9,360,000 $9,360,000 | $10,400,000 $10,400,000 | |
Risk adjustment | ↓15% | |||||
Atr | Cost savings from eliminated tool sprawl and maintenance (risk-adjusted) | $6,630,000 $6,630,000 | $7,956,000 $7,956,000 | $8,840,000 $8,840,000 | ||
Three-year total: $23,426,000 $23,426,000 | Three-year present value: $19,244,102 $19,244,102 |
Evidence and data. Regardless of their organization’s industry and size, each of the interviewees said their company significantly reduced outages and MTTR following the implementation of Cisco FSO.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
has 0 business-critical applications in total. By the end of Year 1, 0 applications might be on AppDynamics. By the end of Year 2, 0 applications might be on AppDynamics. And by the end of Year 3, 0 applications might be on AppDynamics. On average, each application has 0 incidents per year.
Without Cisco FSO, resolving an incident at involves 0 staff engineers and might involve 0 managers and 0 executives.
With Cisco FSO, resolving an incident at might involve only 0 manager(s).
Risks. Forrester recognizes that these results may not be representative of all experiences and that the benefit will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this benefit downward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $13.9 million.
For , this benefit might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Year 1 | Year 2 | Year 3 | |
---|---|---|---|---|---|---|
B1 | Critical, Tier 1, and Tier 2 applications | CompositeComposite | 125125 | 125125 | 125125 | |
B2 | Percent of total applications implemented on Cisco FSO | CompositeComposite | 25%25% | 50%50% | 100%100% | |
B3 | Critical, Tier 1, and Tier 2 applications migrated to Cisco FSO (rounded) | B1*B2 | 3131 | 6363 | 125125 | |
B4 | Average application performance incidents per application production environment | CompositeComposite | 66 | 66 | 66 | |
B5 | Staff members actively involved with triage and resolution for critical systems in the prior environment (rounded) | B1*25%B1*25% | 3131 | 3131 | 3131 | |
B6 | Time spent by staff per incident resolution in the prior environment (hours) | InterviewsInterviews | 66 | 66 | 66 | |
B7 | Average fully burdened hourly rate for a triage staff member | TEI standardTEI standard | $78 $78 | $78 $78 | $78 $78 | |
B8 | Managers actively involved with triage and resolution in the prior environment (rounded) | B5*10% | 33 | 33 | 33 | |
B9 | Time spent by managers per incident resolution in the prior environment (hours) | InterviewsInterviews | 44 | 44 | 44 | |
B10 | Average fully burdened hourly rate for a manager | CompositeComposite | $94 $94 | $94 $94 | $94 $94 | |
B11 | Executive leaders actively involved with triage and resolution in the prior environment | CompositeComposite | 22 | 22 | 22 | |
B12 | Time spent by leadership per incident resolution in the prior environment (hours) | InterviewsInterviews | 11 | 11 | 11 | |
B13 | Average fully burdened hourly rate for an executive leader | TEI standardTEI standard | $136 $136 | $136 $136 | $136 $136 | |
B14 | Subtotal: IT operations and leadership labor costs in the prior environment | B3*B4*((B5*B6*B7)+(B8*B9*B10)+(B11*B12*B13)) | $2,958,888 $2,958,888 | $6,013,224 $6,013,224 | $11,931,000 $11,931,000 | |
B15 | Applications implemented with Cisco FSO (rounded) | B3 | 3131 | 6363 | 125125 | |
B16 | Issues per application with Cisco FSO accounting for a reduction in production issues and false positives | B4*(100%-50%) | 33 | 33 | 33 | |
B17 | Staff members involved with triage and resolution with Cisco FSO solutions (rounded) | B5*(100%-80%) | 66 | 66 | 66 | |
B18 | Time spent by staff for triage and resolution per incident with Cisco FSO solutions (hours) | B6*(100%-90%) | 0.60.6 | 0.60.6 | 0.60.6 | |
B19 | Managers needed for triage and resolution per incident with Cisco FSO solutions (rounded) | InterviewsInterviews | 11 | 11 | 11 | |
B20 | Time spent by managers for triage per incident with Cisco FSO solutions (hours) | Y1: B9*(100%-25%); Y2: B9*(100%-50%); Y3: B9*(100%-75%) | 3.03.0 | 2.02.0 | 1.01.0 | |
B21 | Executive leaders needed for triage and resolution with Cisco FSO solutions (rounded) | InterviewsInterviews | 11 | 00 | 00 | |
B22 | Time spent by executive leaders on triage and resolution with Cisco FSO solutions (hours) (rounded) | Y1: B12*(100%-50%); Y2: B12*(100%-100%); Y3: B12*(100%-100%) | 0.50.5 | 0.00.0 | 0.00.0 | |
B23 | Subtotal: IT operations and leadership labor for triage as Cisco FSO Solutions are deployed | B13*B14*((B15*B16*B5)+(B17*B18*B8)+(B19*B20*B11)) | $58,664 $58,664 | $88,603 $88,603 | $140,550 $140,550 | |
Bt | Labor savings from incident prevention and accelerated MTTR | B14-B23 | $2,900,224 $2,900,224 | $5,924,621 $5,924,621 | $11,790,450 $11,790,450 | |
Risk adjustment | ↓15% | |||||
Btr | Labor savings from incident prevention and accelerated MTTR (risk-adjusted) | $2,465,190 $2,465,190 | $5,035,928 $5,035,928 | $10,021,883 $10,021,883 | ||
Three-year total: $17,523,000 $17,523,000 | Three-year present value: $13,932,594 $13,932,594 |
Evidence and data. Interviewees said that as a result of implementing Cisco FSO, their organizations saw a decrease in the number of major outages, and that improved the reliability of their systems and services. This significant improvement had a direct impact on revenue generation and other key business outcomes.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
Without Cisco FSO, has 0 outages per business-critical application per year, with each outage lasting for 0 hours.
has in annual revenue.
Risks. Forrester recognizes that these results may not be representative of all experiences and that the benefit will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $22.5 million.
For , this benefit might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Year 1 | Year 2 | Year 3 | |
---|---|---|---|---|---|---|
C1 | Applications that directly drive revenue | B1*10%B1*10% | 1313 | 1313 | 1313 | |
C2 | Major outages in the prior environment | CompositeComposite | 66 | 66 | 66 | |
C3 | Average outage time in prior environment (hours) | InterviewsInterviews | 44 | 44 | 44 | |
C4 | Annual revenue | CompositeComposite | $20,000,000,000 $20,000,000,000 | $20,000,000,000 $20,000,000,000 | $20,000,000,000 $20,000,000,000 | |
C5 | Percentage of revenue affected by a major outage | CompositeComposite | 10%10% | 10%10% | 10%10% | |
C6 | Average revenue lost per hour during an outage (rounded) | C4*C5/8,760 hours | $228,311 $228,311 | $228,311 $228,311 | $228,311 $228,311 | |
C7 | Operating profit margin | CompositeComposite | 29.7%29.7% | 29.7%29.7% | 29.7%29.7% | |
C8 | Subtotal: Revenue lost in prior environment due to outages | C1*C2*C3*C6*C7 | $21,156,211 $21,156,211 | $21,156,211 $21,156,211 | $21,156,211 $21,156,211 | |
C9 | Reduction in the likelihood of an outage due Cisco FSO | InterviewsInterviews | 13%13% | 25%25% | 50%50% | |
C10 | Outages with Cisco FSO | C2*(100%-C9) | 5.25.2 | 4.54.5 | 3.03.0 | |
C11 | Reduction in average length of outages due to Cisco FSO | InterviewsInterviews | 20%20% | 40%40% | 60%60% | |
C12 | Average outage time with Cisco FSO (hours) | C3*(100%-C11) | 3.23.2 | 2.42.4 | 1.61.6 | |
C13 | Subtotal: Revenue lost due to outages with Cisco FSO | C1*C10*C12*C6*C7 | $14,668,306 $14,668,306 | $9,520,295 $9,520,295 | $4,231,242 $4,231,242 | |
Ct | Avoided revenue loss from outage prevention | C8-C13 | $6,487,905 $6,487,905 | $11,635,916 $11,635,916 | $16,924,969 $16,924,969 | |
Risk adjustment | ↓20% | |||||
Ctr | Avoided revenue loss from outage prevention (risk-adjusted) | $5,190,324 $5,190,324 | $9,308,733 $9,308,733 | $13,539,975 $13,539,975 | ||
Three-year total: $28,039,032 $28,039,032 | Three-year present value: $22,584,428 $22,584,428 |
Evidence and data. Several interviewees offered high praise for Cisco’s training program, and they emphasized that it offers immense value. These individuals recognized that the program not only sharpened their own expertise in utilizing Cisco FSO, but that it also had a broader impact on their skill sets as internal IT and DevOps employees. The comprehensive nature of the training allowed them to enhance their overall abilities and stay updated with industry best practices.
One key advantage mentioned was the accessibility to Cisco University, which interviewees said offers a wealth of advanced learning materials. Through this platform, team members have the opportunity to delve into specialized topics, explore in-depth technical resources, and gain valuable insights from industry experts. This access to a vast array of educational resources greatly contributed to their professional growth and development.
The impact of the training program on the efficiency and productivity of teams was noteworthy. Interviewees said that by acquiring new knowledge and refining existing skills, team members were able to perform their tasks more effectively and deliver higher-quality results. The shared language and understanding fostered by the training program and Cisco FSO’s user-friendly dashboard further facilitated collaboration among team members with diverse backgrounds and roles. As a result, communication barriers were reduced, which led to smoother workflows and quicker problem resolution.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
might have 0 employees who take part in the implementation of AppDynamics and ThousandEyes.
Risks. Forrester recognizes that these results may not be representative of all experiences and that the benefit will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1.1 million.
For , this benefit might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
---|---|---|---|---|---|
D1 | Internal implementation resources who become certified with Cisco | F1+G2 | 1414 | 1414 | 1414 |
D2 | Capacity of resources in the prior environment | CompositeComposite | 100%100% | 100%100% | 100%100% |
D3 | Capacity of resources after becoming certified | InterviewsInterviews | 120%120% | 125%125% | 130%130% |
D4 | Average fully burdened salary for an internal implementation resource | TEI standardTEI standard | $139,000 $139,000 | $139,000 $139,000 | $139,000 $139,000 |
Dt | Internal resource efficiency and productivity savings | D1*(D3-D2)*D4 | $389,200 $389,200 | $486,500 $486,500 | $583,800 $583,800 |
Risk adjustment | ↓10% | ||||
Dtr | Internal resource efficiency and productivity savings (risk-adjusted) | $350,280 $350,280 | $437,850 $437,850 | $525,420 $525,420 | |
Three-year total: $1,313,550 $1,313,550 | Three-year present value: $1,075,052 $1,075,052 |
Interviewees mentioned the following additional benefits that their organizations experienced but were not able to quantify:
The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement Cisco Full-Stack Observability and later realize additional uses and business opportunities, including:
Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Appendix A).
Ref. | Cost | Initial | Year 1 | Year 2 | Year 3 | Total | Present Value |
---|---|---|---|---|---|---|---|
Etr | Cisco FSO licensing and hybrid environment costs | $5,500 $5,500 | $3,767,500 $3,767,500 | $3,767,500 $3,767,500 | $3,767,500 $3,767,500 | $11,308,000 $11,308,000 | $9,374,715 $9,374,715 |
Ftr | Implementation and ongoing management costs for AppDynamics | $101,178 $101,178 | $238,354 $238,354 | $331,309 $331,309 | $544,775 $544,775 | $1,215,616 $1,215,616 | $1,000,970 $1,000,970 |
Gtr | Implementation and ongoing support costs for ThousandEyes | $52,992 $52,992 | $690,000 $690,000 | $690,000 $690,000 | $690,000 $690,000 | $2,122,992 $2,122,992 | $1,768,920 $1,768,920 |
Htr | Training costs for Cisco FSO | $150,696 $150,696 | $5,106 $5,106 | $108,146 $108,146 | $5,106 $5,106 | $269,054 $269,054 | $248,551 $248,551 |
Total costs (risk-adjusted) | $310,366 $310,366 | $4,700,960 $4,700,960 | $4,896,955 $4,896,955 | $5,007,381 $5,007,381 | $14,915,662 $14,915,662 | $12,393,156 $12,393,156 | |
Evidence and data. Interviewees shared that their organizations incurred licensing fees to use Cisco FSO and that the fees were combined with those for AppDynamics and ThousandEyes. Each interviewee’s organization is on a three-year annual plan, and the costs vary by the workloads and the environments of Cisco monitoring. Some of the organizations with complex environments pay professional services fees to Cisco for ongoing support.
Potential licensing costs calculated by this tool are estimates only and do not constitute quotes. For more information, contact Cisco.
Modeling and assumptions. Forrester assumes the following about the composite organization:
For , the annual licensing fee for AppDynamics might be , and the annual licensing fee for ThousandEyes might be .
For , the ongoing professional services fee for AppDynamics might be , and the ongoing professional services fee for ThousandEyes might be .
might have in additional annual costs.
Risks. Forrester recognizes that these results may not be representative of all experiences and that the cost will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this cost upward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $9.4 million.
For , this cost might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
---|---|---|---|---|---|---|---|
E1 | AppDynamics licensing (3-year annual plan) | CompositeComposite | $2,600,000 $2,600,000 | $2,600,000 $2,600,000 | $2,600,000 $2,600,000 | ||
E2 | ThousandEyes licensing (3-year annual plan) | CompositeComposite | $420,000 $420,000 | $420,000 $420,000 | $420,000 $420,000 | ||
E3 | Ongoing professional services for AppDynamics | CompositeComposite | $250,000 $250,000 | $250,000 $250,000 | $250,000 $250,000 | ||
E4 | Ongoing professional services for ThousandEyes | CompositeComposite | $150,000 $150,000 | $150,000 $150,000 | $150,000 $150,000 | ||
E5 | Additional annual costs needed to support hybrid application environment requirements (storage, S3, networking, WAN, dedicated servers, etc.) | AssumptionAssumption | $5,000 $5,000 | $5,000 $5,000 | $5,000 $5,000 | $5,000 $5,000 | |
Et | Cisco FSO licensing and hybrid environment costs | E1+E2+E3+E4+E5 | $5,000 $5,000 | $3,425,000 $3,425,000 | $3,425,000 $3,425,000 | $3,425,000 $3,425,000 | |
Risk adjustment | ↑10% | ||||||
Etr | Cisco FSO licensing and hybrid environment costs (risk-adjusted) | $5,500 $5,500 | $3,767,500 $3,767,500 | $3,767,500 $3,767,500 | $3,767,500 $3,767,500 | ||
Three-year total: $11,308,000 $11,308,000 | Three-year present value: $9,374,715 $9,374,715 |
Evidence and data. The interviewees’ organizations followed a phased approach when implementing AppDynamics, starting with a proof of concept by running a pilot group of applications. Based on the number and business relevance of the applications, the organizations then proceeded to roll out AppDynamics by prioritizing the most critical business applications. This approach ensured a strategic and targeted implementation of AppDynamics across the organizations’ application landscapes.
The program manager with the government administration organization shared: “When I started the implementation, I did this quarterly. We started like a POC got two or three applications back in 2021. In 2022, we kind of scaled up to 20 apps. I brought in 50 the next year, then 190 to 200 business-critical applications.”
Modeling and assumptions. Forrester assumes the following about the composite organization:
For , 0 internal resource(s) might participate in the implementation of AppDynamics, and each might spend 0 hours on the project.
For , application owner(s) might spend 0 hours implementing AppDynamics initially, 0 hours in Year 1, 0 hours in Year 2, and 0 hours in Year 3.
For , 0 program manager(s) might dedicate time to AppDynamics.
Risks. Forrester recognizes that these results may not be representative of all experiences and that the cost will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this cost upward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1 million.
For , this cost might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
---|---|---|---|---|---|---|---|
F1 | Internal resources who participate in implementation | CompositeComposite | 66 | ||||
F2 | Time dedicated to implementation tasks (hours) | CompositeComposite | 180180 | ||||
F3 | Average burdened monthly cost for an implementation resource | CompositeComposite | $66 $66 | ||||
F4 | Subtotal: Internal implementation labor | F1*F2*F3 | $71,280 $71,280 | $0 $0 | $0 $0 | $0 $0 | |
F5 | Applications implemented | B3 | 55 | 3131 | 6363 | 125125 | |
F6 | Incremental applications | F5-F6PY | 55 | 2626 | 3232 | 6262 | |
F7 | Time spent per additional application integrated and tested (hours) | CompositeComposite | 4040 | 3535 | 3030 | 3030 | |
F8 | Burdened hourly cost for an application owner | CompositeComposite | $101 $101 | $101 $101 | $101 $101 | $101 $101 | |
F9 | Subtotal: Additional app integration labor costs | F5*F7*F8 | $20,200 $20,200 | $109,585 $109,585 | $190,890 $190,890 | $378,750 $378,750 | |
F10 | Miscellaneous API management fees, additional storage fees, and licensing fees | F5*10%*$1,000 | $500 $500 | $3,100 $3,100 | $6,300 $6,300 | $12,500 $12,500 | |
F11 | Subtotal: Implementation costs for AppDynamics | F4+F9+F10 | $91,980 $91,980 | $112,685 $112,685 | $197,190 $197,190 | $391,250 $391,250 | |
F12 | System admin/program managers dedicated to ongoing management | CompositeComposite | 11 | 11 | 11 | ||
F13 | Percentage of time dedicated to upkeep | InterviewsInterviews | 80%80% | 80%80% | 80%80% | ||
F14 | Average burdened annual cost for a system admin/program manager | CompositeComposite | $130,000 $130,000 | $130,000 $130,000 | $130,000 $130,000 | ||
F15 | Subtotal: Cost of ongoing management labor | F12*F13*F14 | $104,000 $104,000 | $104,000 $104,000 | $104,000 $104,000 | ||
Ft | Implementation and ongoing management costs for AppDynamics | F11+F15 | $91,980 $91,980 | $216,685 $216,685 | $301,190 $301,190 | $495,250 $495,250 | |
Risk adjustment | ↑10% | ||||||
Ftr | Implementation and ongoing management costs for AppDynamics (risk-adjusted) | $101,178 $101,178 | $238,354 $238,354 | $331,309 $331,309 | $544,775 $544,775 | ||
Three-year total: $1,215,616 $1,215,616 | Three-year present value: $1,000,970 $1,000,970 |
Evidence and data. As a common practice, the interviewees’ organizations initially established pilot groups to conduct proofs of concept for ThousandEyes. Based on the results and specific deployment requirements, the organizations then proceeded to roll out ThousandEyes across the entire organization. The timeline for implementation varied depending on the unique needs and considerations of each organization’s deployment process.
The manager of application performance at the retail organization shared: “I would say it took a month for ThousandEyes to get up and running. We also have a separate project engagement with [Cisco] to continue to build out our FSO tools, and that has probably six individuals supporting us: one architect and … five developers/engineers.”
Modeling and assumptions. Forrester assumes the following about the composite organization:
For , 0 employee(s) might participate in the implementation of ThousandEyes, and each might spend 0 hours on the project.
For , 0 employee(s) might be responsible for the ongoing maintenance of ThousandEyes.
Risks. Forrester recognizes that these results may not be representative of all experiences and that the cost will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this cost upward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1.8 million.
For , this cost might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
---|---|---|---|---|---|---|---|
G1 | Time spent on implementation (hours) | CompositeComposite | 8080 | ||||
G2 | Employees dedicated to deployment | CompositeComposite | 88 | ||||
G3 | Average fully burdened hourly rate for a DevOps employee | TEI standardTEI standard | $72 $72 | ||||
G4 | Subtotal: Internal implementation labor | G1*G2*G3 | $46,080 $46,080 | ||||
G5 | Employees needed for ongoing maintenance | CompositeComposite | 44 | 44 | 44 | ||
G6 | Fully burdened hourly rate for a DevOps employee | TEI standardTEI standard | $150,000 $150,000 | $150,000 $150,000 | $150,000 $150,000 | ||
G7 | Subtotal: Internal ongoing maintenance labor | G5*G6 | $0 $0 | $600,000 $600,000 | $600,000 $600,000 | $600,000 $600,000 | |
Gt | Implementation and ongoing support costs for ThousandEyes | G4+G7 | $46,080 $46,080 | $600,000 $600,000 | $600,000 $600,000 | $600,000 $600,000 | |
Risk adjustment | ↑15% | ||||||
Gtr | Implementation and ongoing support costs for ThousandEyes (risk-adjusted) | $52,992 $52,992 | $690,000 $690,000 | $690,000 $690,000 | $690,000 $690,000 | ||
Three-year total: $2,122,992 $2,122,992 | Three-year present value: $1,768,920 $1,768,920 |
Evidence and data. In addition to IT engineers and DevOps employees, the business owners of the applications actively participated in the daily usage and maintenance of Cisco FSO. The interviewees emphasized that training was an integral part of the internal investment, ensuring that both IT and business teams were well-equipped to effectively utilize and maintain the platform. This collaborative approach fostered a shared responsibility and understanding of the application’s performance and overall success. The program manager at a government administration organization shared: “Last year, I had between 50 and 70 hours’ worth of training. Core application managers took 20 or 40 hours for all the courses.”
Modeling and assumptions. Forrester assumes the following about the composite organization:
For , 0 employee(s) might receive training initially, and then 0 employee(s) might receive training in subsequent years.
Risks. Forrester recognizes that these results may not be representative of all experiences and that the cost will vary among organizations depending on the following factors:
Results. To account for these risks, Forrester adjusted this cost upward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $249,000.
For , this cost might have a three-year, risk-adjusted PV of .
Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
---|---|---|---|---|---|---|---|
H1 | Resources who participate in training for Cisco FSO | F1+G2 | 1414 | 3030 | 3030 | 3030 | |
H2 | Training time (hours) | CompositeComposite | 4040 | 22 | 22 | 22 | |
H3 | Average fully burdened hourly cost for an end-team member | CompositeComposite | $74 $74 | $74 $74 | $74 $74 | $74 $74 | |
H4 | Subtotal: End-team training costs | H1*H2*H3 | $41,440 $41,440 | $4,440 $4,440 | $4,440 $4,440 | $4,440 $4,440 | |
H5 | Tuition for CiscoU All Access | Cisco | $6,000 $6,000 | $0 $0 | $6,000 $6,000 | $0 $0 | |
H6 | Cost of certification for IT architect | Cisco | $400 $400 | $0 $0 | $400 $400 | $0 $0 | |
H7 | Subtotal: Tuition and certification costs | H1Y0*(H5+H6) | $89,600 $89,600 | $0 $0 | $89,600 $89,600 | $0 $0 | |
Ht | Training costs for Cisco FSO | H4+H7 | $131,040 $131,040 | $4,440 $4,440 | $94,040 $94,040 | $4,440 $4,440 | |
Risk adjustment | ↑15% | ||||||
Htr | Training costs for Cisco FSO (risk-adjusted) | $150,696 $150,696 | $5,106 $5,106 | $108,146 $108,146 | $5,106 $5,106 | ||
Three-year total: $269,054 $269,054 | Three-year present value: $248,551 $248,551 |
The financial results calculated in the Benefits and Costs sections can be used to determine the ROI, NPV, and payback period for the composite organization’s investment. Forrester assumes a yearly discount rate of 10% for this analysis.
These risk-adjusted ROI, NPV, and payback period values are determined by applying risk-adjustment factors to the unadjusted results in each Benefit and Cost section.
Initial | Year 1 | Year 2 | Year 3 | Total | Present Value | |
---|---|---|---|---|---|---|
Total costs | ($310,366)($310,366) | ($4,700,960)($4,700,960) | ($4,896,955)($4,896,955) | ($5,007,381)($5,007,381) | ($14,915,662)($14,915,662) | ($12,393,156)($12,393,156) |
Total benefits | $0 $0 | $14,635,794 $14,635,794 | $22,738,510 $22,738,510 | $32,927,278 $32,927,278 | $70,301,582 $70,301,582 | $56,836,176 $56,836,176 |
Net benefits | ($310,366)($310,366) | $9,934,835 $9,934,835 | $17,841,555 $17,841,555 | $27,919,897 $27,919,897 | $55,385,921 $55,385,921 | $44,443,020 $44,443,020 |
ROI | 359%359% | |||||
Payback | <6<6 | |||||
Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.
Benefits represent the value delivered to the business by the product. The TEI methodology places equal weight on the measure of benefits and the measure of costs, allowing for a full examination of the effect of the technology on the entire organization.
Costs consider all expenses necessary to deliver the proposed value, or benefits, of the product. The cost category within TEI captures incremental costs over the existing environment for ongoing costs associated with the solution.
Flexibility represents the strategic value that can be obtained for some future additional investment building on top of the initial investment already made. Having the ability to capture that benefit has a PV that can be estimated.
Risks measure the uncertainty of benefit and cost estimates given: 1) the likelihood that estimates will meet original projections and 2) the likelihood that estimates will be tracked over time. TEI risk factors are based on “triangular distribution.”
The initial investment column contains costs incurred at “time 0” or at the beginning of Year 1 that are not discounted. All other cash flows are discounted using the discount rate at the end of the year. PV calculations are calculated for each total cost and benefit estimate. NPV calculations in the summary tables are the sum of the initial investment and the discounted cash flows in each year. Sums and present value calculations of the Total Benefits, Total Costs, and Cash Flow tables may not exactly add up, as some rounding may occur.
1 Source: The Future Of Cloud, Forrester Research Inc., July 17, 2023.
2 Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.
3 Source: The Forrester Guide To Incident And Crisis Management, Forrester Research, Inc., May 20, 2022.
4 Source: Margins by sector (US), NYU Stern School of Business, January 2024.
5 Source: Cisco Completes Acquisition of Splunk, Cisco press release, March 18, 2024.
Cookie Preferences
Accept Cookies
A cookie is a small text file that a website saves on your computer or mobile device when you visit the site. It enables the website to remember your actions (data inputs, website navigation), so you don’t have to re-enter data when you come back to the site or browse from one page to another.
Behavioral information collected by our web analytics vendor is used to analyze data pertaining to visitor trends, plan website enhancements, and measure overall website effectiveness. We may also use cookies or web beacons to help us offer you products, programs, or services that may be of interest to you and to deliver relevant advertising. We may use third-party advertising companies to help tailor website content to users or to serve ads on our behalf. These companies may also employ cookies and web beacons to measure advertising effectiveness.
Please accept cookies and the collection of behavioral information to receive full functionality and enhance your experience. If you decline cookies, some features of the website may not function normally.
Please see our
Privacy Policy for more information.