The Total Economic Impact™ Of Cortex IDP
Cost Savings And Business Benefits Enabled By Cortex
A Forrester Total Economic Impact™ Study Commissioned By Cortex, July 2024
Software development remains an enormous subset of the broader technology industry — especially as more and more organizations shift away from hardware solutions to take advantage of the cloud and software as a service (SaaS). Despite the massive importance of software development, developer teams are still struggling with decentralized processes around organizing their people and technology. According to a recent Forrester survey, only one in nine surveyed developers said that their organization has a dedicated team focusing on development experience.1 Organizations that take steps to improve the developer experience in a way that promotes positive business outcomes stand to gain a unique advantage.
Cortex is an internal developer portal (IDP) that centralizes all engineering team data, systems, and standards to eliminate friction in building and maintaining high quality software.
Cortex commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study and examine the potential return on investment (ROI) enterprises may realize by deploying Cortex IDP.2 The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of Cortex IDP on their organizations.
Return on investment (ROI)
224%
Net present value (NPV)
$4.41M
To better understand the benefits, costs, and risks associated with this investment, Forrester interviewed four representatives with experience using Cortex IDP. For the purposes of this study, Forrester aggregated the interviewees’ experiences and combined the results into a single composite organization that is a software company with $500 million in annual revenue and 2,800 employees.
Interviewees said that prior to using Cortex IDP, their organizations struggled to centralize — or often even define — processes around software development. Their organizations were growing rapidly, and they lacked the time or resources to document software components, enforce standards of creation, track ownership over time, and ensure visibility for developers, managers, and executives. Manual, ad hoc, and disjointed processes surrounding how software was built created avoidable inefficiencies that left the interviewees’ organizations scrambling to support the velocity and reliability demanded by organizationwide growth initiatives. Prior attempts to internally develop some semblance of a developer portal or platform were frequently unsuccessful or sidetracked by more urgent projects.
After the investment in Cortex IDP, the interviewees increased visibility into software health, project status, and developer productivity, enabling them to more easily identify bottlenecks and resolve issues. The total cost of ownership of having to build or maintain their own system of record was reduced. Software development with Cortex sped up, with less time spent on maintenance, context-switching, status reporting, and searching for information.
Key Findings
Quantified benefits. Three-year, risk-adjusted present value (PV) quantified benefits for the composite organization include:
- Improved engineering productivity on certain development processes by 20%. The composite organization’s engineering team using Cortex realizes a 20% productivity savings on various tasks impacted by Cortex. This includes 5% savings on avoiding context switching, 5% savings on development due to aggregating information during an incident, 5% savings around managing dependencies, and 5% savings around service and resource provisioning. While employees don’t spend all their development time using Cortex (as developers will use other tools for developing and testing code), Cortex is able to take information from other development tasks to provide these savings on development tasks that directly use this service. The total number of developers using Cortex grows by 10% each year as adoption spreads throughout the organization, totaling a risk-adjusted, three-year PV of $2.7 million.
- Increased time savings on gathering metrics, monitoring migrations and initiatives, and reporting tasks for team leads by 75%. The composite organization uses Cortex to build 300 software components (a mix of services and other projects) each year. Team leads organizing these projects realize a total time savings of 75% due to Cortex aggregating information from throughout the development process eliminating the need to spend dedicated time gathering metrics, building reports, and checking on the status of migrations, initiatives, and developers. The composite organization saves a risk-adjusted, three-year PV of just under $1 million.
- Increased the speed of time to market on impacted projects by 25%. In addition to saving time on various tasks around software development for the projects built with Cortex, the composite organization also reduces the overall time required to deploy new software, complete software or system upgrades, and/or bring new features to market. This equals an additional two weeks for features, updates, and software that directly increase annual revenue, enabling the composite organization to gain a risk-adjusted, three-year PV of $780,000.
- Reduced the total cost of ownership, allowing the reallocation of six engineering FTEs. Cortex offers many prebuilt and integrations obviating the need to build and maintain custom connections across the stack. It also eliminates the need for infrastructure team members to manage previously used service catalogs that required them to manually maintain ownership, software status, package versions, documentation, and dependencies. Cortex also eliminates the need for front-end engineers tasked with building custom experiences on top of these stop-gap solutions. The six FTEs whose work can be reallocated equal a risk-adjusted, three-year PV of $1.9 million for the composite organization.
Quantitative benefits. Benefits that provide value for the composite organization but are not quantified for this study include:
- Improved productivity for security and compliance tasks. In addition to saving time on software development, the composite organization experiences time savings around development tasks related to security, preparing for compliance, and proving software compliance to customers or regulators.
- Improved developer ramp time. Centralizing and unifying team-, role-, and software-specific information and processes enables the composite organization to accelerate onboarding for new hires and reduce time to first meaningful commit.
- Improved ability to reorganize teams while preserving historical knowledge. Standardizing tools and processes across the composite organization makes it easier to undergo structural reorganizations while preserving software and system knowledge when team members are added, change teams, or leave the organization.
- Reduced downtime. By enforcing best practices, Cortex helps the composite organization reduce incidents and time-to-find owners and context during incident response. Less time spent on resolving issues means more productive time back for other projects.
Costs. Three-year, risk-adjusted PV costs for the composite organization include:
- Three FTEs working on initial implementation and two FTEs working on ongoing maintenance. A small team of three junior engineers spends 10 weeks implementing Cortex across the composite organization with a team of two FTEs managing the platform and generating and circulating reports after implementation. This totals a risk-adjusted, three-year PV of $996,000 for the composite organization.
- Eight hours of FTE time per service added to Cortex. The software components developed using Cortex each require 4 hours of time for getting up to speed and deploying with Cortex. Additionally, employees new to Cortex require approximately 8 hours of training to get up to speed with the capabilities of the platform. With 240 initial employees and 300 initial services in Year 1 increasing by approximately 10% per year with increased adoption, the composite organization spends a risk-adjusted $440,000 over three years on this onboarding.
- $65 per Cortex seat holder per month in ongoing licensing fees. Based on the package selected by the composite organization, it pays $65 per month in licensing fees for each seat holder with access to Cortex. A total of 240 users are seat holders at the composite organization, paying the licensing fee to use Cortex for certain tasks outlined in other areas of the study. Over three years, this cost has a risk-adjusted, three-year PV of $536,000.
The representative interviews and financial analysis found that a composite organization experiences benefits of $6.38 million over three years versus costs of $1.97 million, adding up to a net present value (NPV) of $4.41 million and an ROI of 224%.
Key Statistics
-
Return on investment (ROI)
224%
-
Benefits PV
$6.38M
-
Net present value (NPV)
$4.41M
-
Payback
<6 months
Benefits (Three-Year)
Improved engineering productivity Improved management productivityt on service development Increased revenue from improved engineering velocity Reduced total cost of ownerwhipTEI Framework And Methodology
From the information provided in the interviews, Forrester constructed a Total Economic Impact™ framework for those organizations considering an investment in Cortex IDP.
The objective of the framework is to identify the cost, benefit, flexibility, and risk factors that affect the investment decision. Forrester took a multistep approach to evaluate the impact that Cortex IDP can have on an organization.
-
Due Diligence
Interviewed Cortex stakeholders and Forrester analysts to gather data relative to Cortex IDP.
-
Interviews
Interviewed four representatives at organizations using Cortex IDP to obtain data about costs, benefits, and risks.
-
Composite Organization
Designed a composite organization based on characteristics of the interviewees’ organizations.
-
Financial Model Framework
Constructed a financial model representative of the interviews using the TEI methodology and risk-adjusted the financial model based on issues and concerns of the interviewees.
-
Case Study
Employed four fundamental elements of TEI in modeling the investment impact: benefits, costs, flexibility, and risks. Given the increasing sophistication of ROI analyses related to IT investments, Forrester’s TEI methodology provides a complete picture of the total economic impact of purchase decisions. Please see Appendix A for additional information on the TEI methodology.
Disclosures
Readers should be aware of the following:
This study is commissioned by Cortex and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.
Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the study to determine the appropriateness of an investment in Cortex IDP.
Cortex reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.
Cortex provided the customer names for the interviews but did not participate in the interviews.
Consulting Team:
Sam Sexton
Matt Dunham
Drivers leading to the IDP investment
Interviews
| Role | Industry | Employees | Annual Revenue |
|---|---|---|---|
| Engineering manager | HR SaaS | 2,800 | $175M |
| SVP, director of engineering enablement | Sports | 3,000 | $1B |
| Platform engineer | Software | 1,400 | $178M |
| Engineering director | Media | 610 | $500M |
Key Challenges
Before Cortex, the interviewees’ organizations either completely lacked any form of a developer portal or had built their own makeshift solution of documentation, process oversight, and/or service catalog systems. Processes were often ill-defined, as was visibility into which team members were required for which tasks. Gathering and maintaining any information around these issues required significant internal effort.
The interviewees noted how their organizations struggled with common challenges, including:
- Development tasks and maintenance that required significant manual effort. Several of the interviewees told Forrester that their organizations’ prior environments required significantly more manual effort to keep development projects on track and manage developers and projects. The engineering director for the media organization explained: “Our main tool was in-house, which we used in combination with open-source databases, homegrown scripts and files, and spreadsheets. … It required a lot of maintenance and upkeep from our internal teams, and I’d rather have them spending that time making our actual product better.”
- Decentralized information further complicating management and development. In addition to the inherent challenge of managing an in-house tool, disparate files, and the projects themselves, the interviewees’ organizations struggled with locating the information needed to keep everything running smoothly. The platform engineer for the software company explained the difficulty with finding basic information: “We needed to understand how many services we had, the metadata with these services, and that was pretty hard to get. It was a very manual process.”
- The inability to keep up with rapid growth. All these issues were compounded with the reality that several of the interviewees’ organizations were in the middle of rapid growth, meaning they needed to balance managing these issues with delivering more for their organization. The engineering director for the media organization summed this issue up: “We’re adding more features, more repos, more services, and all of those things have continued. Scaling and maintaining these things were one of my main concerns.”
Solution Requirements
The interviewees’ organizations searched for a solution that could:
- Speed up development of software and completion of software-related projects.
- Increase visibility into their organizations’ internally developed software components as well as employee engagement.
- Keep up with rapid company growth.
- Reduce total cost of ownership (TCO) by saving time and effort on maintaining homegrown systems.
Composite Organization
Based on the interviews, Forrester constructed a TEI framework, a composite company, and an ROI analysis that illustrates the areas financially affected. The composite organization is representative of the four interviewees, and it is used to present the aggregate financial analysis in the next section. The composite organization has the following characteristics:
Description of composite. The composite is a software organization that has a total of 2,800 full-time employees and is worth $500 million. It has a global reach and operates with a 75% profit margin.
Deployment characteristics. The composite organization initially deploys Cortex to a small subset of its engineering team. This subset integrates Cortex with all of their activities and uses Cortex for active development work for about one-third of their time. This team, and the number of software components developed using Cortex, increase by 10% each year as adoption increases within the organization.
Quantified benefit data as applied to the composite
Total Benefits
| Ref. | Benefit | Year 1 | Year 2 | Year 3 | Total | Present Value |
|---|---|---|---|---|---|---|
| Atr | Improved engineering productivity | $990,900 | $1,089,180 | $1,201,500 | $3,281,580 | $2,703,672 |
| Btr | Improved management productivity on service development | $365,513 | $402,064 | $442,270 | $1,209,846 | $996,852 |
| Ctr | Increased revenue from improved engineering velocity | $286,875 | $315,563 | $344,250 | $946,688 | $780,231 |
| Dtr | Reduced total cost of ownership | $763,300 | $763,300 | $763,300 | $2,289,900 | $1,898,214 |
| Total benefits (risk-adjusted) | $2,406,588 | $2,570,106 | $2,751,320 | $7,728,014 | $6,378,969 | |
Improved Engineering Productivity
Evidence and data. The interviewees told Forrester that Cortex helped their organization’s engineering teams realize significant time savings in a variety of areas:
-
Interviewees noted much of these time savings came from Cortex
serving as a unifying system of record; Cortex’s ability to immediately
answer questions around the state, dependencies, and functionalities of
different tools and software components saved time in engineer onboarding as
well as time spent managing software components’ alignment to standards of
excellence.
The engineering manager for the HR SaaS organization summed it up: “On average, anytime a developer is going to work on something, they might spend a day or a week trying to orient themselves. [With Cortex], you can avoid that.” -
Cortex automatically aggregated information around solutions
and processes, enabling time savings during incidents and troubleshooting
and eliminating the need to spend time looking for information required to
finalize projects or hit required deadlines.
- The interviewee for the HR SaaS organization stated, “[With Cortex], we’re saving time by fixing the important things first, so that the folks that own and interact with our services don’t have to waste time over the long run unable to do what they need to do.”
- The platform engineer for the software company told Forrester, “[With Cortex], it is much faster to fix issues on day-to-day releases.”
-
Cortex also provided time savings around updating existing
software, service or resource provisioning, and other ongoing maintenance
and project management.
The engineering director for the media organization gave an example of the type of ongoing management effort that was avoided with Cortex: “Managing major version changes and third-party libraries was a pain in the butt; project management kept interrupting our sprints. Teams would have to conduct searches through the code base. The best thing about Cortex is you can set the initiative based on values in a file, and it only alerts teams that actually have to make changes — that really improved the developer experience.” - In addition to saving time on active projects, interviewees explained that Cortex’s Scorecards and reports obviated the need for engineers to set aside time to gather metrics to answer questions from team leads. The engineering manager for the HR SaaS organization stated, “I would say we’re saving time — we had engineers working on pulling metrics for hours every month.”
- Interviewees were impressed with Cortex’s ability to provide these time savings without the need for extra add-ons, additional effort, or additional maintenance. The engineering director for the media organization said: “We were looking for something full-featured. … A lot of what we needed would have been extras or required a lot of maintenance on our part [from other competitors]. … Cortex had what we needed up front.”
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- The composite organization has 70 engineering FTEs using Cortex full-time. This does not represent the total number of seat holders for Cortex, but the time those seat holders spend on projects and work impacted by Cortex.
- As adoption increases within the organization, the number of engineering FTEs working on Cortex increases by 10% (rounded) year over year.
-
The engineer FTEs realize 20% time savings, divided evenly
between the following categories:
- Avoided context switching.
- Avoiding having to actively aggregate information during an incident.
- Reduced time required to manage dependencies.
- Reduced time required for service provisioning.
- To account for employees not fully using newfound time as increased employee productivity, a standard 50% productivity recapture is applied.
- Additionally, 240 engineering hours per year that were previously spent on gathering metrics and information are avoided due to Cortex providing this information with scorecards.
Risks. Factors that could impact the size of the benefit for organizations include:
- The total adoption of Cortex and total time spent per seat holder on activities using Cortex.
- The salary of engineering FTEs.
- The degree to which Cortex saves time on preexisting tasks.
Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $2.7 million.
Improved Engineering Productivity
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| A1 | Engineering FTEs using Cortex for projects | Composite | 70 | 77 | 85 |
| A2 | Fully burdened annual salary for an engineer | TEI standard | $156,000 | $156,000 | $156,000 |
| A3 | Productivity savings on development due to avoided context switching | Interviews | 5% | 5% | 5% |
| A4 | Productivity savings on development due to aggregating information during an incident | Interviews | 5% | 5% | 5% |
| A5 | Productivity savings around managing dependencies and onboarding | Interviews | 5% | 5% | 5% |
| A6 | Productivity savings around service provisioning | Interviews | 5% | 5% | 5% |
| A7 | Subtotal: Total productivity savings due to Cortex | A3+A4+A5+A6 | 20% | 20% | 20% |
| A8 | Teamwide engineering FTE hours per year spent on gathering metrics | Interviews | 240 | 240 | 240 |
| A9 | Fully burdened hourly rate for an engineer | A2/2,080 hours per year | $75 | $75 | $75 |
| A10 | Productivity recapture | TEI standard | 50% | 50% | 50% |
| At | Improved engineering productivity | (A1*A2*A7)+ (A9* A8*A10) | $1,101,000 | $1,210,200 | $1,335,000 |
| Risk adjustment | ↓10% | ||||
| Atr | Improved engineering productivity (risk-adjusted) | $990,900 | $1,089,180 | $1,201,500 | |
| Three-year total: $3,281,580 | Three-year present value: $2,703,672 | ||||
Improved Management Productivity On Software Development
Evidence and data. In addition to saving time for individually contributing developers, Cortex’s aggregation of information on metrics, processes, and employee skills proved valuable for team leaders at the interviewees’ organizations.
- Previously, managers and team leaders repeatedly had to check on the status of ongoing initiatives, such as software, package, or system migrations. According to interviewees, Cortex significantly reduced the time required for checking on these tasks.
- Many team leaders at the interviewees’ organizations previously had to dedicate valuable time to hunting for metrics, assembling reports, and contextualizing that information for VP- or C-level employees at their organizations. Because Cortex gathers metrics from across the organization and aggregates them, this task was eliminated for many of the interviewees’ organizations. The engineering manager for the media organization explained: “Building dashboards, monitoring, and trying to push forward documents was probably 10 manager hours per week. … It’s closer to one or two now.”
- These metrics also let managers know which projects and issues needed to be prioritized, saving them time when searching for information on the health of specific software components and avoiding work on software that did not require as many updates. The engineering manager for the HR SaaS organization broke down how this helped their organization: “Our Cortex scorecards are built so that we have bronze-, silver-, and gold-tier levels for each of our systems, and the criteria to advance them is pretty clear. … It’s super clear where we have to spend effort. It’s a time-saver.”
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- The composite organization builds 300 software components per year using Cortex. This number increases by 10% (rounded) year over year as more software components are built using Cortex.
- For each component built using Cortex, 24 team-leader hours are required for tasks like checking initiatives, gathering metrics, reporting, and other project management.
- Cortex eliminates 75% of the time required for these tasks by aggregating information via scorecards, providing reporting, and streamlining project management.
- A 75% productivity recapture is applied.
Risks. Factors that could impact the size of this benefit for organizations include:
- The number of components built in Cortex each year and the organization’s internal definition of a service.
- The number of project management tasks required for each project.
- The salary of team leaders.
- The degree to which Cortex can save time on project management tasks.
Results. To account for these risks, Forrester adjusted this benefit downward by 5%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $997,000.
Improved Management Productivity On Software Development
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| B1 | Software components built per year using Cortex | Composite | 300 | 330 | 363 |
| B2 | Team lead hours spent on metrics, migration and initiatives, reporting, and other tasks per component | Composite | 24 | 24 | 24 |
| B3 | Time savings on these tasks due to Cortex | Interviews | 75% | 75% | 75% |
| B4 | Fully burdened hourly rate for a team lead | TEI standard | $95 | $95 | $95 |
| B5 | Productivity recapture | TEI standard | 75% | 75% | 75% |
| Bt | Improved management productivity on software development | B1*B2*B3*B4*B5 | $384,750 | $423,225 | $465,548 |
| Risk adjustment | ↓5% | ||||
| Btr | Improved management productivity on software development (risk-adjusted) | $365,513 | $402,064 | $442,270 | |
| Three-year total: $1,209,846 | Three-year present value: $996,852 | ||||
Increased Revenue From Improved Engineering Velocity
Evidence and data. Interviewees told Forrester that in addition to saving effort on projects, Cortex also accelerated the velocity of delivering services and products, increasing revenue as a result.
- The cumulative time savings described in the above benefits enabled the interviewees’ organizations to put out services and product features at a faster rate. The SVP and director of engineering enablement for the sports organization explained, “[Cortex] is going to save us weeks.”
- For certain projects — especially those related to product features — this ability to deliver faster contributed to additional revenue. The platform engineer for the software organization said, “Cortex lets us release new features as fast as possible and, as a result, we have more clients.”
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- Of the software components built within Cortex, 10% are related to developing product features or otherwise have a direct impact on improving revenue.
- Cortex improves the time to market for these revenue-generating projects by 25%, reducing development time from eight weeks to six weeks.
- Each week of a feature or project being available in the market results in $7,500 in additional revenue.
- The composite organization has a profit margin of 75%.
Risks. Factors that could impact the size of this benefit for organizations include:
- The number of projects impacted by Cortex that directly contribute to or generate revenue.
- The degree to which Cortex can reduce time to market.
- The profit margin of organizations.
- The weekly revenue of developed components for organizations.
Results. To account for these risks, Forrester adjusted this benefit downward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $780,000.
Increased Revenue From Improved Engineering Velocity
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| C1 | Revenue-generating projects per year impacted by Cortex (rounded) | B1*10% | 30 | 33 | 36 |
| C2 | Average time to market for revenue-generating projects impacted by Cortex (weeks) | Composite | 8 | 8 | 8 |
| C3 | Reduction in time to market due to Cortex | Interviews | 25% | 25% | 25% |
| C4 | Additional weeks on market per revenue-generating project | C2*C3 | 2 | 2 | 2 |
| C5 | Weekly revenue per revenue-generating project | Assumption | $7,500 | $7,500 | $7,500 |
| C6 | Profit margin | Composite | 75% | 75% | 75% |
| Ct | Increased revenue from improved engineering velocity | C1*C4*C5*C6 | $337,500 | $371,250 | $405,000 |
| Risk adjustment | ↓15% | ||||
| Ctr | Increased revenue from improved engineering velocity (risk-adjusted) | $286,875 | $315,563 | $344,250 | |
| Three-year total: $946,688 | Three-year present value: $780,231 | ||||
Reduced Total Cost Of Ownership
Evidence and data. The interviewees’ organizations attempted to build their own documentation and systems of record before using Cortex and manually managed integrations between their various tools and homegrown systems. For many of the interviewees, Cortex eliminated several of these tasks and responsibilities.
- Cortex’s out-of-box integrations largely eliminated the need to dedicate engineers to manually manage and configure integrations between tools and homegrown systems. The platform engineer for the software company explained: “Before Cortex, we had to do everything manually. … Now, we don’t need to maintain information with integrations.”
- Additionally, the Cortex portal automatically aggregates and displays information that enabled the interviewees’ organizations to reduce their commitment to updating their own systems of record or documentation systems.
- Because of Cortex’s SaaS deployment, interviewees said their organizations no longer needed infrastructure employees dedicated to monitor the system of record and were able to reassign them to other tasks.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- The composite organization’s previous homegrown attempt at a developer portal required two engineers managing integrations, two engineers managing the front end of the service, and two infrastructure team members.
- Cortex enables the composite organization to avoid these tasks, allowing it to reassign these employees to other work.
Risks. Factors that could impact the size of this benefit for organizations include:
- The degree to which a prior developer portal, documentation system, or service catalog requires internal effort to maintain.
- The degree to which Cortex can take over services and functions from these systems.
Results. To account for these risks, Forrester adjusted this benefit downward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1.9 million.
Reduced Total Cost Of Ownership
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| D1 | Engineers managing manual integrations before Cortex | Interviews | 2 | 2 | 2 |
| D2 | Fully burdened annual salary for an engineer | A2 | $156,000 | $156,000 | $156,000 |
| D3 | Engineers managing front end of prior services before Cortex | Interviews | 2 | 2 | 2 |
| D4 | Fully burdened annual salary for a front-end engineer | TEI standard | $149,000 | $149,000 | $149,000 |
| D5 | Infrastructure team members managing prior documentation and service catalog issues | Interviews | 2 | 2 | 2 |
| D6 | Fully burdened annual salary for an infrastructure team member | TEI standard | $144,000 | $144,000 | $144,000 |
| Dt | Reduced total cost of ownership | (D1*D2)+(D3*D4)+ (D5*D6) | $898,000 | $898,000 | $898,000 |
| Risk adjustment | ↓15% | ||||
| Dtr | Reduced total cost of ownership (risk-adjusted) | $763,300 | $763,300 | $763,300 | |
| Three-year total: $2,289,900 | Three-year present value: $1,898,214 | ||||
Unquantified Benefits
Interviewees mentioned the following additional benefits that their organizations experienced but were not able to quantify:
- Improved productivity for security and compliance tasks. While most of the interviewees had specific insight into the experience of their own engineering teams, the engineering director at a media organization explained that there had been significant adoption of Cortex with their organization’s compliance teams: “We can use Cortex for our SOC [security operations center] compliance. We use the reports about vulnerability and the scorecards to prove compliance. It saves them a lot of time.”
- Improved developer ramp time. According to the interviewees, for brand-new developers, Cortex significantly simplified the onboarding process by making responsibilities clearer, reducing the learning curve for development processes, and improving the speed at which users could make meaningful commits and contribute code. The platform engineer at a software organization told Forrester: “Cortex gives new engineers a quick understanding of what they are responsible for. Before Cortex, it was a pretty complex process. … The threshold for new users is much lower now.”
- Improved ability to reorganize and preserve knowledge. The interviewees reported that Cortex standardized processes and tools across their organizations, making reorganizing teams, moving team members to new roles, and preserving knowledge when team members leave the organization much easier.
- Reduced downtime. Being able to gather information more quickly during an incident saved time when searching for information and hastened the resolution of potential outages and issues.
Flexibility
The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement Cortex IDP and later realize additional uses and business opportunities, including:
- The ability to add additional employees and services into Cortex over time. The visibility of Cortex’s time savings within the interviewees’ organizations led to increasing adoption of the system over time among teams and projects. The director of engineering enablement at a sports organization noted that they had achieved the successes reported to Forrester with relatively low adoption, and that as adoption increased, so would the potential benefits.
- The ability to scale with rapid growth. For the interviewees’ organizations that experienced a period of break-neck growth, Cortex enabled them to keep up with maintaining their rapidly multiplying services while still meeting key deadlines.
Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Appendix A).
Quantified cost data as applied to the composite
Total Costs
| Ref. | Cost | Initial | Year 1 | Year 2 | Year 3 | Total | Present Value |
|---|---|---|---|---|---|---|---|
| Etr | Total implementation and ongoing management costs | $103,500 | $358,800 | $358,800 | $358,800 | $1,179,900 | $995,782 |
| Ftr | Ongoing usage and onboarding costs | $0 | $257,400 | $124,740 | $137,214 | $519,354 | $440,182 |
| Gtr | Licensing costs | $0 | $196,560 | $216,216 | $237,510 | $650,286 | $535,827 |
| Total costs (risk-adjusted) | $103,500 | $812,760 | $699,756 | $733,524 | $2,349,540 | $1,971,791 | |
Total Implementation And Ongoing Management Costs
Evidence and data. Interviewees described their experiences with implementing and managing Cortex for their organizations:
- Interviewees told Forrester that their organizations implemented Cortex with a small team of engineers working over the course of several weeks.
- After initial implementation, a small number of engineers are required for housekeeping tasks associated with managing Cortex and generating reports.
Modeling and assumptions. For the composite organization, Forrester assumes the following:
- The composite organization has a team of three engineers working for 10 weeks to implement Cortex.
- Two engineer FTEs are required for ongoing maintenance of the platform.
Risks. Factors that could impact the size of this cost for organizations include:
- The size of an initial implementation team.
- The amount of time required for implementation.
- The amount of engineer time required for ongoing management.
Results. To account for these risks, Forrester adjusted this cost upward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $996,000.
Total Implementation And Ongoing Management Costs
| Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
|---|---|---|---|---|---|---|---|
| E1 | Engineers required to work on initial implementation | Interviews | 3 | ||||
| E2 | Implementation length (weeks) | Interviews | 10 | ||||
| E3 | Fully burdened weekly rate for an engineer | A9*40 hours | $3,000 | ||||
| E4 | Subtotal: Initial implementation effort | E1*E2*E3 | $90,000 | ||||
| E5 | Engineer FTEs required for platform maintenance and report generation/presentation | Interviews | 2 | 2 | 2 | ||
| E6 | Fully burdened annual salary for an engineer | A2 | $156,000 | $156,000 | $156,000 | ||
| E7 | Subtotal: Cortex platform management and report generation costs | E5*E6 | $312,000 | $312,000 | $312,000 | ||
| Et | Total implementation and ongoing management costs | E4+E7 | $90,000 | $312,000 | $312,000 | $312,000 | |
| Risk adjustment | ↑15% | ||||||
| Etr | Total implementation and ongoing management costs (risk-adjusted) | $103,500 | $358,800 | $358,800 | $358,800 | ||
| Three-year total: $1,179,900 | Three-year present value: $995,782 | ||||||
Ongoing Usage And Onboarding Costs
Evidence and data. Interviewees explained that part of using Cortex included getting up to speed with services onboarded onto the platform, as well as the time required to work toward final development.
- In general, interviewees told Forrester that this onboarding time had been required with other systems. They considered Cortex easy to work with in this regard. The SVP and director of engineering enablement at a sports organization explained, “As an engineer, you jump on the tool and register your new service with relative ease.”
- Interviewees also described a fairly basic process of onboarding developers onto using Cortex. Many said that users could pick up the basics very quickly, and more advanced training only required a few hours of training.
Modeling and assumptions. For the composite organization, Forrester assumes the following:
- Each of the software components deployed using Cortex requires 4 hours of team engineering time for getting fully up to speed and 8 hours per new user of Cortex to onboard onto the platform.
- Initially, a pool of 240 seat holders requires training. This increases by 10% each year.
Risks. Factors that could impact the size of this cost for organizations include:
- The number of software components used with Cortex.
- The number of seat holders requiring training with Cortex.
- The amount of time required per service for seat holders to get up to speed.
- The amount of time needed per seat holder to fully train.
Results. To account for these risks, Forrester adjusted this cost upward by 5%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $515,000.
Ongoing Usage And Onboarding Costs
| Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
|---|---|---|---|---|---|---|---|
| F1 | Number of services and projects deployed using Cortex | B1 | 300 | 330 | 363 | ||
| F2 | Fully burdened hourly rate for an engineer | A9 | $75 | $75 | $75 | ||
| F3 | Average engineering hours per service and project to come up to speed on usage with Cortex | Interviews | 4 | 4 | 4 | ||
| F4 | Subtotal: Cost of onboarding services and projects with Cortex | F1*F2*F3 | $90,000 | $99,000 | $108,900 | ||
| F5 | Net-new users needing to be onboarded on Cortex each year | 240 initial users, 10% growth | 240 | 24 | 26 | ||
| F6 | Onboarding hours required per developer | Interviews | 8 | 8 | 8 | ||
| F7 | Subtotal: Cost of onboarding developers to Cortex | F5*F6*F2 | $144,000 | $14,400 | $15,840 | ||
| Ft | Ongoing usage and onboarding costs | F4+F7 | $0 | $234,000 | $113,400 | $124,740 | |
| Risk adjustment | ↑10% | ||||||
| Ftr | Ongoing usage and onboarding costs (risk-adjusted) | $0 | $257,400 | $124,740 | $137,214 | ||
| Three-year total: $519,354 | Three-year present value: $440,182 | ||||||
Licensing Costs
Evidence and data. Cortex charges organizations a monthly per-user license fee for usage of the platform.
Modeling and assumptions. For the composite organization, Forrester assumes the following:
- The composite organization has a total of 240 individual seat holders for Cortex. This includes engineers, team leads, managers, and other roles that require access to the platform.
- The seat holders increase 10% year over year (rounded) as adoption increases within the organization.
- Each seat holder comes with a license fee of $65 per month for a total of $780 per year per seat holder.
Risks. Factors that could impact the size of this cost for organizations include:
- The number of seat holders.
- The rate that seat holders increase over time.
- Any changes in pricing from Cortex
Results. To account for these risks, Forrester adjusted this cost upward by 5%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $536,000.
Licensing Costs
| Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
|---|---|---|---|---|---|---|---|
| G1 | Total seat holders for Cortex | Composite | 240 | 264 | 290 | ||
| G2 | Annual license fee per seat holder | $65*12 months | $780 | $780 | $780 | ||
| Gt | Licensing costs | G1*G2 | $0 | $187,200 | $205,920 | $226,200 | |
| Risk adjustment | ↑5% | ||||||
| Gtr | Licensing costs (risk-adjusted) | $0 | $196,560 | $216,216 | $237,510 | ||
| Three-year total: $650,286 | Three-year present value: $535,827 | ||||||
Consolidated Three-Year, Risk-Adjusted Metrics
Cash Flow Chart (Risk-Adjusted)
Total costs Total benefits Cumulative net benefits-
The financial results calculated in the Benefits and Costs sections can be used to determine the ROI, NPV, and payback period for the composite organization’s investment. Forrester assumes a yearly discount rate of 10% for this analysis.
These risk-adjusted ROI, NPV, and payback period values are determined by applying risk-adjustment factors to the unadjusted results in each Benefit and Cost section.
Cash Flow Analysis (Risk-Adjusted Estimates)
| Initial | Year 1 | Year 2 | Year 3 | Total | Present Value | |
|---|---|---|---|---|---|---|
| Total costs | ($103,500) | ($812,760) | ($699,756) | ($733,524) | ($2,349,540) | ($1,971,791) |
| Total benefits | $0 | $2,406,588 | $2,570,106 | $2,751,320 | $7,728,014 | $6,378,969 |
| Net benefits | ($103,500) | $1,593,828 | $1,870,350 | $2,017,796 | $5,378,474 | $4,407,178 |
| ROI | 224% | |||||
| Payback | <6 months | |||||
Appendix A: Total Economic Impact
Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.
Total Economic Impact Approach
-
Benefits represent the value delivered to the business by the product. The TEI methodology places equal weight on the measure of benefits and the measure of costs, allowing for a full examination of the effect of the technology on the entire organization.
-
Costs consider all expenses necessary to deliver the proposed value, or benefits, of the product. The cost category within TEI captures incremental costs over the existing environment for ongoing costs associated with the solution.
-
Flexibility represents the strategic value that can be obtained for some future additional investment building on top of the initial investment already made. Having the ability to capture that benefit has a PV that can be estimated.
-
Risks measure the uncertainty of benefit and cost estimates given: 1) the likelihood that estimates will meet original projections and 2) the likelihood that estimates will be tracked over time. TEI risk factors are based on “triangular distribution.”
The initial investment column contains costs incurred at “time 0” or at the beginning of Year 1 that are not discounted. All other cash flows are discounted using the discount rate at the end of the year. PV calculations are calculated for each total cost and benefit estimate. NPV calculations in the summary tables are the sum of the initial investment and the discounted cash flows in each year. Sums and present value calculations of the Total Benefits, Total Costs, and Cash Flow tables may not exactly add up, as some rounding may occur.
Appendix B: Supplemental Material
Related Forrester Research
Developer Experience Is New Territory For Most, Forrester Research, Inc., May 13, 2024.
Top Recommendations For Development Leaders, 2024, Forrester Research, Inc., May 6, 2024.
Appendix C: Endnotes
1 Base: 30 developer experience leaders and nine developers; Forrester’s Q4 2023/Q1 2024 Developer Experience Survey.
2 Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.
PDF
