The Total Economic Impact™ Of IBM Z IntelliMagic Vision For z/OS
Cost Savings And Business Benefits Enabled By IBM Z IntelliMagic Vision for z/OS
A Forrester Total Economic Impact™ Study Commissioned By IntelliMagic, An IBM Company, July 2024
The mainframe industry is undergoing significant changes as market dynamics evolve. Enterprises rely on mainframes for critical functions such as large-scale transaction processing, high-security systems, and real-time quantum-safe encryption. But mainframe operation can be complex and overwhelming. To address these needs, IBM Z IntelliMagic Vision for z/OS (IntelliMagic Vision) offers a comprehensive solution that reduces business continuity risk and optimizes resource allocation within organizations’ mainframes. By leveraging advanced capabilities, IBM Z IntelliMagic Vision for z/OS empowers mainframe teams to navigate the future with performance and availability confidence, ensuring seamless operations and enabling growth opportunities.
IntelliMagic Vision is a performance management and optimization software for IBM Z systems that modernizes mainframe analytics with advanced intelligence. It offers flexible data analysis through custom visualizations, helping technical specialists identify risks and optimize workloads. With interactive insights and customizable reports, it enables effective management of application availability, performance, and costs, while also providing accelerated learning for newer staff and streamlined issue analysis for experienced staff.
IntelliMagic, an IBM Company commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study and examine the potential return on investment (ROI) enterprises may realize by deploying IntelliMagic Vision.1 The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of IntelliMagic Vision on their organizations.
Return on investment (ROI)
161%
Net present value (NPV)
$2.55M
To better understand the benefits, costs, and risks associated with this investment, Forrester interviewed four representatives with experience using IntelliMagic Vision. For the purposes of this study, Forrester aggregated the interviewees’ experiences and combined the results into a single composite organization that is a US financial services, banking, and insurance provider with an annual revenue of 19 billion dollars, serving 15 million customers.
Interviewees said that prior to using IntelliMagic Vision, their organizations handled performance management in an ad hoc manner using either standard z/OS components or conducting mainframe performance management tasks manually using status quo reporting. However, these solutions yielded limited success in proactively avoiding application availability problems and in solving problems quickly. The incumbent solutions’ limitations hindered mainframe teams’ ability to gain insights, work efficiently, and mitigate risks, while mainframe talent and skills shortages worsened the challenges in managing and maintaining the systems.
After the investment in IntelliMagic Vision, the interviewees integrated IntelliMagic Vision within their daily mainframe operations where they now are able to have a clear and timely understanding of what is happening in their mainframe environment. Key results from the investment include enhanced visibility into performance for key applications and reduced business continuity risks. It has also enabled teams to see and leverage safe opportunities for optimizing resource allocation, resulting in cost savings, and addressed talent gaps for a smooth transition and minimal disruptions in the mainframe teams.
Key Findings
Quantified benefits. Three-year, risk-adjusted present value (PV) quantified benefits for the composite organization include:
- Business continuity risk reduction and dependability. IBM Z IntelliMagic Vision for z/OS provides organizations with enhanced visibility into their mainframe system, helping them proactively anticipate and prevent application downtime, which can amount to up to 830 minutes per year. With near real-time performance monitoring and advanced data analysis capabilities, IntelliMagic Vision enables organizations to swiftly resolve issues and even prevent them from occurring, resulting in improved profitability and smoother operations. By taking a proactive approach enabled by explainable AI capabilities, the solution helps to ensure business continuity, eliminate costly downtime, and enable seamless operations.
- Hardware and software costs savings of 1.18 million. By providing evidence and data, IBM Z IntelliMagic Vision for z/OS allows organizations to understand the utilization of resources in their mainframe environment. This understanding helps them predict future needs, allocate resources effectively, and exchange operational expenditures for capital expenditures. With this proactive approach to resource management, organizations can avoid surprises and optimize their mainframe environment.
- Productivity savings in the mainframe operations team. IntelliMagic offers a solution to address talent gaps within mainframe teams caused by retirements. By leveraging IntelliMagic Vision, organizations can effectively ensure the transition of complex mainframe operations from retiring senior performance analysts to less tenured personnel, minimizing risk and disruption to critical business workloads. Organizations can avoid the need to significantly increase team sizes by more effective and efficient mainframe monitoring with the help of IntelliMagic Vision.
Unquantified benefits. Benefits that provide value for the composite organization but are not quantified for this study include:
- Successful partnership. The partnership between the IntelliMagic Vision team and its customers contributes to successful implementation and adoption of IntelliMagic Vision. All interviewees stressed the positive and productive relationship, ensuring mutual success and continuous advancements in their mainframe operations.
- Enhanced access to mainframe performance operations best practices and trends. IBM Z IntelliMagic Vision uses aggregated customer data to identify best practices and provide valuable recommendations in mainframe operations. This empowers customers to optimize performance based on consolidated information.
- User-friendly onboarding. IntelliMagic Vision is intuitive and easy to use, allowing even non-mainframe personnel to understand and review mainframe performance information. This simplifies and fast tracks the onboarding and training of early-career mainframe staff, proving invaluable in equipping them with necessary skills in the talent-scarce mainframe industry.
- Performance management improvement. Due to its enhanced visibility into how resources are being used in the mainframe, the interviewees shared how they were better able to improve their mainframe performance management, for example by proactively detecting abnormal transactions or application behavior.
Costs. Three-year, risk-adjusted PV costs for the composite organization include:
- Total vendor fees of $1.51 million. This includes implementation costs by IntelliMagic and annual IBM Z IntelliMagic Vision for z/OS subscription fees.
- Total internal costs of $79,000. This includes implementation costs by a senior mainframe performance analyst, as well as monthly internal maintenance costs.
The representative interviews and financial analysis found that a composite organization experiences benefits of $4.14M over three years versus costs of $1.58M, adding up to a net present value (NPV) of $2.55M and an ROI of 161%.
Key Statistics
-
Return on investment (ROI)
161%
-
Benefits PV
$4.14M
-
Net present value (NPV)
$2.55M
Benefits (Three-Year)
Business continuity risk reduction and dependability Total hardware and software cost savings Productivity savings in the mainframe operations teamTEI Framework And Methodology
From the information provided in the interviews, Forrester constructed a Total Economic Impact™ framework for those organizations considering an investment in IBM Z IntelliMagic Vision for z/OS.
The objective of the framework is to identify the cost, benefit, flexibility, and risk factors that affect the investment decision. Forrester took a multistep approach to evaluate the impact that IBM Z IntelliMagic Vision for z/OS can have on an organization.
-
Due Diligence
Interviewed IntelliMagic stakeholders and Forrester analysts to gather data relative to IBM Z IntelliMagic Vision for z/OS.
-
Interviews
Interviewed four representatives at organizations using IBM Z IntelliMagic Vision for z/OS to obtain data about costs, benefits, and risks.
-
Composite Organization
Designed a composite organization based on characteristics of the interviewees’ organizations.
-
Financial Model Framework
Constructed a financial model representative of the interviews using the TEI methodology and risk-adjusted the financial model based on issues and concerns of the interviewees.
-
Case Study
Employed four fundamental elements of TEI in modeling the investment impact: benefits, costs, flexibility, and risks. Given the increasing sophistication of ROI analyses related to IT investments, Forrester’s TEI methodology provides a complete picture of the total economic impact of purchase decisions. Please see Appendix A for additional information on the TEI methodology.
Disclosures
Readers should be aware of the following:
This study is commissioned by IntelliMagic, an IBM Company and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.
Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the study to determine the appropriateness of an investment in IBM Z IntelliMagic Vision for z/OS (IntelliMagic Vision).
IntelliMagic reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.
IntelliMagic provided the customer names for the interviews but did not participate in the interviews.
Drivers leading to the IBM Z IntelliMagic Vision for z/OS investment
Interviews
| Role | Industry | Region | Revenue | Customers | Mainframe Team |
|---|---|---|---|---|---|
| Mainframe infrastructure lead | Financial services, insurance, and banking | USA | $36 billion | 13 million members | 4 |
| Mainframe operations team lead | Financial services and insurance | USA | $9 billion | 16 million members | 5 |
| Mainframe lead | Financial services and banking | USA | $4 billion | 3 million customers | 9 |
| Principal engineer | Financial services and banking | South America | $30 billion | 80 million customers | 15 |
Key Challenges
Before implementing IntelliMagic Vision, the environments for the four interviewees’ mainframe teams were characterized by the use of a primarily manual legacy solution to conduct trending and export utilization reports of the mainframe. A dedicated individual handled performance measurement, and the analysis and processes of the mainframe were done on an ad hoc basis. One interviewee mentioned that their organization had an old legacy solution on-premises for their mainframe performance operations management. Another interviewee highlighted that they did not have any solution in place because they could not find one capable of handling their company’s large environment.
The interviewees noted how their organizations struggled with common challenges, including:
- Limited functionality of legacy performance analysis solutions within a complex mainframe. Mainframes are complex systems that require deep knowledge and expertise to manage effectively. Legacy Solutions used for reporting on the mainframe was static and lacked the necessary coding capabilities to generate dynamic visual reports. This made it difficult for the mainframe teams to gain insights and take timely action. Additionally, the legacy solution’s learning curve was steep. The principal engineer at a financial services and banking organization disclosed that their organization encountered challenges in locating a solution that could adequately handle their substantial amount of storage.
- Inefficient and slow information retrieval with organizational business risk. The mainframe operations team lead at a financial services and insurance company expressed dissatisfaction with the legacy solution’s inadequate dynamic capabilities and slow information retrieval. This resulted in significant delays and inefficiencies in their workflow, posing a potential business risk to the organization. The team’s inability to promptly access and analyze performance data hindered their decision-making and timely actions, jeopardizing the smooth operation of critical business processes. Addressing this pain point was crucial to mitigate the associated business risk and ensure the seamless functioning of the mainframe environment.
- Lack of adaptability and advancement. The rigid and slow processes of the legacy solution did not effectively address performance problems, according to the mainframe lead at financial services and banking company. The reports generated by the solution were adequate but did not keep up with advancements made within the mainframe environment.
- Skills shortage. Finding and retaining skilled professionals who can manage and maintain mainframe systems is a significant challenge. The mainframe workforce is aging, and there is a lack of new talent entering the field.
Figure 1
“In the next two years, will you be increasing or decreasing your use of a mainframe?”
Base: 282 global infrastructure hardware technology decision-makers assigned to the server survey track whose organization uses a mainframe
Source: Forrester’s Infrastructure Hardware Survey, 2023
Solution Requirements/Investment Objectives
The interviewees’ organizations searched for a solution that could:
- Enhance data visibility and understanding. The solution needed to enable the creation of dynamic and flexible visual representations instead of relying primarily on tables and static reporting, providing a more comprehensive and easily understandable view of the voluminous and complex data. This would enhance understanding and decision-making from the data for a wider audience and facilitate transparent processes aligned with organizational goals.
- Enable comprehensive information management. The solution had to ensure that all relevant information was readily available, eliminating the need for extensive coding and enabling the generation of dynamic visual reports. This would provide a quick and efficient way to access information, supporting informed decision-making and operational effectiveness.
- Provide a modern approach and user-friendly interface. Interviewees sought a solution with a modern approach and a user-friendly interface to facilitate ease of use and adoption. This could enhance usability and promote efficient utilization of the solution across the organization.
Composite Organization
Based on the interviews, Forrester constructed a TEI framework, a composite company, and an ROI analysis that illustrates the areas financially affected. The composite organization is representative of the four interviewees, and it is used to present the aggregate financial analysis in the next section. The composite organization has the following characteristics:
Description of composite. The composite organization is a nationwide financial services, banking, and insurance provider based in the United States. With an annual revenue of $19 billion, it serves 15 million customers and employs a workforce of 20,000 individuals. Within the mainframe operations team, the firm has a total of six members: five senior mainframe performance analysts and one junior analyst.
Deployment characteristics. The composite organization implemented IBM Z IntelliMagic Vision for z/OS on the cloud and subscribed to the System, CICS, Db2, Disk and Replication, and MQ modules.
Quantified benefit data as applied to the composite
Total Benefits
| Ref. | Benefit | Year 1 | Year 2 | Year 3 | Total | Present Value |
|---|---|---|---|---|---|---|
| Atr | Business continuity risk reduction and dependability | $796,800 | $796,800 | $796,800 | $2,390,400 | $1,981,524 |
| Btr | Total hardware and software cost savings | $352,000 | $544,000 | $544,000 | $1,440,000 | $1,178,302 |
| Ctr | Productivity savings in the mainframe operations team | $279,900 | $365,400 | $559,800 | $1,205,100 | $977,024 |
| Total benefits (risk-adjusted) | $1,428,700 | $1,706,200 | $1,900,600 | $5,035,500 | $4,136,850 | |
Business Continuity Risk Reduction And Dependability
Evidence and data. The interviewees have reaped substantial advantages from the enhanced visibility into the mainframe system. This heightened visibility has empowered their teams to proactively anticipate and prevent previously unseen issues, enabling them to address potential disruptions before they occur. As a result, they have successfully safeguarded application availability and business continuity, eliminated costly downtime, identified cost savings opportunities that do not increase risk of downtime, and ensured seamless operations.
Consistent monitoring and management have played a crucial role in detecting, resolving, and preventing issues. This proactive approach has led to a significant reduction or elimination of user-impacting downtime, resulting in improved availability, performance, and resiliency.
- A Forrester survey found that 85% of respondents’ companies face unplanned downtime at least every other month — with 41% facing unplanned downtime every month. Downtime disrupts business operations and, in turn, causes loss of productivity and revenue. IT leaders indicated that the costliest aspects of downtime are lost revenue (53%), lost productivity (47%), and the negative impact on corporate reputation (41%).2
-
A principal engineer at a financial services and banking organization said that IntelliMagic Vision has been an invaluable tool for his team and the wider organization, giving them the ability to prevent crises and secure consistent operations. The interviewee said: “We have many systems. Without IntelliMagic Vision, we wouldn’t be able to understand how those systems work together.” After implementation, they saw an increase in smoother operations and improved profitability for whole financial institution.
By leveraging IntelliMagic Vision’s predictive and near-real-time performance monitoring capabilities, the mainframe operations team could identify and address issues before they escalated into system failures, avoiding disruptions to their banking services and financial losses. The software’s data consolidation and analysis capabilities provided the team with a comprehensive view of the system’s performance, enabling them to swiftly resolve issues or even prevent them before they occurred.
The principal engineer said, “On average, once per month, IntelliMagic Vision has helped us prevent applications from stopping, ensuring uninterrupted service delivery and saving us from costly system failures.” - A mainframe operations team lead from a financial services and insurance company said that their organization has implemented business continuity and disaster recovery measures, including synchronous and asynchronous replication, to ensure data availability and minimize downtime. In addition to their mainframe’s high stability, the interviewee highlighted their appreciation for the better and enhanced visibility on the mainframe for observability purposes using IntelliMagic Vision. They said, “IntelliMagic Vision allowed us to open the black box of the mainframe, so it’s not a black box for your organization and for your management anymore.”
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- It has an average of 830 minutes of downtime per year.3
- For the composite organization Forrester assumes an average cost per minute of $12,000 per minute of downtime. This assumed value remains conservative, especially for the financial services industry.
- The downtime experienced by the composite organization cannot be solely attributed to the mainframe. It is assumed that only 10% of the downtime can be attributed to issues related to the mainframe operations.
Risks. This benefit can vary by organization based on different factors, including the following:
- The assumed cost per minute can vary significantly per organization, as mainframe downtime has a direct impact on revenue by affecting applications that are core to multiple lines of business.
- Ensuring technical compatibility poses a potential risk when implementing IntelliMagic Vision, as it may require seamless integration with existing systems, applications, and infrastructure. Failure to address compatibility challenges could result in disruptions or integration issues during the transition process.
- Even when IntelliMagic Vision detects a problem, it still relies on the expertise of a mainframe performance analyst to promptly resolve the issue and prevent potential downtime.
Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1.7 million.
Business Continuity Risk Reduction And Dependability
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| A1 | Average minutes of unplanned downtime per year | Forrester research | 830 | 830 | 830 |
| A2 | Average cost per minute of unplanned downtime | Assumption | $12,000 | $12,000 | $12,000 |
| A3 | Reduction in unplanned downtime due to IntelliMagic Vision resources’ preventive monitoring | Assumption | 10% | 10% | 10% |
| At | Business continuity risk reduction and dependability | A1*A2*A3 | $996,000 | $996,000 | $996,000 |
| Risk adjustment | ↓20% | ||||
| Atr | Business continuity risk reduction and dependability (risk-adjusted) | $796,800 | $796,800 | $796,800 | |
| Three-year total: $2,390,400 | Three-year present value: $1,981,524 | ||||
Total Hardware And Software Cost Savings
Evidence and data. Most of the interviewees mentioned to Forrester how the solution helped them to understand the utilization of different resources in their mainframe environment. They felt better equipped to understand the drivers of performance and utilization and better predict future needs. For example, they were able to positively exchange future opex expenditures, such as MIPS/MSUs incremental costs, by capex. They were able to proactively respond to emergent issues, avoiding surprise spikes in resource utilization and degraded performance.
- One interviewee’s organization converted COBOL applications to Java, utilizing z/OS Connect APIs rather than IBM MQ for message transport between applications. Because z/OS Connect can use zIIP engines, the interviewee was able to save future MIPS/MSUs software costs. Predictive analytics from IntelliMagic Vision allowed the interviewee to project resource utilization several months out in advance showing that they would need additional zIIP processors to handle the new messaging workload. Running the new z/OS Connect instructions on general processors would drive up costs. These additional costs would have been around $300,000 a month in software costs. Using IntelliMagic Vision allowed them to execute a single capex investment of $1.2 million in additional zIIPs to meet increased workload needs and avoiding ongoing MSU charges from general-purpose CPU usage.
- Another interviewee explained how they were able, in just 6 months, to save 1.1 PB in storage, saving them $1,000 per TB. With IntelliMagic Vision, they were able to track the utilization of storage for each application, identifying several LUNs (logical unit number) without any application I/O for a period of years, meaning the storage space reserved was not utilized.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- The composite organization invests $240,000 in zIIP to accommodate conversion from IBM MQ to z/OS Connect APIs for passing messages between workloads. Doing so displaces annual ongoing MIPS/MSU charges related to IBM MQ usage of general processors.
- The composite organization saves 440 TB of storage from unutilized LUNs allocated to application with a yearly TCO of $1,000 per TB.
Risks. This benefit can vary by organization based on different factors, including the following:
- Not all organizations will have workloads they can move from general processors to zIIP.
- Potential savings from unutilized LUNs depends on each organization and will vary significantly.
Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of nearly $1.2 million.
Total Hardware And Software Cost Savings
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| B1 | MIPS/MSU incremental costs avoided by moving IBM MQ to z/OS Connect running in zIIP processors | Interviews | $240,000 | $240,000 | $240,000 |
| B2 | Investment in specialty zIIP processors to accommodate z/OS connect workloads | Interviews | $240,000 | ||
| B3 | Subtotal: Incremental software cost avoided from zIIP engines optimization | B1-B2 | $0 | $240,000 | $240,000 |
| B4 | Avoided cumulative mainframe storage purchase (TB) | Interviews | 440 | 440 | 440 |
| B5 | Storage total annual cost of ownership per TB | Interviews | $1,000 | $1,000 | $1,000 |
| B6 | Subtotal: Storage savings | B4*B5 | $440,000 | $440,000 | $440,000 |
| Bt | Total hardware and software cost savings | B3+B6 | $440,000 | $680,000 | $680,000 |
| Risk adjustment | ↓20% | ||||
| Btr | Total hardware and software cost savings (risk-adjusted) | $352,000 | $544,000 | $544,000 | |
| Three-year total: $1,440,000 | Three-year present value: $1,178,302 | ||||
Productivity Savings In The Mainframe Operations Team
Evidence and data. The interviewees revealed a common challenge regarding retirements within their teams and the subsequent difficulties in hiring and training new talent. IBM Z IntelliMagic Vision for z/OS played a vital role in addressing this issue by supporting the organizations in filling the talent gap when senior performance analysts retired.
With the implementation of IntelliMagic Vision, the interviewees were able to successfully replace retired senior performance analysts with newer analysts and provide them with the necessary training and onboarding support. This enabled a smooth transition and ensured the new hires could effectively perform their roles within the organization with little to no impact on the mainframe team’s performance operations.
- Prior to the implementation of IntelliMagic Vision, the mainframe infrastructure lead at a financial services, insurance, and banking organization discerned the necessity for the incorporation of three more senior-level mainframe performance analysts to attain ideal mainframe operations. Following the implementation of IntelliMagic Vision, the infrastructure lead and their team effectively surmounted the persistent hurdle of procuring skilled and suitable senior mainframe performance analysts, thereby successfully recruiting and training less-tenured analysts, within an optimizing team structure.
- Without IntelliMagic Vision, the infrastructure performance team at a financial services and banking company would have needed to double in size, from 15 members to 30, in order to adequately monitor the mainframe and cover areas such as storage, Db2, CICS, and system management. Due to the prevailing tight labor market, as mentioned earlier, the principal engineer mentioned that locating the necessary 15 staff members to fulfill the staffing requirements would be a challenge.
- Implementing IntelliMagic Vision had a positive impact on the mainframe teams at the financial services and banking company, according to the mainframe lead. Not only did it address the talent shortage in the mainframe industry, but it also resulted in the elimination of substantial work related to sourcing performance management information from other teams. IntelliMagic Vision streamlined collaboration and enhanced the efficiency between the mainframe performance team and other teams, such as developers, by offloading processing from the mainframe. This, in turn, reduced the performance team’s dependency on multiple teams responsible for supporting the distributed environment. As a result, the mainframe performance teams were freed from the time and effort required for patching and upgrading activities, which were previously necessary to maintain the distributed environment. This improved workflow fosters seamless coordination and cooperation among teams, leading to enhanced productivity and streamlined operations.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- The composite organization has five senior mainframe performance analysts and one junior analyst, with the expected retirement of four of its senior performance analysts over a three-year period.
- The composite organization is able to find and hire a junior mainframe performance analyst in Year 1 to cover the retirement of two senior members. In Year 2, the composite hires a second junior analyst in anticipation of the retirement of the two senior members in Years 2 and 3.
- By utilizing IntelliMagic Vision, the composite organization is able to effectively handle the growing workload with a smaller team as well as address the shortage of skilled talent that the mainframe team is currently grappling with.
- The average fully burdened annual cost of a senior mainframe performance analyst is $216,000. The average fully burdened annual cost of a junior mainframe analyst is $121,000.
Risks. This benefit can vary by organization based on different factors, including the following:
- Junior performance analysts may not possess the same level of experience as senior analysts within the mainframe team, which could potentially pose a risk to a smooth transition. The presence of legacy applications and custom-coded manual reports and processes further increases the complexity and potential challenges during the transition process.
- Due to the talent shortages in the market, a senior mainframe analyst may need to be hired, which could introduce an additional risk due to the existing talent gap. This risk arises from the challenges associated with finding and securing qualified candidates with the necessary expertise and experience in mainframe performance operations.
Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $977,000.
Productivity Savings In The Mainframe Operations Team
| Ref. | Metric | Source | Year 1 | Year 2 | Year 3 |
|---|---|---|---|---|---|
| C1 | Senior mainframe performance analysts retired in the mainframe team compared to the initial team | Composite | 2 | 3 | 4 |
| C2 | Average fully burdened yearly cost of a senior mainframe performance analyst | Interviews | $216,000 | $216,000 | $216,000 |
| C3 | Junior mainframe performance analysts hired to the initial team | Composite | 1 | 2 | 2 |
| C4 | Average fully burdened yearly cost of a junior mainframe performance analyst | Interviews | $121,000 | $121,000 | $121,000 |
| Ct | Productivity savings in the mainframe operations team | A1*A2-A3*A4 | $311,000 | $406,000 | $622,000 |
| Risk adjustment | ↓10% | ||||
| Ctr | Productivity savings in the mainframe operations team (risk-adjusted) | $279,900 | $365,400 | $559,800 | |
| Three-year total: $1,205,100 | Three-year present value: $977,024 | ||||
Unquantified Benefits
Interviewees mentioned the following additional benefits that their organizations experienced but were not able to quantify:
-
Partnership with the IntelliMagic Vision team. Across all interviews, respondents expressed their satisfaction with the highly beneficial partnership established with the entire team supporting IntelliMagic Vision. This partnership significantly contributed to the successful implementation, deployment, and adoption of IntelliMagic Vision.
The principal engineer at a financial services and banking company said the partnership between the IntelliMagic Vision team and their organization has led to ongoing enhancements and improvements in the software. The responsiveness and support provided by IntelliMagic play a crucial role in achieving optimal results for their organization. This collaboration has fostered a positive and productive relationship, ensuring mutual success and continuous advancements in their operations.
- Enhanced access to mainframe performance best practices and trends. IntelliMagic’s ability to aggregate and analyze customer data plays a crucial role in identifying best practices within mainframe operations. By studying how customers utilize their product, IntelliMagic gains valuable insights that allow them to recommend optimization strategies and industry trends through their product offerings. This enhanced access to mainframe operations best practices and trends empowers customers to make informed decisions and improve performance based on consolidated information derived from aggregated customer data.
-
User-friendliness and onboarding. Two of the interviewees specifically highlighted the user-friendliness of IntelliMagic Vision. The mainframe lead at a financial services and banking company mentioned that even non-mainframe personnel could easily understand the system.
The interviewees highlighted IntelliMagic Vision as a tool that simplifies the onboarding and training of newly hired junior employees. Particularly in the mainframe industry, where skilled talent is in short supply, IntelliMagic Vision has proven to be an invaluable asset to rapidly equip junior mainframe performance analysts with the necessary skills. - Mainframe performance management improvements. Most of the interviewees mentioned how the solution was able to help them improve performance of the mainframe since, as one stated: “Everyone can see when there is a performance problem, everyone can see and look for their data; application people can see how their applications are using the system. (Some already changed their applications because they found problems from IntelliMagic Vision outputs.)”
Flexibility
The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement IntelliMagic Vision and later realize additional uses and business opportunities.
As shown in Figure 1, companies are increasing their use of mainframes. This implies that the workload for the mainframe operations team will increase in the future. By having IntelliMagic Vision in place today, companies will be better able to anticipate the workload for their mainframe performance team as it grows. This will ensure stable operations and revenue for the wider organization.
Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Appendix A).
Quantified cost data as applied to the composite
Total Costs
| Ref. | Cost | Initial | Year 1 | Year 2 | Year 3 | Total | Present Value |
|---|---|---|---|---|---|---|---|
| Dtr | IBM Z IntelliMagic Vision for z/OS subscription costs | $110,000 | $561,000 | $561,000 | $561,000 | $1,793,000 | $1,505,124 |
| Etr | Total internal maintenance costs | $19,536 | $23,760 | $23,760 | $23,760 | $90,816 | $78,624 |
| Total costs (risk-adjusted) | $129,536 | $584,760 | $584,760 | $584,760 | $1,883,816 | $1,583,748 | |
IBM Z IntelliMagic Vision For z/OS Subscription Costs
Evidence and data. Three interviewees stated their organization utilizes IBM Z IntelliMagic Vision for z/OS as a cloud solution. They incur costs for service and support, as well as for storage and server administration. The licensing fees are determined through a contractual agreement, and the organization has a beneficial license agreement with IntelliMagic that grants them perpetual licenses for certain modules based on their contributions.
- Additional services. In addition to the features of IBM Z IntelliMagic Vision for z/OS, the contract also encompasses a range of services. One notable service, as highlighted by an interviewee, is the provision of 4 hours of consulting services per month from IntelliMagic, which is included in the contract.
- Cloud vs. on-prem. According to one interviewee, migrating IntelliMagic Vision from on-premises to the cloud resulted in significant cost savings, resource optimization, and risk reduction.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- The implementation costs, which include services provided by “IntelliMagicians” (as their implementation team personnel are called) total $100,000.
- The composite organization incurs an annual subscription fee of $510,000, including CICS, Db2, Disk and Replication, and MQ modules. The subscription also covers a certain number of hours of consulting and support by IntelliMagic consultants.
Risks. This cost can vary by organization based on different factors, including the following:
- The IBM Z IntelliMagic Vision for z/OS subscription fee may differ due to the need for additional modules depending on the size and complexity of the mainframe environment.
- Implementation time and related costs might differ based on implemented modules, mainframe complexity, and client needs.
Results. To account for these risks, Forrester adjusted this cost upward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1.5 million.
IBM Z IntelliMagic Vision For z/OS Subscription Costs
| Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
|---|---|---|---|---|---|---|---|
| D1 | Professional services (implementation) | Interviews | $100,000 | ||||
| D2 | Technology usage (subscription) and professional services | Interviews | $510,000 | $510,000 | $510,000 | ||
| Dt | IBM Z IntelliMagic Vision for z/OS subscription costs | D1+D2 | $100,000 | $510,000 | $510,000 | $510,000 | |
| Risk adjustment | ↑10% | ||||||
| Dtr | IBM Z IntelliMagic Vision for z/OS subscription costs (risk-adjusted) | $110,000 | $561,000 | $561,000 | $561,000 | ||
| Three-year total: $1,793,000 | Three-year present value: $1,505,124 | ||||||
Total Internal Maintenance Costs
Evidence and data. During the interviews, all interviewees mentioned incurred internal costs associated with both the implementation and ongoing maintenance of IntelliMagic Vision. The implementation process varied depending on the size and complexity of their mainframe systems. For instance, one interviewee mentioned their organization’s mainframe team assigned one full-time equivalent (FTE) to handle the implementation, which took approximately a month to complete, while another interviewee allocated additional team members to ensure a successful implementation.
- According to the mainframe lead at a financial services and banking organization, one FTE required approximately one week for the initial proof of concept and three weeks for full deployment.
- The interviewees reported varying levels of recurring maintenance tasks for the IntelliMagic Vision deployment, representing up to 10% of an FTE per year. The mainframe operations team lead at a financial services and insurance company estimates that by the end of the year, they will have spent around 10 days fixing issues related to the automated process.
Modeling and assumptions. Based on the interviews, Forrester assumes the following about the composite organization:
- One senior performance analyst and one junior analyst are responsible for implementing and maintaining IntelliMagic Vision in the composite mainframe environment. The junior analyst participates in the deployment to enhance their skills through training, while the senior analyst has a supervisory role.
- The implementation process will take a total of four weeks (160 hours) of focused work from the junior analyst, along with an additional 80 hours from the senior analyst. The senior analyst also dedicates 10% of their yearly time to tool maintenance.
- The average fully burdened annual cost of a senior mainframe performance analyst is $216,000.
Risks. This cost can vary by organization based on different factors, including the following:
- The implementation time for IntelliMagic Vision may vary depending on the complexity of the mainframe system, potentially requiring additional time or involvement from different members of the mainframe team to ensure successful implementation.
- Fully burdened staff costs vary according to the geographic area where they are based.
Results. To account for these risks, Forrester adjusted this cost upward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $79,000.
Total Internal Maintenance Costs
| Ref. | Metric | Source | Initial | Year 1 | Year 2 | Year 3 | |
|---|---|---|---|---|---|---|---|
| E1 | Average fully burdened hourly cost of a senior mainframe performance analyst | Interviews | $104 | ||||
| E2 | Internal deployment by senior mainframe performance analyst, including overseeing, coaching, customization, testing, and launch (hours) | Interviews | 80 | ||||
| E3 | Average fully burdened hourly cost of a junior mainframe performance analyst | Interviews | $59 | ||||
| E4 | Internal deployment by junior mainframe performance analyst, including building, customization, testing, and launch (hours) | Interviews | 160 | ||||
| E5 | Subtotal: Cost of implementation | E1*E2+E3*E4 | $17,760 | ||||
| E6 | Average fully burdened yearly cost of a senior mainframe performance analyst | Interviews | $216,000 | $216,000 | $216,000 | ||
| E7 | Ongoing administration (management, maintenance, and updates) as % of FTE | Interviews | 10% | 10% | 10% | ||
| E8 | Subtotal: Ongoing costs | E6*E7 | $21,600 | $21,600 | $21,600 | ||
| Et | Total internal maintenance costs | E5+E8 | $17,760 | $21,600 | $21,600 | $21,600 | |
| Risk adjustment | ↑10% | ||||||
| Etr | Total internal maintenance costs (risk-adjusted) | $19,536 | $23,760 | $23,760 | $23,760 | ||
| Three-year total: $90,816 | Three-year present value: $78,624 | ||||||
Consolidated Three-Year Risk-Adjusted Metrics
Cash Flow Chart (Risk-Adjusted)
Total costs Total benefits Cumulative net benefits-
The financial results calculated in the Benefits and Costs sections can be used to determine the ROI, NPV, and payback period for the composite organization’s investment. Forrester assumes a yearly discount rate of 10% for this analysis.
These risk-adjusted ROI and NPV are determined by applying risk-adjustment factors to the unadjusted results in each Benefit and Cost section.
Cash Flow Analysis (Risk-Adjusted Estimates)
| Initial | Year 1 | Year 2 | Year 3 | Total | Present Value | |
|---|---|---|---|---|---|---|
| Total costs | ($129,536) | ($584,760) | ($584,760) | ($584,760) | ($1,883,816) | ($1,583,748) |
| Total benefits | $0 | $1,428,700 | $1,706,200 | $1,900,600 | $5,035,500 | $4,136,850 |
| Net benefits | ($129,536) | $843,940 | $1,121,440 | $1,315,840 | $3,151,684 | $2,553,102 |
| ROI | 161% | |||||
Appendix A: Total Economic Impact
Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.
Total Economic Impact Approach
-
Benefits represent the value delivered to the business by the product. The TEI methodology places equal weight on the measure of benefits and the measure of costs, allowing for a full examination of the effect of the technology on the entire organization.
-
Costs consider all expenses necessary to deliver the proposed value, or benefits, of the product. The cost category within TEI captures incremental costs over the existing environment for ongoing costs associated with the solution.
-
Flexibility represents the strategic value that can be obtained for some future additional investment building on top of the initial investment already made. Having the ability to capture that benefit has a PV that can be estimated.
-
Risks measure the uncertainty of benefit and cost estimates given: 1) the likelihood that estimates will meet original projections and 2) the likelihood that estimates will be tracked over time. TEI risk factors are based on “triangular distribution.”
The initial investment column contains costs incurred at “time 0” or at the beginning of Year 1 that are not discounted. All other cash flows are discounted using the discount rate at the end of the year. PV calculations are calculated for each total cost and benefit estimate. NPV calculations in the summary tables are the sum of the initial investment and the discounted cash flows in each year. Sums and present value calculations of the Total Benefits, Total Costs, and Cash Flow tables may not exactly add up, as some rounding may occur.
Appendix B: Supplemental Material
Related Forrester Research
Select The Best Mainframe Modernization Approach For Your Enterprise, Forrester Research, Inc., February 21, 2024
Tackle The Overwhelming Challenge Of Mainframe Modernization, Forrester Research, Inc., February 29, 2024
Appendix C: Endnotes
1 Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.
2 Source: The Real Costs Of Unplanned Downtime, a commissioned study conducted by Forrester Consulting, August 2019.
3 Source: Forrester’s Infrastructure Hardware Survey, 2023.
PDF
