Additional Value Realized From More Engaging Digital Experiences

Evidence and data. Interviewees saw gains in business performance from leveraging Optimizely One’s solutions to build more dynamic and engaging digital experiences. Starting with Optimizely’s Content Marketing Platform (CMP), marketers broke down siloed operations by gaining full visibility of content workstreams. This solution helped to streamline campaign planning around audiences reached and management of timeline for deployment, accelerating time to market for content.

The flexibility of testing optimizations on web content, alongside the ease of use of Optimizely’s Content Management System, enabled marketers to quickly test custom functionality for buttons, page visuals, layouts, and any other updates to digital experiences. Regular testing and iteration on content drove higher performance in accessibility, with sites loading faster, better performance with SEO rankings, and time spent with improved navigation for customers. The assistant VP of digital enablement at an insurance organization said: “Our SEO was vastly improved with Optimizely. The SEO scores went from a C-level to an A. Our accessibility scores weren’t fully WCAG [Web Content Accessibility Guidelines]-compliant or to the standards we set for ourselves, and we’ve upgraded all of those with Optimizely.”

Integrations with Optimizely Data Platform enabled interviewees to segment audiences based on customer data, and then Optimizely Personalization helped them to tailor messaging and visuals to specific audiences. With Optimizely Commerce’s AI-based search results and recommendations based on customer behavior, interviewees made sure that customers saw relevant products. Altogether, these capabilities drove higher conversion rates at organizations as more engaged customers found products they were interested in. The principal of digital growth at a leasing organization said: “We onboarded Product Recommendations, and we tested with the same website user interface for ‘shop similar items’ functionality. We saw the click-through rate on the product recommendations page improve 200% at a 99% significance over our control.”

Modeling and assumptions. For the composite organization, Forrester assumes the following:

• On an annual basis, the organization’s web portals receive 125 million digital sessions (visitors who spend a few minutes and/or visit multiple pages).

• Each year, an additional site is built that otherwise would not have been built without Optimizely One enhancing productivity.

• Accessibility to web portals improves through optimization, better SEO, and faster site loading. Sessions increase by 2% to 5% as the organization improves its digital foundation.

• The average conversion rate on digital sessions prior to Optimizely One was 2%. After leveraging the solutions, the conversion rate improves by 4% in Year 1 and increases to an 8% improvement in Year 3. Gains for the organization are driven by higher-quality content and more effective customer targeting making for more personalized digital experiences.

• The average value of a digital customer order is $100. This represents the value of a converted session where a customer may have signed up for a service or purchased one or more items.

• A profit margin of 10% is applied to the revenue results.

Risks. Differences across organizations that may impact this benefit include the following:

• The prior state annual number of sessions on organization’s digital portals and conversion rate on those sessions.

• The average value of conversions and the profit margin applied.

• The impact level of digital performance on revenue.

Results. To account for these risks, Forrester adjusted this benefit downward by 25%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $5.2 million.

Greater Data Analysis Value

Evidence and data. Optimizely Analytics enabled interviewees to move away from importing data to spreadsheets or using tools that require manual review of data for insights. With Analytics, they saw data automatically uploaded to dashboards, improving the visibility of dozens of tracked metrics.

At interviewees’ organizations, the Analytics dashboard and visualizations accelerated the speed with which analysts could share insights across teams. This ease of use also enabled marketers and developers to self-serve analysis. Faster turnaround of insights reduced the time spent on decision-making and accelerated time to market for changes to the digital experience, helping to shorten time to value. The director of analytics engineering at a SaaS organization said: “Without a tool like Optimizely Analytics, we would have to go to data scientists and spend a bunch of time with them. Because they often could be unavailable, the product manager might have to make decisions based on either suboptimal data or on no data at all. So the solution here is to equip them with an easy-to-use tool to build the right dashboard of metrics and quickly query the right metrics.”

Interviewees paired Optimizely Analytics with optimization tests to better understand test results. At the automotive organization, analysts created dashboards in Analytics to see how specific value metrics like lead conversion rate were impacted by optimizations. The senior director of product analytics said, “It’s a huge time saving for us because we just run tests and pull in that data, and it’s distilled right there across our value events.”

This approach influenced teams at the organization to run optimization tests against Analytics for specific metrics to better understand the levers driving changes in results. The senior director of product analytics said: “Part of our maturity with Optimizely is learning to not just run more experiment tests but to run the right experiment. We’re spending more time building experiments to test for specific cases, so we’re not chasing down insights across multiple tests.”

Another key factor for interviewees was Optimizely’s data security compared to their legacy solution. The director of analytics engineering at a technology organization said: “Optimizely allows us with the current data models we’re exploring to bring in and analyze sensitive data that complies with our privacy security requirements. That wasn’t possible before.”

Modeling and assumptions. For the composite organization, Forrester assumes the following:

• There are 25 data analysts leveraging Optimizely Analytics, and each contributes $400,000 in revenue to the organization annually.

• The revenue generated by each employee improves 20% in Year 1 as they apply insights to better target and engage customers, leading to higher order values and additional recurring customer value. By Year 2, the improvement grows to 25%; by Year 3, it increases to 30%.

• A profit margin of 10% is applied to the revenue results.

Risks. Differences across organizations that may impact this benefit include the following:

• The number of data analysts leveraging Optimizely Analytics.

• The size of the organization in terms of number of employees, revenue, and profit margin.

• The level of analysis carried out on data prior to Optimizely Analytics.

Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $491,000.

Productivity In Creating Digital Experiences And Content

Evidence and data. The seamless integration of Optimizely One solutions helped to centralize operations for interviewees, enabling them to maximize the bandwidth of creatives and marketing. The assistant VP of digital enablement at an insurance company said: “We’re not having to hire a bunch of new people to manage three or four sites — we can consolidate operations into one authoring team. … When it comes to time savings, we see it as an opportunity to repurpose time for value-added content.” They added that Optimizely’s Content Marketing Platform helped consolidate workflows, with it providing greater visibility into teams’ operations and their content.

Previously, the manufacturing organization had assets sitting in different locations based on team members’ individual workstreams, leading to multiple renditions of a file type appearing on a site. Optimizely Digital Asset Management resolved file issues, the manufacturer’s VP of digital experience said: “We now have one central source of truth maintained by a team. Our CMS users and marketing creative teams are benefiting from Digital Asset Management because they can quickly find the right assets that are approved and integrate those right into the experience.”

Optimizely One’s efficiencies helped interviewees’ developers to create more engaging experiences that they otherwise wouldn’t have bandwidth to address. The principal of digital growth at a leasing organization said: “Optimizely is very flexible for our developers to modify experiences and increase their turnaround time. So if marketing asks for a landing page, they can then quickly deploy that. We have the flexibility and speed to create impact we couldn’t have had otherwise.”

The assistant VP of digital enablement at an insurance company added: “When it’s difficult to maintain content, you don’t make as engaging content; you don’t keep it as refreshed. There are better capabilities we’re starting to employ like content recommendation that help with personalization.” Features like these helped to keep content fresh for customers and required little additional hands-on time from team members.

Modeling and assumptions. For the composite organization, Forrester assumes the following:

• Ten developers work on building digital experiences, allocating half of their time to the work.

• Productivity in creating experiences accelerates with dynamic CMS templates that are easy to integrate and update, as well as increased optimization testing to validate digital experiences and features. Output improves by 30% in Year 1 and matures to 40% by Year 3.

• Ten marketers spend half of their time building content for digital experiences. Their productivity increases with Optimizely CMS, particularly around technical work, accelerating by 25% in Year 1 and rising to 35% in Year 3.

• Fifty marketers use CMP to create content for both digital and nondigital campaigns, while also orchestrating and managing campaign execution and content delivery. They then use CMS to quickly deliver that content at scale. They dedicate 10% of their time toward work specifically within these solutions. Optimizely’s ease of use and flexible capabilities cut their time spent with this work in half.

• The TEI methodology incorporates the assumption that a group of employees who experience a productivity benefit will rededicate some time to additional work and a portion to nonwork. For this time savings benefit, half of the recouped time goes toward additional work, such as building new digital experiences. The other half of time saved enables employees to take longer breaks or work fewer late shifts, qualitatively making for an improved employee experience.

Risks. Differences across organizations that may impact this benefit include the following:

• The number of developers and time dedicated to building digital experiences.

• The number of marketers and time spent working on digital experiences and creating content.

• The types of tools leveraged before adopting Optimizely One.

• The average fully burdened annual salary for employees.

Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $771,000.

Legacy Solution Cost Savings

Evidence and data. By adopting Optimizely One, interviewees were able to move on from their legacy solutions’ management and/or licensing costs. Some interviewees saw significant cost savings: For the VP of digital experience at a manufacturing organization, the costs of their prior commerce and content management solution were becoming too high to maintain. The interviewee said: “We looked at the cost to continue to maintain that solution: hosting, licensing, ongoing support. We compared that to a SaaS-based solution, and we found we could lower our cost. At the same time, we could improve our agility and customer experience. We based our decision on understanding the customer perspective and the future experiences we needed to deliver.”

For other interviewees, retiring legacy solutions was only part of a larger push to modernize their capabilities. The assistant VP of digital enablement at an insurance organization said: “We’re better positioned with a modern solution that can help us take advantage of things like AI that can bring personalization to content. It’s not just about cost savings so much as being in a better place for growth and taking advantage of strategic opportunities that the technology enables.”

The director of analytics engineering at a SaaS company noted that Optimizely Analytics helped them significantly with compute costs. Their legacy solution was poorly configured, and they had to run dedicated instances to power compute. The interviewee said: “If we’re thinking about our total compute footprint, it’s going down, and we’re going to save money. We’re optimizing our use of compute because we have more control over data in our environment and when we need compute power.”

Modeling and assumptions. For the composite organization, Forrester assumes the following:

• The legacy solutions the organization plans to phase out total $400,000 in annual costs.

• Each year, the organization phases out technology that overlaps with an Optimizely One solution, saving one-third of annual costs.

Risks. Differences across organizations that may impact this benefit include the following:

• The number of legacy solutions that the organization plans to phase out.

• The fees and oversight costs for legacy solutions.

• The speed with which the organization plans to phase out legacy solutions.

Results. To account for these risks, Forrester adjusted this benefit downward by 20%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $650,000.

Unquantified Benefits

Interviewees mentioned the following additional benefits that their organizations experienced but were not able to quantify:

• Optimizely AI capabilities streamline operations. Interviewees’ use cases for Optimizely One’s AI tool Opal are in their early stages, but they have been encouraged by their experiences thus far. They found Opal’s ability to generate and format testing plans has streamlined setting up optimizations for features and digital experiences. In addition, querying Opal for dashboard results analysis, testing summarization, and follow-on test ideas has accelerated their analysis of optimization tests. The principal of digital growth at a leasing organization said that Optimizely’s use of agentic AI could unlock further value: “Some of the agents they’re introducing in Opal could be very, very useful. Especially if it comes to the point of being able to handle some of the more menial regular tasks like outputting formatted document summaries of tests that need to be seen by a stakeholder group. If I can get an agent set up that does that kind of thing, I’ll run that agent every week.”

• Employee experience positivity increases from enhanced operations. Once the interviewees were set up with Optimizely One solutions, their teams were eager to leverage them. The assistant VP of digital enablement at an insurance organization said: “We’ve got multiple business areas saying they want to move our website over to the new CMS platform. They’re knocking on the door, saying, ‘We heard you have this better thing.’” A key driver of excitement was ease of use, with the solutions encouraging self-service among employees. Employees no longer had to wait for engineers or developers to add website features — they could insert them into templates and begin testing. This change reduced tensions around lengthy development timelines, improving relations between teams. Employees also felt more confident about applying updates to digital experiences because of Optimizely’s proprietary Stats Engine, which gives experimenters forward-looking recommendations based on statistical significance. As a result, teams moved forward with stronger belief that they could use these insights to improve key metrics like revenue uplift or conversions.

Flexibility

The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement Optimizely One and later realize additional business value, including:

• Incorporating offline data into Optimizely Analytics to improve customer experiences. Interviewees were interested in incorporating offline events data in Optimizely Analytics to better understand how digital experiences influence audiences beyond what they had been tracking. At the automotive organization, transactions are primarily offline with automobiles, the senior director of product analytics said: “With that use case, we appreciate the power of Optimizely’s Warehouse Native Analytics Solution, especially on the B2B side, where all that data resides. Down the road, we want to tie user engagement data from the digital experience with a lot of our system data and put that integrated data into Optimizely Analytics for analysis.”

• Greater flexibility in website delivery. Interviewees appreciated that Optimizely One enabled them to architect their websites for a variety of use cases. With the Optimizely CMS, users could mix and match template elements to fit their goals. The VP of digital experience at the manufacturing organization said that they had improved the speed of releases on changes to experiences from one to two per month to a consistent two per month. Tweaks they applied more regularly included the way images are sized, new commerce features, and adding to site navigation.
  Interviewees also discovered how to effectively augment those experiences using optimization tests and personalization in tandem. The principal of digital growth at a leasing organization said: “For team reporting with experimentation tests and personalization, we always have a controlled state in place to measure the difference in incremental impact of a modified experience. We know which users are not in that modified experience, so we can compare revenue outcomes of the two and then normalize that by math with our analytics team.” This level of control gave marketing teams the confidence to modify experiences as necessary.

Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Total Economic Impact Approach).