The Total Economic Impact™ Of Mastercard Test & Learn

Reduced Inventory And Operational Costs Through Data-Driven Assortment Optimization

Evidence and data. Across interviews, customers described using controlled tests to simplify what they carry and operate — removing items that do not earn their place, validating that changes do not create unintended sales declines, and reducing downstream complexity. In restaurants, this often took the form of menu optimization (testing whether removing items creates negative impact). In retail, it manifested in assortment and inventory decisions at the SKU/brand level, including quantifying inventory reduction and avoiding costly missteps at scale.

• The interviewee at the regional quick-service restaurant chain described running a menu optimization test to remove low-selling items that were creating waste, ultimately eliminating those items after confirming there was no negative sales impact. The business intelligence manager noted: “We tested removing a few low-performing items and saw flat-to-positive sales, so we pulled them across stores. That helped eliminate items we were carrying and wasting.”

• From the national quick-service restaurant chain, the interviewee discussed using controlled store selection and measurement to assess menu changes and evaluate trade-offs across items before deciding on broader rollout. The customer insights consultant shared, “When we test new menu items, we can measure uplift and trading from other items, then decide whether to roll them out.”

• In the B2C specialty retail context, the interviewee highlighted how controlled testing informed decisions balancing operational changes against shrink risk. The director of automation explained, “Testing showed that removing security cases did not generate sufficient sales to offset increased theft risk, so we reversed the change and retained the controls.”

Modeling and assumptions. This benefit quantifies cost savings from removing low-selling menu items, thus reducing waste and inventory carrying costs across the quick-service restaurant network. Based on the interviews, Forrester assumes the following about the composite organization:

• The composite organization carries an average of 75 items on the menu before Test & Learn.

• Test & Learn informs assortment optimization opportunities, resulting in the removal of five low-selling menu items — 6% of all items — from the menu.

• The average annual inventory carrying and waste cost reduction per menu item is $400.

• The number of quick-service restaurant locations is 1,100. Of these, 35% benefit from Test & Learn results in Year 1, while adoption across locations increases to 40% in Year 2 and 45% in Year 3.

Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:

• The breadth and complexity of a restaurant’s or retailer’s assortment. Organizations with larger item or SKU counts and more complex merchandising environments may realize greater inventory reduction benefits, while those with more limited assortments may see more modest impact.

• The extent to which merchandising and assortment decisions are tested before rollout. Organizations that consistently validate product, pricing, and assortment changes through controlled testing will achieve stronger inventory optimization and loss avoidance than those with more limited testing adoption.

Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of nearly $2.0 million.

Incremental Profit Lift Through Optimized Pricing, Promotions, And Assortment Decisions

Evidence and data. Interviewees said they used controlled tests to quantify the incremental impact of pricing, promotions, and assortment changes and to scale initiatives that met defined thresholds — validating pricing and promotion changes, quantifying incremental impact, and scaling only when results meet business thresholds. In restaurants, this often meant protecting transactions during price changes and using break-even logic for new initiatives. In retail, it included selectively rolling out initiatives where the economics work and avoiding broader exposure where they do not.

• The business intelligence manager at the regional quick-service restaurant highlighted the use of holdout testing to validate pricing changes and measure transaction impact prior to rollout. They explained: “Every time we do a price increase, we analyze it by holding back a subset of stores. … [We] go forward with 80% to 90% of your stores but hold out 10% … and see what the impact is to transactions."

• The same interviewee described using controlled testing to evaluate changes to the in-store experience, resulting in a measurable 2% lift in sales from a menu board redesign. The business intelligence manager said, “We’ve seen about a 2% lift in sales from a menu board total redesign that has been running for 90 days.”

• At the national quick-service restaurant chain, the interviewee described structuring tests around defined profitability thresholds and relying on test/control selection to confirm lift before scaling. The customer insights consultant explained, “We did our own financial analysis hypothesizing that we need to see this much trip lift with this much check erosion to break even, and then we use Test & Learn to select the test stores and the control stores … to read the right significance level.”

• The merchandising data lead at the B2B and B2C specialty retailer emphasized how results informed selective rollout, allowing the organization to expand initiatives only where performance was proven while limiting exposure elsewhere. They stated, “We rolled it out in 30% of the stores and didn’t have the loss from the other 70% where it was not a good situation.”

Modeling and assumptions. This benefit measures additional operating profit generated when Test & Learn is used to inform and validate pricing changes. Based on the interviews, Forrester assumes the following about the composite organization:

• The composite organization’s annual revenue is $2.5 billion, of which 25% is impacted by using Test & Learn in Year 1, increasing to 35% in Year 2 and 45% in Year 3 and reflecting broader application of Test & Learn across pricing, promotions, and assortment decisions and growing cross-functional usage over time.

• Revenue lift through optimized pricing, promotions, and assortment decisions attributable to Test & Learn is 0.6% in Year 1, increasing to 0.7% in Year 2 and 0.8% in Year 3, reflecting improved decision effectiveness over time as the composite organization refines test design, targeting, and rollout strategies through accumulated learning and repeated experimentation.

• The composite’s operating margin is 12%.

Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:

• The maturity and consistency of an organization’s testing program. Organizations that run frequent, well-executed tests and scale successful initiatives are more likely to achieve measurable lift.

• Variability of outcomes across initiatives. Not all tests produce positive results, so overall impact depends on scaling winners and discontinuing underperformers.

• The share of revenue subject to testing. Organizations that apply Test & Learn across a larger portion of pricing, promotions, and operations will see greater impact than those with narrower usage.

Results. To account for these risks, Forrester adjusted this benefit downward by 5%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $1.7 million.

Improved Marketing Effectiveness Through Incremental Impact-Driven Campaign And Spend Optimization

Evidence and data. Interviewees said they used controlled tests to determine which campaigns drove incremental impact and to decide which approaches to repeat or scale. For restaurants, this included measuring lift from customer-facing changes (e.g., menu board redesigns) and iterating offer structures. Retail interviewees reinforced that the same discipline applies across promotional channels, enabling faster learning and more confident investment decisions.

• The customer insights consultant at the national quick-service restaurant chain described running multiple offer variants across different customer segments to determine which structure drives stronger incremental response. They explained: “Even within one week, we might run multiple versions of the same offer — different audience cuts, different thresholds. Then we can analyze them to see which one performed better and use that targeting going forward.”

• The interviewee at the B2B and B2Cspecialty retailer pointed to how controlled testing extends beyond merchandising into promotional and marketing channels. The merchandising data lead noted, “We also do testing on promotions and marketing, whether it be on the web, in-store, or even radio.”

• The director of automation at the B2C specialty retailer outlined how testing informed a decision to avoid scaling a marketing investment when incremental sales did not materialize. The interviewee said, “We didn’t see any sales increase materialize, and it would have cost in the eight digits on an annualized basis with no real upside, so we did not move forward.”

Modeling and assumptions. This benefit captures direct cost savings from optimizing a portion of the marketing budget through controlled testing of offers, promotions, and campaigns. Based on the interviews, Forrester assumes the following about the composite organization:

• The composite’s annual marketing budget is 4.5% of total annual revenue, of which 25% can be optimized with Test & Learn.

• In the first year of implementation, marketing spending efficiency improves by 3%. This impact grows to 3.5% in Year 2 and 4.0% in Year 3.

Risks. Forrester recognizes that these results may not be representative of all experiences. The following factors may impact this benefit:

• The size and maturity of the organization’s marketing investment. Organizations with larger, more sophisticated marketing programs will see greater optimization opportunities, while those with smaller budgets may realize more limited gains.

• The extent to which marketing activity is tested and optimized. Organizations that consistently test campaigns, offers, and targeting strategies are more likely to achieve efficiency gains than those with limited or sporadic testing.

• The share of marketing spend that is addressable through testing. Benefits depend on how much of the marketing budget can be influenced by Test & Learn, with broader application driving greater impact.

Results. To account for these risks, Forrester adjusted this benefit downward by 10%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $2.2 million.

Unquantified Benefits

Interviewees mentioned the following additional benefits that their organizations experienced but were not able to quantify:

• Greater trust in results and decisions. According to interviewees, Test & Learn increased decision confidence by making results more objective and easier to socialize across stakeholders, reducing the time spent debating whose numbers are right and increasing willingness to act on outcomes. The business intelligence manager at the regional quick-service restaurant explained that long-tenured stakeholders often defaulted to gut feelings, and that external credibility helped results land with less skepticism. They said, “Just seeing the Mastercard logo lends itself to being a little more credible — having a third-party platform and team to give an unbiased view of whatever it is we’re testing.”

• Stronger experimentation culture and operating discipline. Interviewees described a shift toward a more disciplined experimentation cadence — designing tests upfront, selecting stores/controls more rigorously, and using results to guide rollouts — rather than making changes and trying to explain outcomes after the fact. The interviewee at the B2B and B2C specialty retailer described how the organization adopted a more consistent cadence for collaborating on test design and using results to inform decisions. The merchandising data lead noted: “I think really it’s the culture that this mindset brings with it, that we’re not just going to make a decision; we’re going to collaborate and run a test and get numbers. It’s taught us to look for those high-risk, high-reward things.”

• Reusable learning and organizational memory. Beyond the immediate outcome of a single test, interviewees highlighted the value of accumulating learnings that could inform future decisions — helping teams avoid repeating mistakes and improving how they frame questions over time. The business intelligence manager at the regional quick-service restaurant described the value of retaining prior decision analysis and using it to influence future decisions. They explained: “Having a well-designed test gives you well-designed analysis that you can use to influence future decisions. Having a catalog of past decisions and analysis has been a big dividend.”

• Embedded expert support and thought partnership. Interviewees repeatedly emphasized that value comes not only from the platform but also from Mastercard’s ongoing support model — standing calls, office hours, coaching, and practical guidance to improve test design and execution. The interviewee at the national quick-service restaurant chain described Mastercard’s support as materially extending capacity for a lean team. The customer insights consultant pointed out: “The Mastercard team does a lot of work for us. They save us a lot of time; it’s almost like having extra team members.”

Flexibility

The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement Test & Learn and later realize additional uses and business opportunities, including:

• Extending experimentation from store/site testing into customer-level testing — more targeted offers, loyalty, and segment-specific decisioning. Interviewees mentioned the future opportunity of expanding beyond store-level experiments to run controlled tests at the customer level, enabling more targeted promotions, loyalty initiatives, and segment-specific measurement that is not possible with store-only designs.
  The interviewee at the fast casual restaurant chain explained that the same testing approach used for sites can be applied at the customer level via a dedicated module called Test & Learn for Customers. The interviewee at the B2B and B2C specialty retailer described customer-level testing as a “natural next step” and explained how it would enable targeted offers and promotions by customer segment (e.g., loyalty customers and installer accounts). The merchandising data lead stated: “We talked about Test & Learn for Customers. Those are things that would be the natural next steps. If we had Test & Learn for Customers, we could take their account data and run promos, ads, and offers targeted to a test group versus a control group of customers.”

• Adding complementary modules (e.g., Market Basket Analyzer/Menu Analyzer) to expand the breadth of insights and decision support. Interviewees also described flexibility in expanding the solution footprint by adopting additional modules that broaden analytical capabilities beyond their current baseline deployment — often through trials before committing to ongoing use. The director of automation at the B2C specialty retailer explained that their organization is evaluating additional capability through an active trial, with intent to decide whether to add it to the contract. They said, “We are currently on a Market Basket Analyzer trial, and we’re running a trial for it this year to see if that’s something we want to add on to the contract going forward.”

Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Total Economic Impact Approach).