Improving Educational Outcomes With Microsoft Azure OpenAI Service

Within the education sector, AI improves internal service delivery across institutions by streamlining administration and empowering educators with best-in-class content. Putting AI in the hands of students improves communication and engagement and can result in better education outcomes, such as higher graduation and employment rates. 

Microsoft commissioned Forrester Consulting to interview 20 representatives at 16 organizations and conduct a Total Economic Impact™ (TEI) study to better understand the benefits, costs, and risks associated with the investment in Microsoft Azure OpenAI Service.1 In the full TEI study, Forrester found that an industry-agnostic composite organization would see risk-adjusted financial benefits ranging between $45.9 million and $197.4 million over three years.

This abstract will focus on the education sector’s use of Azure OpenAI Service and its value to their organizations. As part of the broader TEI study, Forrester interviewed representatives from two education organizations:

  • A chief information officer (CIO) at a higher education institution in the Americas with 48,000 students and 1,700 employees.
  • The head of AI at a department of education organization in the APAC region with 850,000 students and 100,000 employees.

Projected efficiency gain in content generation by Year 3*

10% to 60%

*For the composite organization presented in the full TEI study

Prior to using Azure OpenAI Service, the interviewees noted their education organizations relied on teachers and staff to manually create and manage materials and plans as well as to provide administrative support. Any digital support offered to students was general and, therefore, not tailored to the student or their needs. The interviewees’ organizations wanted to invest in technology to make education and the related services more accessible to students and more attractive to educators, driving improved educational outcomes and teacher satisfaction.

Azure OpenAI Service enables organizations to develop custom AI applications that use Microsoft’s underlying AI infrastructure. This strategy enables AI to impact as many applications, business processes, and employees as possible while still being accessible and manageable for AI developers. The interviewees’ education organizations used Azure OpenAI Service to build AI solutions to meet their specific needs in the following ways:

  • The CIO at a higher education institution indicated that they were able to roll copilot out more broadly to all employees by running it on Azure OpenAI Service.
  • The same CIO also described two ongoing custom projects to better support students. One project focused on building a 3D avatar that students engaged with to practice their soft skills and communication to prepare for interviews and the like. The other project was an AI tutor powered by Azure OpenAI Service that assisted students in studying various subjects.
  • The head of AI at a department of education organization built a student and faculty support hub using an internal chatbot to handle queries from students and teachers alike.
  • The same interviewee also reported using Azure OpenAI Service solutions to generate draft lesson plans for teachers, which the teachers then reviewed and adjusted as needed.

Projected improvement in chatbot resolution rate by Year 3*

10% to 50%

*For the composite organization presented in the full TEI study

INVESTMENT DRIVERS

Prior to adopting Azure OpenAI Service, the interviewees faced various challenges in their legacy environments, including:

  • Teacher time spent on non-student-facing activities. Interviewees from education organizations cited large volumes of manual administrative work and extended bureaucratic processes. Prior to the investment in Microsoft Azure OpenAI Service, educators were largely responsible for this nonteaching work. The head of AI at a department of education organization described the impact to the teaching profession in their region, stating, “One of the reasons the teaching profession is not really attractive is because of the amount of nonteaching (administrative and bureaucratic) work they have to do.” Additionally, the head of AI indicated that manual processes and extended content delivery timelines did nothing to motivate educators to update their materials or curriculum on a consistent basis, resulting in stale content.

“Making lesson plans from scratch takes twice the amount of time it takes to teach it.”

Head of AI, global department of education

  • Inaccessibility of services and information. Interviewees in the education industry described complex environments of students with different needs and varying levels of comprehension. Servicing these students with a one-size-fits-all approach was inadvertently restricting access to certain services and information. To illustrate the challenge, the CIO from a higher education institution said: “Fifty percent of our students come from outside of the country. English is not their first language. They can be shy in the classroom.”
  • Restricted technology. Security and compliance concerns regarding data access and usage restricted certain technologies in the education space. In some regions, publicly available AI tools were banned in public schools but permitted in private schools. As a result, some students reaped the advantages of these tools, while others did not, leaving them less equipped to use and understand these tools in their future careers. To level the playing field, many education institutions hoped to provide access to private versions of these AI tools, allowing students to use educational generative AI (genAI) tools.

The interviewees’ organizations adopted Azure OpenAI Service to provide better experiences for students and teachers alike by:

  • Leveling the playing field in private versus public school experiences for students.
  • Providing STEM-focused opportunities and access to relevant technologies to prepare students for future careers.
  • Educating students on best practices for safely and responsibly using generative AI tools.
  • Relieving the manual administrative work required of teachers and redistributing time savings to the classroom.
  • Attracting more educators to the profession.

Key RESULTS

The results of the investment for the interviewees' organizations include:

Expanded service accessibility. Interviewees at education organizations projected that their use of Azure OpenAI Service would prove critical in improving key education outcomes, such as graduation rates and employment rates, attracting both more students and, therefore, more investment.

  • Interviewees reported that they created more interactive courses that helped students prepare for future careers. The CIO at a higher education organization described an initiative with Azure OpenAI Service to build a course that taught consultative selling skills: “With Azure OpenAI Service, we created a course where students go through a sales process and must make a sales pitch to a generative AI [client] and submit it for evaluation. The students could try this multiple times; the first time they took it, they would get scores in the mid-40s, but by the time they did it a few times they would get scores in the 90s. The students who have gone through this say that it’s an unbelievable experience.”

“We use an Azure OpenAI Service tool to allow students to do mock interviews for different employers and different personality types. After, the tool tells you how you did, what went well, and what you need to improve.”

CIO, higher education

  • For interviewees in the education sector, the most important success metrics to track were improvements to grades, graduation rates, and employment rates. The CIO at a higher education organization noted that they plan to use Azure OpenAI Service to improve these outcomes: “We will want to see if there is an increase in the students’ job rate and graduation rates. We have struggled to increase our graduation rate for many years. This is the biggest initiative we are doing to improve it.”

The head of AI at a department of education organization described similar drivers behind establishing an AI tutor for their students, which had the potential to reduce socioeconomic gaps in educational outcomes: “Between 25% to 30% of our students come from lower income families with parents who didn’t finish high school. When they get home with homework, their parents may not be there to help them. The tool serves as a tutor and the goal is to measure improvements in educational outcomes.”

“If students have a quiz tomorrow, they can use the AI tool to generate mock quizzes to prepare them. The tools are based on internal lesson data, so it cannot hallucinate.”

CIO, higher education

Increased operational efficiencies. Azure OpenAI Service expedited internal processes and automated certain tasks to provide operational efficiencies in a few different areas, including customer support and content creation.

  • Interviewees stated that with Azure OpenAI Service, they generated more educational content, including lesson plans and test preparation materials. In most cases, this meant that educators who previously spent time manually creating content could instead focus on reviewing and adjusting content, leading to considerable time savings. Interviewees noted their organizations could also increase productivity in content creation by asking AI tools to develop short-form, long-form, or derivative content for almost any purpose, all while keeping on message, on brand, and on tone. The ability to tailor content to specific students made personalization at scale achievable.2
    The head of AI at an education department explained: “We used to have educators write their own sample lesson plans. Creating one could take 2 hours for every hour of lesson. By them switching to QA work, the time investment can take less than an hour per hour of lesson plan.”
  • With Azure OpenAI Service, the interviewees’ organizations rolled out AI capabilities to more staff, including school administrators. The head of AI at a department of education organization explained that their school administrators used an internal chatbot to communicate with parents. Additional AI capabilities created emails about payments, provided documentation, and completed other administrative tasks.

Improved service delivery. With Azure OpenAI Service, the interviewees improve both teacher and student experiences. Students gained more access to the educational materials that they needed, whereas teachers spent less time on administrative prep work and more time in the classroom.

  • The interviewees’ organizations used AI to engage with students as well as create tools that supported student activities directly. The CIO noted their higher education institution provided more digital support to students by developing a chatbot and analyzing its interactions with students. By monitoring satisfaction scores and dropout rates, their organization added to its underlying knowledge base and improved the self-service portal.

“Our goal is to have 80% of student queries done through self-service.”

CIO, higher education

  • Azure OpenAI Service expedited content delivery timelines for the interviewees’ organizations. Therefore, educators dedicated more time to content creation catered to individual student needs. The head of AI at a department of education organization described how teachers utilized a bot to synthesize text for comprehension or translate text for students. In this way, teachers created more variations of a single lesson or exercise to meet the needs of more of their students. The head of AI described how teachers were encouraged to update their existing content: “Before, a teacher might be more likely to use a lesson plan from a previous year; now, they might be encouraged to continue updating it because it doesn’t take as much time as it did before.” Better document search capabilities with AI also enabled teachers to find relevant content more easily. Therefore, they were more likely to use internally developed content versus using search engines to copy materials from other institutions or countries that might not meet the same quality standards.

TOTAL ECONOMIC IMPACT ANALYSIS

For more information, download the full study: “The Projected Total Economic ImpactOf Azure OpenAI Service In Reinventing Customer And Constituent Engagement,” a commissioned study conducted by Forrester Consulting on behalf of Microsoft, July 2024.

STUDY FINDINGS

While the value story above is based on two interviews, Forrester interviewed 20 total representatives at 16 organizations with experience using Microsoft Azure OpenAI Service and combined the results into a three-year financial analysis for a composite organization. Projected quantified benefits include:

  • Better engagement with prospective service users driven by 10% to 20% increase in top of funnel prospects and 20% to 40% improvement in conversion rate.
  • Better engagement with current and existing service users driven by 20% to 30% reduction in churn due to better user experience.
  • Content generation time savings of 30% to 60%.
  • Improved deflection rates of 20% to 50% in contact center calls requiring a human support agent.

Disclosures

Readers should be aware of the following:

This study is commissioned by Microsoft and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.

Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the study to determine the appropriateness of an investment in Microsoft Azure OpenAI Service.

Microsoft reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.

Microsoft provided the customer names for the interviews but did not participate in the interviews.

Appendix A: Endnotes

1 Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s
technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders.

2 Source: Generative AI Brings Superpowers To Portfolio Marketers, Forrester Research, Inc., March 5, 2024.

Cookie Preferences

Accept Cookies

A cookie is a small text file that a website saves on your computer or mobile device when you visit the site. It enables the website to remember your actions (data inputs, website navigation), so you don’t have to re-enter data when you come back to the site or browse from one page to another.

Behavioral information collected by our web analytics vendor is used to analyze data pertaining to visitor trends, plan website enhancements, and measure overall website effectiveness. We may also use cookies or web beacons to help us offer you products, programs, or services that may be of interest to you and to deliver relevant advertising. We may use third-party advertising companies to help tailor website content to users or to serve ads on our behalf. These companies may also employ cookies and web beacons to measure advertising effectiveness.

Please accept cookies and the collection of behavioral information to receive full functionality and enhance your experience. If you decline cookies, some features of the website may not function normally.

Please see our Privacy Policy for more information.