AI Operational Efficiency: Navigating GenAI’s True Cost

Working toward AI operational efficiency is becoming critical as more organizations implement AI—so how do you measure AI efficiency? While experts predict that AI will add more than $15 trillion in economic value to the world in the next decade, over 50% of custom AI initiatives launched this year will be abandoned by 2028 due to cost, complexity, and technical difficulties.

‍

As artificial intelligence (AI) becomes implemented at an increasing clip, the question of reducing AI costs grows. What is the best strategy for AI operational efficiency? 

‍

Last year was a breakout year for AI. ChatGPT brought AI to the mainstream, while enterprise leaders have been looking to advancements in AI to help their organizations boost efficiency, reduce costs, and become more competitive in a global economy that has been struggling since the COVID-19 pandemic.

‍

The growing adoption of enterprise AI is causing an explosion in value in the tech sub-sector. McKinsey recently predicted that the “value at stake from AI can reach $15 trillion” over the next decade. Generative AI is expected to provide nearly a third of that value.

‍

In response to this growing recognition, companies have increased the adoption rate of AI technology. The McKinsey Global Survey found that one-third of respondents “say their organizations are using gen AI regularly in at least one business function.” In contrast, “40 percent of respondents say their organizations will increase their investment in AI overall because of advances in gen AI.” 

‍

Gartner also found that one-third of technology companies plan to invest $1 million or more in AI and machine learning (ML) initiatives in the next two years. 

‍

Source - As AI becomes more common across industries, finding opportunities for AI operational efficiency becomes critical.

‍

Errol Rasit, Managing Vice President at Gartner, believes that

“Technology organizations are increasing investments in AI as they recognize its potential to not only assess critical data and improve business efficiency but also create new products and services, expand their customer base and generate new revenue.”

That same research found that companies are allocating more money to AI than any other emerging technology except cloud computing. The highest investment areas are AI-based computer vision, composite applications, and data science and analytics programs. 

‍

Enterprises are scrambling to take advantage of emerging AI technology while seeking increased AI operational efficiency. Still, adoption is hampered because of the wide range of technologies covered under the umbrella of AI and its myriad potential use cases. 

‍

Anand Rao, Global AI Lead and US Innovation Lead for the Emerging Technology Group at PwC says,

“Most people probably think they know what AI is and does, but it’s a term encompassing many technologies, processes, and functions, so it’s difficult to pin down. It’s definitely not a one-size-fits-all field. That can make it challenging to determine a return on investment.” 

‍

In addition, many leaders still need guidance on implementing AI into their organizations while effectively maintaining regulatory compliance, data privacy, IP protection, and systems reliability. Finding opportunities for AI operational efficiency is also a major hurdle, given the high price of custom AI development and off-the-shelf solutions. 

‍

‍‍

Companies are also looking to unlock AI data insights and AI operational efficiency, and increase competitiveness in a tight economy with a looming recession. Executives, in particular, are looking to AI as a game-changing tool they expect “to cause significant or disruptive change in the nature of their industry’s competition in the next three years.”

‍

The Price of Progress: AI’s Unique Expenses

‍

Tech firms specializing in AI are pursuing AI operational efficiency as they seek to commercialize their product and generate a profit. Generative AI platforms, like ChatGPT, are among the most costly to operate.

‍

Independent researchers estimated that the average cost of a single ChatGPT query is 36 cents, and the company spends an average of $700,000 per day to keep the large language model (LLM) running. 

‍

Alphabet, Google’s parent company, echoed the need for more AI operational efficiency when Chairman John Hennessy told Reuters that AI LLMs “likely cost 10 times more than a standard keyword search.” 

‍

Morgan Stanley found that popular search engines could be burdened with substantial cost increases if they transition to AI responses rather than traditional search. It’s estimated that Google may pay more than $6 billion in expenses in 2024 if half of its queries use the new AI function. 

‍

The high costs associated with all forms of AI, not just Generative AI and LLMs, are unique and have little precedent. CNBC writes, "The high cost of training and "inference"—actually running—large language models is a structural cost that differs from previous computing booms. Even when the software is built or trained, it still requires a huge amount of computing power to run large language models because they do billions of calculations every time they return a response to a prompt."

‍

Today, AI firms backed by venture capital and Big Tech money are absorbing most of these costs as they prioritize increasing market share and reaching new consumers. However, over the long term, these companies will pass the extraordinary expenses associated with AI onto consumers as they prioritize profitability and meet investor demands once market saturation is reached. 

‍

The Pricey Path of AI Implementation

‍

Generative AI and LLM services, like OpenAI, are still incurring operational costs and posting losses during this phase. On the other hand, other enterprise AI platforms, like productivity assistants, data science platforms, and production floor analysis tools, come with a hefty price tag that puts AI operational efficiency front of mind. For example, McKinsey found that deploying an off-the-shelf AI model, such as those commonly used in data science and supply chain management, costs $2 million.  

‍

That's a modest price when there's a clear use case and easily definable return on investment (ROI). However, many enterprises are attempting to develop custom-built in-house AI platforms to meet their needs. These bespoke solutions come with a prohibitively high price tag. McKinsey recently estimated that it costs $10 million to customize an existing model or up to $200 million to develop an AI model from the ground up.

‍

Given the high costs associated with AI, companies must carefully assess the potential benefits and expected ROI before committing to a significant AI transformation—intentional implementation can yield AI operational efficiency in the long run. 

‍

AI Operational Efficiency Means Improving ROI

‍

While companies are scrambling to join the AI movement, researchers worry that the race to integrate AI into enterprise operations may not bear fruit. Gartner predicts that "by 2028, more than 50% of enterprises that have built large AI models from scratch will abandon their efforts due to costs, complexity, and technical debt in their deployments." 

‍

IBM echoes this finding: "Few AI projects deliver the financial value shareholders expect. In fact, average ROI on enterprise-wide initiatives is just 5.9%—well below the typical 10% cost of capital."

‍

Two of the main reasons why AI projects are likely to be shelved are a failure to achieve forecasted results and the difficulty in assessing AI's ROI.

‍

PWC recently dove into this problem, writing, "While our most recent AI survey found that businesses are beginning to reap AI benefits, the reality is they're not often seeing a financial return—or worse, not even covering their investments. Compounding the challenge is the fact that many organizations struggle to define ROI for AI in the first place." 

‍

Despite the challenges associated with determining ROI, businesses can improve their odds of success, not to mention AI operational efficiency, in several ways. Gartner identifies three AI use cases with their timeline and expected ROI. 

‍

Gartner identifies three use cases with their ROI metrics: 

‍

Quick Wins: Businesses can score quick wins by implementing AI productivity assistants like Microsoft Co-Pilot, Google Workspace, ChatGPT, etc. These programs are cheap and easy to use and will produce rapid productivity increases, demonstrating immediate ROI.

‍

‍Differentiating Use Cases: These projects require a one- to two-year investment and are more robust implementations of AI, such as predictive maintenance, supply chain automation, and AI-based customer service departments. These ambitious projects allow organizations to leverage data to streamline processes, generate revenue, and increase competitiveness. 

‍

Transformational Initiatives: Large-scale, transformational projects are high-cost, high-risk initiatives that should only be pursued once your organization has demonstrated success with the two previous use cases. Companies can pursue transformation initiatives, like efforts to de-silo data and create organizational-wide data lakes, data warehouses, and analytics platforms, to generate actionable insights and a truly data-driven organization once they’ve mastered the basics.

‍

Enterprise leaders pursuing AI cost savings should start with quick wins and differentiating use cases to demonstrate value before approaching the C-suite and board with transformational initiatives. By showcasing the ROI of small-scale projects, securing large budgets and stakeholder support for more extensive AI programs becomes easier. 

‍

The True Value of AI ROI

‍

It's also important to carefully select the metrics used to determine ROI. With a clear list of desired benefits, you can accurately assess the results of your AI initiative and associated AI operational efficiency. 

‍

"Most people probably think they know what AI is and does, but it's a term that encompasses many technologies, processes and functions, so it's difficult to pin down," writes Anand Rao, Global AI lead at PWC. "It's definitely not a one-size-fits all field. That can make it challenging to determine a return on investment." 

‍

He also says, "Be sure your ROI calculation accounts for both the time value of the money invested and the uncertainty of the benefits."

Rao recommends looking at the following areas when calculating the ROI of your AI investments and resulting AI operational efficiency:

• Time savings
• Productivity increase
• Cost savings
• Revenue increase
• Better Experience
• Skills retention
• Agility

‍

Finally, while organizations must start small with AI, the most impressive benefits come from sustained investments and intelligent scaling. Accenture found that "companies in our study that are strategically scaling AI report nearly 3X the return from AI investments compared to companies pursuing siloed proof of concepts." 

‍

The most successful AI adopters use the following tried-and-true strategy to launch AI initiatives, assess progress, intelligently scale successful use cases, and realize AI cost savings in the process:

‍

1. Identify a Use Case and Solvable Problem
2. Conduct an AI Feasibility Analysis
3. Collect and Prepare Data
4. Deploy the AI Platform
5. Monitor and Assess the ROI

‍

This approach will help enterprises spend money wisely and clearly articulate the benefits of these programs to senior leadership and stakeholders. McKinsey also recommends using four technical enablers to scale your next AI initiative: "incorporating data products such as feature stores, using code assets, implementing standards and protocols, and harnessing the technology capabilities of machine learning operations (MLOps)."

‍