70% of AI Projects Fail: You Need an AI Readiness Assessment

What is an AI readiness assessment? First, let’s examine what can happen in its absence.

‍

In 2012, IBM had a bold vision: revolutionize cancer treatment with artificial intelligence. Watson for Oncology was supposed to democratize cancer expertise globally. Instead, it became a $62 million cautionary tale about the perils of rushing into AI implementation without proper assessment and preparation.

‍

By 2021, internal documents revealed the stark reality: Watson was providing "unsafe and incorrect" cancer treatment advice. The system, trained on data from a single institution, couldn't adapt to different healthcare contexts, making it unusable for physicians. The project that promised to transform healthcare ended up being quietly sold off—a victim not of AI's limitations but of poor planning, which an AI readiness assessment might have mitigated. Had an assessment been carried out, the team at Watson might have discovered issues with the data used to train Watson: insufficient in amount, lacking in quality, and heavy-handed in its use of synthetic data. 

‍

This isn't an isolated incident. Across industries, enterprises are rushing to implement AI without first understanding their readiness for such transformation:

‍

• Tutoring company iTutor Group paid a $365,000 settlement in 2023 when its AI hiring software automatically rejected 200 applicants based on age.
•  Air Canada paid damages to a passenger in 2024 after an AI chatbot misrepresented information concerning bereavement discounts; a tribunal held the airline liable for the error.
•  Zillow laid off 2,000 employees due to errors in its predictive machine learning algorithm.

‍

Less than half of companies are seeing positive ROI from AI investments. The rest are learning an expensive lesson about the critical importance of AI readiness assessments the hard way. 2024 data from Bain & Company shows that many organizations feel that in-house expertise, data, and existing tech platforms are holding them back from moving faster with generative AI. The root cause isn't the technology itself—it's a fundamental failure to assess organizational readiness for AI adoption.

‍

‍

Reality Check: AI Ambitions v.s. Organizational Reality

Working with companies hoping to glean the gold AI promises, I’ve noticed a significant gap between ambition and readiness. It’s one that an AI readiness assessment can begin to bridge, but it also highlights worrisome blindspots that need to be confronted. My advice is that it’s better to ask uncomfortable questions and confront misunderstandings now than when it’s too late.

1. Process Blindness

When I consult with organizations hoping to leverage AI, I sometimes ask a difficult two-part question:

‍

How would you improve this process without AI, and why hasn’t that happened yet?

‍

Companies sometimes tend to look at AI as an antinode for processes that are already clunky or lacking in some way. Sometimes, the answer to my question is that AI will only complicate or further ingrain those sub-optimal processes. Consider the ongoing issues with Big Tech’s content moderation—Facebook’s AI flags 97% of content removed from the platform even though it often lacks the necessary cultural context, nuance, and linguistic understanding (it was likely trained primarily on English-language data). “Content moderation has not kept up with the threats because it is not in the financial interest of the tech companies,” says Hany Farid, professor at the University of California, Berkeley, School of Information.

‍

In these cases, I have to wonder: is AI what’s been missing, or are there other absent factors that you need with or without AI? Examples include customer service, document optimization, and on-time delivery. The clear issue in the Facebook example I listed above is the motivation to devote resources to a stronger solution.

‍

In some cases, like creative workflows, adding AI may complicate your processes further. Incredibly, we can create content in seconds, but with creative work, there is always a human-level validation and review cycle. In many cases, the benefit of having a creative output that’s almost there is outweighed by a new validation and review process that you didn’t need before.

‍

I see this in coding, too. One of the criticisms experienced developers have about generative coding is that while a given output may look credible on its face, there may be bugs that are not obvious and require more review time.

‍

This is why the process I’m describing here is so critical. If you’re not careful, you may create processes that may not have existed before, bringing us back to understanding your process, your workflows, and your data. AI can be wildly powerful, but it would never be a panacea. Conduct a readiness assessment, have the difficult conversations with stakeholders, 

2. Different Orgs, Different Data—Different AI Solutions

‍

Navigating the complex landscape of AI requires understanding a crucial distinction between generative AI and machine learning. While both offer powerful capabilities, they serve fundamentally different business needs. It comes down to what data your organization utilizes.

‍

Machine learning applications—which have been around longer than some of the newer generative AI models—rely on good, labeled dat and a structured approach. For example, banking and credit card companies have used machine learning for decades for fraud detection, but generative AI wouldn’t work for transaction errors. You can’t accept potential hallucinations and privacy concerns regarding transaction data, so paying attention to that data would be necessary before implementing any general AI application.

‍

Generative AI sometimes relies on unstructured data, and that’s okay because generative AI was created to imitate unstructured data and perform tasks like summarization without much instruction. Generative AI could work perfectly for a creative company with a lot of media, graphics, and art, for example.

‍

If it’s unclear which type of AI would be best in your specific use case, you’ve understood my point: AI is not a one-size-fits-all situation; you must consult an expert. An AI readiness assessment is the perfect first step in that process.

‍

Conducting Your AI Readiness Assessment

Companies that conduct comprehensive AI readiness assessments before implementation are significantly more likely to succeed in their AI initiatives. Organizations often see AI as a silver bullet, but without proper evaluation and preparation, they add AI to broken processes.

‍

An AI readiness assessment helps organizations start a crucial conversation with stakeholders about their AI journey. While its formatting can vary, the goal of an assessment is to identify blind spots, open up new areas of discussion, and get an initial barometer reading on your company’s AI readiness.

‍

Gigster’s AI readiness assessment is the perfect place to start: it’s a structured evaluation that examines five critical dimensions of your organization:

‍

• Data Readiness: What kind of data does your company utilize, and what AI application makes sense for your goals? In Watson’s case, limited training data from a single institution led to recommendations that often failed to align with local guidelines or real-world cases.
• People and Skills: Is your workforce ready to leverage this technology?
• Technology Infrastructure: How does this AI solution fit into your system? In Meta’s case, the AI’s struggle to consistently identify and remove harmful content indicates potential issues with integration into existing systems.
• Existing AI Initiatives: Does this AI solution play well with other AI at work in your company?
• Strategic Vision: How does AI serve the big picture in your company?

‍

The assessment, including a comprehensive 20-question analysis, clearly shows your organization's preparedness for AI implementation. Ultimately, you get a report that can help align stakeholders and secure buy-in across your organization.

‍

The Hard Part: Creating a Change Management Strategy

Organizations integrating change management based on their AI readiness assessment see 47% higher success rates. Yet, only 43% of employees rate their organizations as good at change management—down from 60% in 2019. Integrating AI into an organization is the most difficult part, so external consultation is advised.

‍

‍

 A decreasing number of employees (down from 60% in 2019 to 43% in 2024) feel that their organization manages change well. Since change management involving AI/ML necessitates more skill to manage more change than expected, organizations have an increased risk of making a major blunder. The need for a strong AI change management strategy is compounded (understandably so) by employee fears about job displacement, which can derail even the most promising initiatives.

‍

A successful AI change management plan must be both comprehensive and flexible. Here are a few critical elements:

‍

1. Start by establishing clear governance structures and decision-making processes for AI initiatives. 
2. Define roles and responsibilities across the organization, from executive sponsors to end users. 
3. Create detailed communication plans that address different stakeholders' needs and concerns. These plans should include regular updates on progress, successes, and lessons learned. 
4. Develop risk mitigation strategies that address both technical and organizational challenges. 
5. Build flexibility to adapt the plan based on feedback and changing circumstances.

‍

Recent studies by Oak Engage have identified three key resistance factors to organizational change:

‍

• Fear of the unknown (38%)
• Lack of awareness about the reasons for change (39%)
• Organizational mistrust (41%)

‍

This is also where an AI readiness assessment becomes invaluable. By providing transparency into the organization's AI capabilities and implementation plans, you can address these concerns head-on, amongst stakeholders and eventually across the entire organization. The assessment creates a shared understanding of challenges and opportunities, helping build trust and buy-in and offset some of the key resistance factors identified above.

‍

Your AI readiness assessment should inform every aspect of your change management strategy, from initial planning through final implementation.

‍

Conclusion: The Path Forward

Success in enterprise AI implementation requires more than choosing the latest technology—it demands a clear-eyed assessment of your organizational processes. Before rushing to implement AI solutions, ask critical questions, especially the uncomfortable ones: Why haven’t the desired changes happened without AI? How will AI affect our workflows? An AI readiness assessment helps surface these fundamental issues, preventing the costly mistake of using AI to patch broken processes or inadvertently creating new complications in previously streamlined workflows.

‍

Perhaps most importantly, understanding your organization's unique data landscape should drive your AI strategy. This isn't a one-size-fits-all journey. Start with an AI readiness assessment to determine your actual status, then partner with experts who can guide your organization toward the right AI solution for your specific needs.

‍