AI Projects: Where They Fail and the Questions Software Vendors Don’t Want You to Ask

“I’m not being difficult, but I only have one question before I drop off. Every time we hear about new software or new technology, especially when discussing AI projects and AI implementation, there’s two things that always happen.

1. The vendor always says it’s easy to integrate, an easy transition, easy to learn.
2. The second thing is: that is never true!

So as far as setting expectations, what’s the reality here? Where do these types of AI projects go wrong, and why do AI projects fail?”

I’ve heard this twice in the last 7 days from a CFO and a VP of Finance in totally different industries when evaluating Kefron AP, and I’ve heard it more in the last 18 months than my entire career before that, particularly as AI adoption in business continues to grow, so I thought I’d share my views on this question in general.

The Role of the Software Vendor in AI Adoption in Business

AI is everywhere right now but the stark reality is 1 in every 3 AI projects fail to reach meaningful business outcomes or are scrapped before value is realized, highlighting ongoing AI implementation challenges. Scary, eh?

The inflection point of AI becoming mainstream really began in November 2022 with the public release of ChatGPT. From 2023 onwards, AI became the dominant narrative in the media and for software development companies, accelerating AI adoption in business.

There is no doubt that most software companies have incorporated AI into everything they do, including their products for customers and their AI implementation strategies but there is plenty out there that are using “smoke and mirrors” to simulate AI in their demonstrations.

Every software vendor promises the same thing:

“AI will automate the majority, so you focus on the minority, deliver instant insights, and transform your function in < 3 months.”

The fact that 1 in 3 AI projects fail to deliver results isn’t because the technology doesn’t work, but because the hard parts of AI implementation are often overlooked and software vendors aren’t doing enough to support new customers with the transition and long-term AI adoption.

Similarly, software buyers aren’t asking the right questions when evaluating AI projects and AI implementation.

Why AI Projects Fail: 3 Key AI Implementation Challenges

Here are 3 areas where AI technology projects commonly fail and questions you should be asking before you put pen to paper when considering AI adoption in business.

1. Misalignment on expectations in AI implementation

Slight dig at my fellow sales professionals here, but stick with me!

When a business is in a technology sales process, the number of interactions they have with the vendor obviously depends on the potential investment in AI projects and AI implementation.

Here’s a snapshot of the typical number of interactions a business has before making a commitment based on the Annual Contract Value (ACV).

 

Typical Interactions When Implementing AI

In every interaction, expectations are set, and key functionality is discussed in relation to AI implementation and project delivery.

The problem? Most technology sales/pre-sales professionals are running multiple engagements at once, so they are dealing with hundreds of interactions with different prospective customers in any given month.

That’s a lot of expectations, functionality, and custom requirements. Unfortunately, over a 3-6 month sales cycle, things have the potential to be missed, misinterpreted or misunderstood, which is one of the key reasons why AI projects fail.

What buyers may not appreciate is when a technology project stalls or gets abandoned because of a misalignment of expectations, a terrible outcome for a sales professional. In most tech businesses, commission isn’t paid until after a client has successfully gone live, or it’s clawed back if a project is abandoned. What’s more detrimental to a technology sales professional is their loss of credibility within their own organisation. Their product, operations, implementation and development teams lose faith in their ability, and that’s usually a difficult place to come back from.

Question to ask your vendor:

Can you create a solution design document that outlines and summarises all of the key functionality, custom requirements, mutual decisions and any known risks to the project before we make a commitment?

To be clear; this isn’t a proposal. It’s a clear and specific document outlining all of the engagements and key decisions mutually made throughout the process. It helps keep both parties accountable to each other and avoids unnecessary delays or finger pointing in the project delivery phase.

With the advancement of AI and call recording technology, there is no excuse not to produce this.

2. Integration – The Biggest AI Implementation Challenge

Integration is often where AI projects meet reality. Most AI systems work extremely well in isolation, but connecting them reliably into ERP systems, CRM’s, Point of Sales and legacy infrastructure is where complexity and risk bear its annoying little head, making AI implementation more difficult and slowing AI adoption in business.

It’s extremely rare to find a system that does not support an integration process via API or file sharing of another platform or system. Unfortunately, the complexity of the integration process is either underestimated or mis-represented.

Here’s where problems crop up:

• Legacy Systems aren’t designed to integrate easily, creating AI implementation challenges

• Data quality is poor. The integration part of your AI project will expose this

• Ownership of the integration process. It’s rare to be able to assign one — you’ve got IT Teams, Software Vendors, ERP Vendors, Internal Process Owners and External System Integrators or Partners that need to be aligned

Here’s the table to fill out with your software vendor and stakeholders to assign ownership:

This is just a template and you may not require all of these owners depending on the integration method and approach.

Is everyone aligned on the below?

3. Monitoring Business Impact & ROI in AI Adoption

Let’s say you’ve nailed alignment on expectations and integration (thanks to this article 😉).

You’re not out of the woods yet. Even after go-live, businesses are consistently abandoning AI projects because they’re not realising the value of their investment, which directly impacts AI adoption in business.

You should start with super clear ROI objectives. Here’s some examples:

• In < 6 months, we want 80% of this process to be automated, saving 32 hours per month.

• In < 6 months, we want to re-deploy one FTE to focus on {INSERT VALUE ADDED WORK}.*

• In 18 months, we want to execute on this acquisition without hiring an additional FTE.

*If you can’t articulate what you would do with additional time given back to your organization; it’s unlikely your AI project will be approved. If your answer is “focus on valued added work” with no specifics, you’re in trouble!

These objectives should be front and centre and shared explicitly with your software vendor to keep them accountable post go-live.

Question to ask your vendor:
These are our objectives for this initiative. How can you show me progress towards these over the next 3-6-12-18 months? If you ask this question and you’re met with silence or a long-winded answer – red flag!

Conclusion

There’s a few other areas to think about in order to de-risk your investment in an AI project and improve AI adoption in business that I’ll cover off in another article soon, specifically related to the importance of User Acceptance Testing, User Adoption & Training Materials but I’m conscious that our attention spans are shorter and shorter and 5 pages of text is enough for now.

_____________________

Author: James Kearns 

With a background in Management Consultancy, James takes a process driven approach to understand his client’s requirements with a view to maximising automation, to give as much time back as possible to busy finance departments. James has been involved in over 100 Kefron Accounts Payable automation projects and regularly shares industry expertise and advice on successful software implementations online –  from building business cases to stakeholder management and everything in between.