Introduction

In this three-part series we discussed the challenges that enterprises face when it comes to adopting AI – especially those organizations that have traditionally not been in the business of software engineering:

1. Identifying the right opportunities (Part I)
2. Product development (Part II)
3. Avoiding common pitfalls

This final part will conclude the series by going over a few common pitfalls of AI product development and how to avoid them. This list is by no means exhaustive and focuses on business and organizational pitfalls¹ rather than those related to AI theory and technology.

Avoiding Common AI Pitfalls

Get a business sponsor

AI projects are by their very nature complex and show value only slowly. In this regard they are more akin to R&D or innovation-type projects that require an initial investment without immediate payback. It’s thus all the more important to secure strong support from the side of the business that is supposed to benefit from (and often funds) the work.

A good proportion of data science teams struggle, because they have entered an antagonistic, one-sided relationship with the business where they are seen as service providers. As a result, these teams loose their ability to identify with the business’ goals and start to look inward, focusing on technical challenges instead of driving business value.

Getting a business sponsors who is invested into the team’s success can help mitigate these risks. A sponsor ensures that the business relationship is collaborative and that goals are shared across teams. Developing an effective relationship with business stakeholders can be difficult² and data science teams can benefit from someone who can liaise between them and the stakeholders. Finally, a sponsor can also help support project funding discussions by communicating timelines and risks to decision makers.

Start simple

Once a business sponsor has been found it is time to define the scope of the project. Data scientists and ML engineers are an especially ambitious bunch – which is great – but often associated with a tendency to overengineer and overachieve. Especially at the start of a project it is expedient to scale back the aspirations of the team and focus on what value can be delivered quickly with a reasonable amount of effort.

This initial, imperfect solution can then be used as a springboard to get the buy-in to develop a more sophisticated product. The added benefit is that user feedback can be collected. The question of whether an AI product provides value to the user or customer carries the largest risk and can often be answered without taking the most sophisticated AI approach.

It takes discipline and experience to identify the business problems that have the right complexity and don’t require unreasonable time and effort before value is measurable. But a good guideline is to halve the complexity of the AI solution that was initially proposed. In many enterprises ample value can be delivered even after radically scaling back the scope, especially when this is the first attempt of rolling out AI or machine learning at scale for a particular business unit.

Start with augmentation over automation

Augmentation and human-in-the-loop solutions are a good example of simplifying AI product design during the initial roll-out. Although full automation might be the end goal, a transitional augmentation phase has many advantages.

Augmentation can be an effective way to incrementally work towards automation. It often takes time to narrow down the exact problem to solve with AI. Individuals who used to complete a task manually can help capture all the implicit assumptions and business rules when they are part of the augmentation phase.

Augmentation can also speed up time-to-market. An AI application can already start generating value by augmenting the human user on the easiest-to-automate tasks. The experience can demonstrate whether it is worth automating the entire workflow.

Sometimes augmentation can free up enough time³ for humans to focus on tasks where we’re still superior to machines. In the end, this approach can find the sweet spot between augmentation and automation, which would have been overlooked by attempting full automation from the get-go.

Of course, augmentation can also help minimize risk where an error may compromise human safety or increase financial risk. The benefit of fully automating a task may not outweigh the cost of the AI making occasional mistakes.

In summary, aiming for augmentation first can help de-risk AI projects, especially in enterprises with low AI maturity.

Don’t get distracted by data

This final AI pitfall refers the lack of data access or availability that hobbles so many projects in the beginning. This is especially frustrating for data scientists who joined the project to work on machine learning models and want to get started as quickly as possible.

While the reasons for a lack of data can be manifold, they often shift focus away from the business problem. Instead, time is spent deliberating with data management, data governance, or IT teams on how to gain access to already existing data. Or plans have to be made to collect or buy new data.

This is not to say that data isn’t important, but time spent on data availability is time not spent on working toward a great user experience. Data, model, and user experience are interdependent and cannot be solved independently. Spending time on data alone risks acquiring the wrong data or creating data infrastructure that is incompatible with the use case. Data lakes, which enterprises spend years filling, are a good example. AI product development is sometimes delayed by these projects and often requires significant adjustments to data and data infrastructure once started.

It should not be forgotten that building models and exposing them to the user can help inform the data acquisition process. Many of the questions around volume, state, and predictiveness of the data can only be answered by starting the modelling process. The question of whether the model’s insights are needed, useful, and actionable can only be answered once the user is exposed to them.

Conclusion

Not every team or project will encounter the same AI product development pitfalls. Many problems can be avoided with experience, foresight, and adherence to best practices. But these pitfalls also stem in part from patterns that bias teams to behave in certain ways.

Teams might be reluctant to interact with the business to get a sponsor, because these are usually people from a different background, who speak a different language. Teams might be ambitious, thinking they can or should solve the challenge by themselves. It’s easy to get enamoured with new, shiny frameworks and start designing solutions that are too complex and built on technology that people have little experience with.

So the journey to embrace best practices is also always a fight against the forces that pull us away from adopting them.

Read about “how to identify AI opportunities” in Part I and “AI product development” in Part II of this series.