The Most Frustrating Aspects of Generative AI

Here at Rightpoint, we are believers in the transformational power of AI. Generative AI is a fascinating paradigm shift for customer, employee, and product experiences. We've seen many of our customers begin this shift, whether enabling their employees to be more efficient with Generative AI tools or reimagining customer experiences with Generative AI human-like interactions.

While this is truly a once-in-a-generation change in how we all work and serve our various customers, it is important to recognize how wild the last 18+ months have been. Almost every client conversation, statement of work, or project deliverable these days has the question, “Well, what about AI?”

Don’t get us wrong. We’re bullish on Generative AI and the positive impacts it can have and will continue to have on reshaping industries and organizations, but, as with any new innovations, it can also be a source of frustration for many. We’re going to share some of the frustrations we often hear from our clients and our point of view on how to deal with them.

Why Are There So Many Tools That Do the Exact Same Thing?

Picture this…You’ve been tasked with defining how to leverage Generative AI for cost efficiency or revenue enablement. You start researching online, going to conferences, and talking to software vendors. You learn about language models that are out of the box, tools you can customize, and new features within the platforms you already have. You think, “Let's create a plan to design/build, pilot, launch, and scale.” But then, two months later, new tools come out, new models arise, and you start questioning yourself, “Did we make the right decisions? Do I need to start the process over? What do I tell my leadership?”

This is the situation we are all faced with today. You are not alone.

In this case, extreme competition is breeding extreme confusion. There are some platforms that are creating holistic, end-to-end Generative AI solutions for your workforce, while others are focused on niche industry, process, or moment-specific use cases. And right now, the market is messy with a lot of competing platforms – some early winners, some emerging players, and others fading. We are stuck in cycles of change that will make your head spin if you follow too closely.

As Benjamin Franklin once said, “Out of chaos comes order.” The order part is emerging, but hasn’t arrived just yet. For now, pick a language model, define your core use cases and mid-term strategy, and begin rolling out experiences to employees and customers. You’ll learn much more by doing versus trying to keep up with every technological shift in the industry and trying to build the ‘perfect’ strategy.

Quantifying Return on Investment at Scale is Complicated

There are plenty of studies and articles that will tell you to ‘get out of piloting’ and ‘start scaling’ Generative AI. They tell you that your organization can only see benefits if you invest and scale now and stop waiting just by doing a few pilots and testing a variety of language models.

Let’s keep in mind a few things about these studies…

Most of these studies are organizations ‘rating’ themselves. Depending on who takes the survey can have a large impact on how the results pan out. Generally, companies want to see themselves as ‘leaders,’ so inherently maybe these studies are a bit skewed?

They create generalized categories like ‘champions’ and ‘laggards.’ While interesting to read ‘what’ you should do, we all know the ‘how’ to do it is the difficult part.

Ok, so let’s say the message of scaling now IS the right thing to do. Theoretically, you should be both (1) launching quick-win Generative AI tools across the organization and (2) finding ways to leverage the technology for large-scale, unique value use cases.

Even if you are doing both 1 and 2, how do you measure success?

Well, for customers, that might be straight forward if you have clear KPIs to measure against (e.g, reduced customer call-ins, increase in online purchases, etc.) However, leading indicators may be less straight forward. For example, if customers are using Generative AI tools to discover, learn, and select the best products or experiences to purchase online, this breaks the traditional tracking flows that identify how, when, and where customers are interacting with your digital experiences.

For employee experiences, the complications continue. If an employee used to use five applications to do a task and now uses one, how are you measuring that success besides surveying them? Also, how sure are we that employees can recognize time saved? Are they able to accurately measure that? And how long will team members be willing to be surveyed?

In addition, with a constantly shifting landscape, your organization may want to sacrifice measuring ROI to instead roll-out and launch a ‘new’ Generative AI experience. If breakthrough functionality comes to market, it might mean you have to sacrifice the ‘perfect measurement framework’ for a clearly improved experience for your users.

Finding ways to measure success will likely be different than the enterprise-wide measurement frameworks and systems your organization has in place today. We highly suggest communicating early and often about ROI expectations with your leadership team. Two things can be true at the same time: Yes, your organization should start investing and learning about a new technology that could transform experiences, and, yes, you should also set realistic goals of how best to track your progress; just don’t let it bog down your momentum.

The Capabilities and Limitations of the Tool Are Not Obvious to Users

Think of all the ways we try and simplify digital experiences today – hover-overs, pop-ups, step-by-step walkthroughs, short videos, etc. We’ve developed such intuitive and easy-to-use digital experiences that within seconds most users expect to quickly understand what the application does, how to use it, and what its limitations are. That expectation is the standard.

Ok…so those rules apply to Generative AI?

Not so much. Generative AI tools have your entire company and the world’s knowledge at its fingertips. But…it doesn’t magically know what you want. You must prompt it for what you need – with specific intent, details, expectations, and information. These new tools usually start with a blank canvas, a daunting experience for novice users.

It gets even more complicated. Sometimes these tools provide inaccurate data, are unable to answer, or flat-out hallucinate. For example, Generative AI tools sometimes don’t know when to say ‘I don’t know’ - they are programmed to help you find an answer; so just like an overly confident human, the tool may seem like it is giving you the right answer, when in reality, the incorrect answer is being provided.

Does this meet the ‘standard’ digital experiences have set over the past decade? Nope.

We must have a different mindset when using Generative AI tools. We cannot just be users, we must be explorers; willing to tweak, test, and re-try prompts until we get what we need. It’s an ongoing learning journey that will grow and evolve in the weeks, months, and (soon to be) years we engage with these powerful experiences.

With this in mind, we suggest preparing your organization for a significant investment in user enablement and adoption. Whether customers or employees, we must rethink and reimagine how to get humans to trust and believe in these tools as important experiences for them to engage with. This cannot be an afterthought like it might have been in the past.

Why Are Some People Using These Tools Like They Are 100% Accurate?

Have you used a Generative AI tool embedded in a meeting recently and your coworker sends the output to everyone and says, “Here are the meeting notes.” You look at the notes and immediately you notice (1) these aren’t action items, (2) content is missing, and (3) the notes don’t match the context of the conversation.

The reality is that humans NEED to be in the loop today. Yes, we all want these tools to take away from things we don’t enjoy doing (notetaking, anyone?), but we are promoting ineffective ways of working if we are willing to ignore inaccuracies just so we can personally move faster. Reality is – miscommunications will slowly add up, confusion will spread, and overall productivity will take a hit.

It’s important that team members are honest stewards of Generative AI. Let’s create cultures of learning, where team members feel comfortable saying not only what is working, but what is not. This means making sure our management teams and leaders are willing to hear the difficult experiences, while also pushing the teams forward to continue iterating and testing how best to leverage Generative AI.

How Can Rightpoint Help You?

While we wish we had a magic wand that could fix these problems, the reality is just like any technology shift, there are going to be growing pains. Let us be clear – we are all in on Generative AI. The question is not IF you should pay attention, it’s more about HOW you should get involved.

At Rightpoint, we believe in a pragmatic, human-centered, and quantitative approach to figuring out where Generative AI makes the most sense for your business. We start with the customer or employee experience to envision, define, design, and build magical moments that your customers and employees will be excited about. We are technology experts – selecting the best AI technology for your situation and problem statement. And we don’t just stop there – we zero-in on supportive enablement and adoption. We work side by side with your customers or employees to make sure all the effort you put into building a new experience doesn’t just sit on a shelf.

Reach out to us if you’d like to learn more about how we can help you on your Generative AI journey.