James Mesbur, VP of Product (Conversational AI) at Micoworks
Organizations continue to struggle to harness the power of Generative AI while mitigating the risks it brings.
ChatGPT and sky-high expectations
The launch of ChatGPT by OpenAI in December 2022 created a new set of expectations around what a Conversational AI experience should be. Suddenly, awkward interactions, misunderstandings, and grammatical errors became unacceptable literally overnight, and the world realized AI had finally mastered the ability to understand and produce natural, human language.
Or had it?
As 2023 progressed, the fervor surrounding Generative AI was red hot. While overall venture investment was down, over $50B went to Generative AI and other AI companies, including the (in)famous $10B that Microsoft poured into OpenAI. Much of this money flowed to companies building foundational models or providing infrastructure, including OpenAI competitors Mistral, Anthropic, Inflection AI, and Aleph Alpha.
On the other hand, companies launching successful conversational AI products built on top of Generative AI have been few and far between. There have been several prominent failed launches, tons of hype, but practically zero proven successes. By success, I mean a company launching something and showing an actual quantifiable improvement in a business KPI compared to a pre-Generative AI state.
Organizations continue to struggle to harness the power of Generative AI while mitigating the risks it brings.
"During a gold rush, sell shovels"
This pattern is reminiscent of the California Gold Rush during the mid-1800's, when the people making the real money were those providing lodging and provisions or selling shovels and other extraction equipment, while those searching for gold found harsh living conditions, high prices, and zero return on their investment.
So how can you set your organization up for success in this new world of extreme executive pressure to 'use Generative AI', and not fall prey to the hype? First, let's consider a couple of common scenarios encountered at organization after organization over the past 18 months or so:
Scenario 1: The Optimistic Executive
You are a high-ranking executive, and you want your team to 'do Generative AI'. You are fascinated by the potential, don't want to be left behind, and expect enormous cost benefits based on what you've been seeing in press releases. You may have been burnt in the past by technology trends (think blockchain, NFT's, the metaverse), but you feel that somehow this time things are different, and Generative AI will change the way companies operate, the way customers interact with businesses, and is truly a revolutionary technology that your team must jump on immediately to supercharge your productivity and profit potential.
That said, if your team isn't raising the alarm bells and pushing back, you should be concerned.
Scenario 2: The Cautious Tech Leader
You're a CTO, Technology leader, or Customer Experience executive, and have been told by your CEO to 'do Generative AI'. You oversee day-to-day operations and understand 'how the sausage is made'. You are skeptical of Generative AI because of everything you've seen in the news about the colossal failures, the data security, privacy, and legal risks, hallucinations, and misapplication of Generative AI to use cases which makes the customer experience worse, not better. You also recognize that Generative AI is not cheap and requires specific, expensive skills in your team to execute.
That said, if your leadership isn't pushing you to explore the possibilities of Generative AI in the context of your business, you should be concerned.
So where to start?
How can you operate in this new world of ambiguity, wild claims, and mad rush to deliver something, anything, regardless of quality?
My advice to both the Optimistic Executive and the Cautious Tech Leader would be to take a deep breath and remember that business fundamentals don't change just because a new technology exists. Unless you're in the business of building foundational models or providing infrastructure or tools, it is prudent to recognize that your customers don't care about technology; they care about solving for their pain points in whatever way makes the most sense, is cost-effective, and fits into their current operations smoothly.
So just like always, the focus should be on understanding your customers deeply and making rational decisions about technology based on actual projected impact.
The Success Checklist for Generative AI Projects
I originally put this 10-question checklist together for a presentation I gave to a room full of 100 executives here in Tokyo. It's Business 101, but I feel a lot of fundamentals get thrown out the window in fast-paced environments such as this new era of ChatGPT, so it can be worth slowing down for a moment and revisiting the basics to make sure you remain on the right track.
The Checklist
- What are we solving for?
- What is the existing situation: baseline KPIs?
- What are our goals in doing this?
- Why haven't we already done this another way: bottlenecks?
- If we address bottlenecks, do we need Generative AI?
- If we use Generative AI, why do we believe the outcome will be better?
- What will the incremental cost be?
- Is there ROI?
- If not, are we OK with that because of other benefits?
- How will our organization and operational model need to change to support these new projects?
I'll go through item #1 in this article and continue in the next newsletter.
1. What are we solving for?
This seems so simple, yet I can't count the number of times I've seen organizations get this wrong. The pattern is:
- See exciting new technology
- Decide we must use it
- Throw it at any problem you can find
- Hope
This is of course the exact opposite approach that you should be following. Just like with any technology, Generative AI is simply another tool to consider using when solving business problems, but the direction should always be to start with the problem before jumping to the solution. Without firm rigor at the start of any project, Generative AI or not, the result will be confusion, chaos, and dysfunction in your organization.
With a hammer, everything looks like a nail: Customer Support Chatbot
The number one most common mistake I've seen organizations make over the past year has been to assume that Generative AI can replace their customer support agents. They try ChatGPT, are amazed at the apparent understanding and fluency shown by the responses, and think "Hey, let's do ChatGPT for our customer support chatbot! Just upload our data into ChatGPT and let it figure out the rest!"
What could possibly go wrong?
The first thing to do is understand what your organization is actually solving for. Customer Support exists because something in the business is fundamentally broken, and human intervention is needed to put things right. Anyone who has spent more than 5 minutes in a call center listening to actual calls understands immediately that Customer Support is not an FAQ. Customer Support is not quick and easy responses to common inquiries. Customer Support requires knowledge, experience, access to appropriate tools and automation, and in many cases authority to make decisions and act on them.
With Customer Support specifically, there are likely hundreds of bottlenecks, broken processes, inconsistent or nonexistent knowledge in the knowledge base, lack of automation, and countless other issues that are currently driving customer inquiries and creating complexity for operators dealing with them. This is what needs to be solved for, and Generative AI is the wrong tool.
A Square Peg in a Square Hole: Customer Support Human Augmentation
So where can Generative AI play a role in Customer Support? While inappropriate for a standalone customer-facing chatbot, the beauty of Generative AI and large language models is their ability to perform language-based tasks with high quality. There are at least 2 great use cases that are common pain points across most Customer Support organizations where Generative AI shines:
- Post-call work: after every interaction with customers, operators typically spend several minutes summarizing the conversation and filling out a bunch of drop-down menus to categorize it so it can be reported on. Generative AI is great at the summarization task and assigning pre-defined categories to text.
- Quality assurance: this is typically done manually, with supervisors listening to a small number of calls each month and then scoring them based on a standard multi-question form. Again, with the right prompting, Generative AI can score calls at a scale that humans cannot, providing tremendous value-add.
Stay tuned for the next installment where we'll work through a few more items in the list. In the meantime, when deciding whether Generative AI is right for your organization, start with the problem rather than jumping to the solution!
