How We Built AI Conversations

How do you have those tough conversations with employees? This year, I was lucky enough to take part in developing a ground-breaking new tool that helps managers to develop their communication and leadership skills.

AI Conversations features realistic scenarios that managers might encounter in their own workplace, such as coaching conversations and delegation challenges. You get the chance to talk with a team member who’s brought to life by AI technology.

Like a character in a story, they have their own personalities and realistic responses. Your conversation ends with detailed feedback and suggestions for improvement when you have those challenging conversations in real life.

In this blog, I share our experience of working on this innovative new product. We’ll cover how we approached the process of crafting skillful AI-human interactions from a Content perspective, and what we learned in the process.

Getting to Grips With AI

No one on our team began this process as an AI expert. In fact, some of us were nervous about its potential to take our jobs! Many of us had tried it out a bit, asking ChatGPT to summarize ideas or generate poems in, say, the style of Roald Dahl.

At work, we’d just started exploring its potential uses. Generative AI could be fun, and with increasing exposure, we recognized that it could be a powerful tool for learning and creation.

When the AI Conversations project was first proposed, excitement mingled with trepidation. We worried that it was big and complicated for our small team, especially given the ambitious timescale.

Would we have to imagine entire workplace conversations? And how would we feed those into the AI? But a few chats with our project partners at Learning Pool reassured us. We learned that we’d create realistic management scenarios, and the AI would produce the conversations. We embraced this new challenge and moved ahead.

Deciding on Topics for AI Conversations

We know that having challenging conversations is one of the things that managers often dread. Armed with data around popular and highly requested topics from our client base, we focused on six key topic areas we know will resonate with line managers, regardless of their industry or sector. We also wanted to reflect the move, for many, to virtual or hybrid working environments.

The six scenarios that we developed were:

1. Managing Poor Performance

2. Goal Setting with the GROW Coaching Model

3. Getting a Project Back on Schedule

4. Delegating Customer Service Tasks

5. Managing Resistance to Hybrid Work Requirements

6. Managing a Disengaged Member of your Virtual Team

Developing Realistic AI Scenarios

The scenario development process began with Managing Editor Keith Jackson and Senior Editor/Writer Jonathan Hancock meeting at a seaside café. I’ll let Keith tell you about it in his own words:

“Fuelled by industrial amounts of coffee, we did what we do best: write compelling, engaging, yet practical and relevant scenarios. Jonathan and I are experienced journeymen in the field of work and management. We both had long experience of difficult management situations, and we were able to use that experience to write characters and scenes that our learners will encounter in their own careers.”

Given that we were adopting new skills, the Content Team used a buddying approach to develop the scenarios further. Once the first one was refined, the writer shared their learning and expertise with the next writer, and so on. This helped us to develop robust scenarios – and the skills to create them at pace.

Developing the Key Characters

So, we had our scenarios fleshed out. Time to create the characters that users would converse with! Each team member took charge of a scenario and imagined three different personalities that they might find in that situation.

For example, what type of people were likely to struggle with time management at work? Perhaps a new manager who lacked confidence, an employee with ADHD, or someone who focused on relationships at the expense of productivity.

Each of these became a character for whom we wrote AI prompts to program different priorities and speech patterns. With AI Conversations, when users select a scenario, the character they interact with is chosen at random.

So, you might meet a change-resistant team member who won’t return to the office; a disengaged direct report who feels cut adrift in the world of remote working; or an ambitious worker angling for promotion.

How We Prompted and Tested AI Conversations

Because we were working with a new technology, there were a few kinks to work out. Before testing our scenarios, the Learning Pool team gave us feedback on what elements we should tweak.

A key skill when writing prompts is not to overdo it! If you tell a generative AI program that your character “uses emojis,” it’s likely to use them in every statement. So, you have to temper your wording – saying, for example, that the character uses emojis “occasionally.”

Once we’d adjusted our prompts, it was time for Learning Pool to see how our scenarios performed!

AI Conversations Come to Life

Our partners at Learning Pool tested the AI interactions in a “sandbox” area, and we went through a final round of quality assurance to iron out a few wrinkles.

It was amazing to see the characters that we’d created interacting with us! We were surprised to see how realistic they were. Like humans, some of them did their best to wriggle out of tough conversations, while others checked in about your feelings. One team member ran a test where they tried to speak about anything other than the scenario, and the AI brought them back to the topic.

Once managers have practised having these challenging conversations, and received their feedback, that’s not the end of the story. Each module is supported by a package of carefully curated Mind Tools resources. These include videos, animations, infographics, and articles to help embed and expand the learner’s knowledge and skills.

Final Thoughts on Creating AI Conversations

The Mind Tools Content Team found it a refreshing and highly rewarding experience to work with Learning Pool on this project. We were out of our comfort zones, working with cutting-edge technology, and we learned a great deal from the process.

We’ve ended up with a high-quality, credible, effective learning tool that we’re confident can carry the Mind Tools and Learning Pool names into the market. We look forward to seeing managers and leaders interacting with the characters we created and practice navigating difficult conversations in an innovative way!