Tag Archives: Cognitive Computing

Formula for cherry picking the right topics for your chatbot

Original Article Posted to LinkedIn on 22nd June 2017

So you’re building an AI powered chatbot to help customers on your website, but where do you start? Unless you’re in the position of knowing exactly what your customers will talk to your chatbot about, you’ll need to make some assumptions and pick your topics carefully.

A focus on domain specificity for your chatbot will benefit your customers and ensure you’re investing in the right thing.

This might sound obvious but, from my experience, too many companies are being tempted by the lure of “all knowing” chatbots and question answering systems that end up being all too generic and fall short of customer expectations.

As CTO at ICM Hub, I come at this subject from a particular perspective: we’re focused on AI automated customer service for airlines. Although our solution uses machine learning and natural language processing, delivered mostly in the form of a chatbot, our focus isn’t so much on the technology as it is on the impact we have on the final customer’s experience.

Getting specific is beneficial!

The technology needed to build chatbots and question answering systems is here, and there are businesses already investing in the development of these systems. However, one of the things I’ve seen a lot over the last fews years is a tendency to focus on the wrong thing: breadth. I think it is because people feel that knowing a little about a lot is better than knowing a lot about a little, but I’m not sure I agree. Lots of us will relate to taking the time to call a helpline, send an e-mail or start a web chat with a specific question or issue, only to be disappointed that the answer is too generic and doesn’t address our question at all.

General or domain agnostic solutions have their place, search engines are a great example: they cover massive amounts of content across every subject and can connect users with relevant information quickly and efficiently. They don’t, however, answer your question (yes, there are exceptions) or solve your problem, rather they help point you in the right direction. This approach is fine for a search engine, as our expectations are that they will do exactly that. On the other hand, when we take the time to engage with a company directly, we often expect more than simply being pointed in the right direction: we want personalised help.

Deciding what topics to cover

There’s a concept used within the world of question answering and information retrieval systems known as “the long tail graph”. I used it quite regularly during my time at IBM Watson to explain to businesses and development teams how to identify good topics to cover. The graph takes the form y = frequency and x = uniqueness and looks something like below.

The idea is that you focus the attention on topics that fall on the left hand side of the graph because they are the questions that occur most frequently and they are the easiest topics with which to train the machine learning. For question answering systems that focus on FAQs this is a good approach for the short tail, but the longer I’ve worked with businesses on conversational solutions (e.g. chatbots) for customer service the more I see that there is a need for an alternative perspective.

Consider value & complexity

Focusing so much on frequency and uniqueness leaves you at risk of spending time covering topics that offer the least value/cost or the highest complexity. This really isn’t how you want to get started: ideally you want to select the valuable topics that are most easily implemented. To deal with this, we find ourselves adding cost and complexity to our analysis to identify a number of candidate topics to cover.

Our approach is to analyse the domain using data from existing communication channels so we have a measure of frequency, cost, uniqueness and complexity. Cost represents the current cost of dealing with those topics while complexity is a discrete measure of how complex (1-10) the topic would be to automate. Visually the graph would look the same, but the topics that lie toward the top left can be quite different than those of other approaches.

y = frequency * cost | x = uniqueness * complexity

By following the above approach we ensure that we’re investing in the most frequently asked topics that carry highest value and are easiest to implement. The result is a reduction in the time needed to implement a solution capable of providing a return on investment and an increased chance at long term success. The idea is to get to a usable release and begin to get that valuable stream of data from the users as quickly as possible.

Don’t forget about breadth, just iterate

The idea that you’ll identify the most valuable topics to cover and only support those is the wrong mindset. Once you’ve released a chatbot, users will start to ask it questions; at this point, instead of guessing exactly what users will ask and how they’ll ask it, you’ll have real data to analyse.

My advice is to resist the temptation to start adding the topics that you THINK are needed and “let the data guide you”. You can then focus on the things your customers are actually asking that aren’t yet covered by looking again at the frequency, cost and complexity. As a result of this approach you will naturally begin to add the kind of breadth you were tempted to do from the beginning, but with greater confidence and a lower risk of wasting time.

What challenges have you faced selecting topics for a chatbot and how have you overcome them?


Recursive questions with IBM Watson Conversations

The approach detailed in the post is now out of date for many use cases, please see the update post Using Slots to collect data in IBM Watson Conversations.

IBM Watson released a new API earlier this year called Conversations which enables developers to build cognitive conversational solutions like chatbots and virtual assistants.

A common scenario when creating a chatbot is the need for clarification questions, for example you are creating a bot that helps users open a bank account. The user might say “I’d like to open a current account”, the bot understands that the user intends to “open account” and the account type is “current account” (AKA checking account in the US). What if the user says “I’d like to open an account”? In this case, in order to help the user, we’d need to know the type of account they want to open.

There are a number of way to approach the collection of the required information in a chatbot. The image below shows an approach of splitting the conversation based on the information provided.

Screen Shot 2016-08-30 at 13.07.41

If the users intent is classified as “Open_Account” and an entity of the type “Account_Type” is found then the first node will fire. If however the intent is classified as “Open_Account” but there is no entity then the second node will fire. This approach will work just fine for these examples but if we have many more pieces of information we require to complete a request, then this approach will be more difficult to maintain.

One of our goals when designing dialogs should be to reduce duplication and provide flexibility to the end-user. Reducing duplication will lower the cost of maintaining solutions, while flexibility in the way users can interact will provide a much better user experience. The following image shows a better way to approach this flow.

Screen Shot 2016-08-30 at 14.31.30

The key to this approach lies in utilising the “continue from” features of the Conversations service so the flow will respond intelligently and intuitively to users.

The first thing to note is the “Continue from…” node at the top of the second column of nodes. This tells the dialog to continue straight from the initial node with the #Open_Account condition to the next level. Note also that the output of the first node is empty, this is required as we don’t want our bot to say anything we just want it to move on and evaluate the second level of conditions. You can check that the node will not output anything by making sure that the output is set to {} in the advanced view as shown below.

Screen Shot 2016-08-30 at 14.07.00

The second thing to note is the second column of nodes. The first node has a condition on “Account_Type”, the second has a dummy condition of “true” (This always evaluates to true). This tells our dialog that if the user input has an account type then go ahead and fire the first node, if not ask the user what type of account they want.

The last thing we add is a new “Continue from”, the last node which is set to “User input”. This tells the dialog that every time we hit this node return the user input and evaluate this stack again (i.e Recursive).

This approach ensures our bot gets the information it needs to continue through the conversation but it also locks the user into the flow. In a future post I will explain techniques for allowing users to break out of the flow which is important when our bot doesn’t understand or when the user wants to talk about something else.

Accessing synonyms with IBM Watson Conversations

A couple of my clients have asked me how to access the synonym mentioned by a user when working with the IBM Watson Conversation service. Accessing the matching entity is well documented but it wasn’t apparent how to get the actual word or words mentioned by the user that the entity was matched on.

Having asked the question internally and received the answer I thought I would share in lieu of the documentation being updated. The answer is actually in the documentation here but without explanation so my clients and I missed it.

The scenario is pretty common and straight forward, lets say you have an entity called Account_Type with an entry for Bronze with a synonym of Basic. Your dialog flow for opening an account asks the user what type of account they would like to open, to which they say “I’d like to open a basic account please”. At this point your dialog is configured with:

“Okay, I’ll open a @Account_Type for you now.”


Example of outputting a matched Entity

The result will be a replacement of the @Account_Type placeholder with “Bronze”. This can be confusing for users and communicates a sense of inflexibility that is less than ideal. Instead we can access the synonym mentioned by the user with:

<?entities["Account_Type"][0].literal ?>

We configure our output as:

Okay, I’ll open a <?entities["Account_Type"][0].literal ?> for you now.


Example of outputting the synonym used

The result will now be that the placeholder is replaced with the word “basic”.