Building Chatbots using Python — Part 0

In this blog, we will learn how to build our first chatbot. After gaining a bit of historical context, we will set up a basic structure for receiving text and responding to users, and then learn how to add the basic elements of personality. And finally, we will learn to build a rule-based system for parsing text.

Photo by Hitesh Choudhary on Unsplash

When you build a chatbot, it is necessary to understand what chatbots do and what it looks like. We have heard of Google Allo, Siri, IBM Watson, etc. The fundamental problem that these bots try to solve is to become an intermediary and help users become more productive. They do this by allowing the user to worry less about how the information will be retrieved, and the input format needed to attain specific data. Bots tend to become more intelligent as they handle user data input and gain more insights. Chatbots are successful because they give you precisely what you want.

A brief history of Chatbot

Conversational software, commonly known as ‘Chatbot,’ is not a new concept! Chatbot development is way more comfortable than it was a few years ago, but chatbots did exist decades ago. However, the popularity of chatbots has increased exponentially in the last few years.

Command-line applications (Console applications) were introduced in the 1960s with the keyboard+video screen terminal. We need to type instructions using a rigorous syntax for using a command-line app, but it is much closer to human language than the underlying machine instructions.

In 1966, ELIZA, the first Chatbot, was created. This now-famous program was able to hold a conversation by using a pattern-matching and substitution technique. Despite the relatively simple code behind ELIZA, it was a pretty compelling conversationalist. In this blog post, we will build our minimal version of the ELIZA chatbot, which will lay the foundation for making some more complex chatbots.

Image from Wikipedia

EchoBot

Let us begin with a simple task by building a bot called EchoBot. It merely echoes back to you whatever you say to it. Initially, all the bots that we will make in this blog series will receive messages in python code and print their responses. Later, we will learn to connect our bots to various messaging apps.

To build an EchoBot, we define a ‘respond’ function, which takes a message as an argument and returns an appropriate response.

bot_template = "BOT : {0}"
user_template = "USER : {0}"
# Define a function that responds to a user's message: responddef respond(message):# concatenate the user's message to the end of a standard botrespone bot_message = "I can hear you! You said: " + message# Return the result return bot_message# Test functionprint(respond("hello!"))
output

Here we define a function using the keyword ‘def,’ then the name of the function, then its arguments in parentheses, and then a colon. The body of the function is indented by one level. We specify the output generated by the function using the ‘return’ keyword. If a function doesn’t have a ‘return’ statement, it returns a ‘None’ type.

# Create templates
bot_template = "BOT : {0}"
user_template = "USER : {0}"
# Define a function that sends a message to the bot: send_messagedef send_message(message):# Print user_template including the user_message print(user_template.format(message))# Get the bot's response to the message response = respond(message)# Print the bot template including the bot's response. print(bot_template.format(response))
# Send a message to the bot
send_message("hello")
output

We can insert variables into a string in Python by using the string’s ‘format’ method. Inside the respond function is a string containing curly brackets. These act as placeholders, and will get replaced by the value of the argument we pass when we call ‘format.’ To keep track of everything that’s being said, we’ll define another function called ‘send_message’ that prints what the user just said, gets the response by calling the respond function, and then prints the bot’s response.

Creating a personality

Creating an engaging personality is a critical part of chatbot development. It’s one of the key differences as compared to any other kind of software.

Why creating a personality for the Chatbot is so essential? — Let us suppose, with our Chatbot, all we could do was type precise instructions to our bot. We would just have a command-line application and not a chatbot.

Most chatbots are rooted in a messaging app that people are comfortable using to talk to their friends. And we can expect that the users of our will want to make a bit of smalltalk before trying out any functionality that they came for. It’s not much effort to code up some responses to common questions and is worth it for the improved user experience.

Smalltalk and Including variables

# Define variables
name = "Shrusti"
weather = "sunny"
# Define a dictionary with the predefined responsesresponses = {"what's your name?": "my name is {0}".format(name),"what's today's weather?": "the weather is {0}".format(weather),"default": "default message"}# Return the matching response if there is one, default otherwisedef respond(message):# Check if the message is in the responses if message in responses:# Return the matching message bot_message = responses[message] else:# Return the "default" message bot_message = responses["default"] return bot_message

The simplest thing we can do is use a python dictionary, with user messages as the keys and responses as the values. For example, say we define a dictionary called ‘responses,’ with the messages “what’s your name?” and “what’s today’s weather” as keys, and suitable responses as values. Next, we define a function called ‘respond,’ which accepts a message as a single argument. This function tests if a message has a defined response by using the ‘in’ keyword, that is, “if message in responses.” This statement only returns ‘True’ if the message corresponds to one of the dictionary’s keys. Notice that this will only work if the user’s message exactly matches a key in the dictionary. In later parts of this blog, we will build much more robust solutions. Notice that if there isn’t a matching message, the ‘return’ keyword will never be reached so that the function will return None.

Since the world outside is always changing, our bot’s answers have to be able to as well. The first thing that we can do is add some placeholders to the responses. For example, instead of “The weather is sunny,” you can have a template string, like “it’s {} today.” Then later, you can insert a variable ‘weather_today’ by calling format weather today.

Choosing responses

# Import the random module
import random
name = "Shrusti"
weather = "sunny"
# Define a dictionary containing a list of responses for each messageresponses = {"what's your name?": ["my name is {0}".format(name),"they call me {0}".format(name),"I go by {0}".format(name)],"what's today's weather?": ["the weather is {0}".format(weather),"it's {0} today".format(weather)],"default": ["default message"]}
# Use random.choice() to choose a matching responsedef respond(message): if message in responses: bot_message = random.choice(responses[message]) else: bot_message = random.choice(responses["default"]) return bot_message

It gets dull hearing the same responses over and over again, so it’s an excellent idea to add a little variety! To return completely different responses, we can replace the values in the responses dictionary with lists. Then when we are choosing a response, we can randomly select an answer from the appropriate list. To do this, import random, and use the ‘random.choice’ function, passing the list of options as an argument. For now, we are still relying on the user message matching our predefined messages exactly, but we’ll soon move to a more robust approach.

Asking questions

import randomresponses = {"question": ["I don't know :(","you tell me!"],"statement": ["can you back that up?","tell me more!","oh wow!",":)","how long have you felt this way?","I find that extremely interesting","why do you think that?"]}
def respond(message):# Check for a question mark if message.endswith("?"):# Return a random question return random.choice(responses["question"])# Return a random statement return random.choice(responses["statement"])# Send messages ending in a question marksend_message("what's today's weather?")send_message("what's today's weather?")
# Send messages which don't end with a question marksend_message("I love building chatbots")send_message("I love building chatbots")
output

A great way to keep users engaged is to ask them questions or invite them to go into more detail. This was actually one of the things which made the ELIZA bot, so fun to talk to. Instead of using a bland default message like “I’m sorry, I didn’t understand you,” you can use some phrases that invite further conversation. Questions are a great way to achieve this. “Why do you think that?”, “How long have you felt this way?”, and “Tell me more!” are appropriate responses to many different kinds of message, and even when they don’t quite match are more entertaining than a boring fallback

Text processing with regular expressions

Regular expressions are a very useful tool for processing text. We will use them to match messages against known patterns, extract key phrases, and transform sentences grammatically. These are the core pieces we need to create our ELIZA style bot.

The regex behind ELIZA

Much of the ELIZA system’s magic relied on giving the impression that the bot had understood you, even though the underlying logic was extremely simple.
For example, asking ELIZA, “do you remember when you ate strawberries in the garden?” ELIZA would respond: “How could I forget when I ate strawberries in the garden?”. Part of what makes this example so compelling is the subject. We are asking about memories, which we associate with our conscious minds and our sense of self. The memory itself, of eating strawberries in the garden, invokes powerful emotions. But if we pick apart how the response is generated, we see that it’s quite simple.

Pattern matching and Extracting keyphrases

To build an ELIZA-like system, we need a few key components. The first is a simple pattern matcher. This consists of rules for matching user messages, like “do you remember x” To match patterns we use regular expressions. To use these in Python, we ‘import re’ Regular expressions are a way to define patterns of characters and then seeing if those patterns occur in a string. In regular expressions, the dot character is special and matches any character. The asterisk means “match 0 or more occurrences of this pattern”, so “dot star” matches any string of characters.

Adding parentheses in the pattern string defines a ‘group’. A group is just a substring that we can retrieve after matching the string against the pattern. We use the match object’s ‘group’ method to retrieve the parts of the string that matched. The default group, with index 0, is the whole string. The group with index one is the group we defined by including the parentheses in the pattern.

import re
import random
rules = {'do you think (.*)': ['if {0}? Absolutely.','No chance'],'do you remember (.*)': ['Did you think I would forget {0}',"Why haven't you been able to forget {0}",'What about {0}', 'Yes .. and?'],'I want (.*)': ['What would it mean if you got {0}','Why do you want {0}',"What's stopping you from getting {0}"],'if (.*)': ["Do you really think it's likely that {0}",'Do you wish that {0}','What do you think about {0}','Really--if {0}']}# Define match_rule()def match_rule(rules, message): response, phrase = "default", None# Iterate over the rules dictionary for pattern, responses in rules.items():# Create a match object match = re.search(pattern, message) if match is not None:# Choose a random response response = random.choice(responses) if '{0}' in response: phrase = match.group(1)# Return the response and phrase return response.format(phrase)# Test match_ruleprint(match_rule(rules, "do you remember your last birthday"))
output

Grammatical transformation

To make responses grammatically coherent, we will want to transform the extracted phrases from first to second person and vice versa. In English, conjugating verbs is easy, and simply swapping “I” and “you,” “my,” and “your” works in most cases. We can use another function from the ‘re’ module for this: ‘re.sub’. This substitutes patterns in a string. For example, take the sentence “I walk my dog”. re.sub “You walk your dog.”

# Define replace_pronouns()def replace_pronouns(message):     message = message.lower()     if 'me' in message:# Replace 'me' with 'you'         return re.sub('me', 'you', message)     if 'my' in message:# Replace 'my' with 'your'         return re.sub('my', 'your', message)     if 'your' in message:# Replace 'your' with 'my'          return re.sub('your', 'my', message)     if 'you' in message:# Replace 'you' with 'me'          return re.sub('you', 'me', message)    return message
print(replace_pronouns("my last birthday"))print(replace_pronouns("when you went to Florida"))print(replace_pronouns("I had my own castle"))
output

Putting it all together

The final step is to combine these logical pieces together. We start with a pattern and a message. We extract the key phrase by creating a match object using pattern dot search and then use the group method to extract the string represented by the parentheses. We then choose a response appropriate to this pattern and swap the pronouns so that the phrase makes sense when the bot says it.
We then insert the extracted phrase into the response to partially echo back what the user talked about, giving the illusion that the bot has understood the question and remembers this experience.

# Define respond()def respond(message):# Call match_rule    response = match_rule(rules, message)    if '{0}' in response:# Replace the pronouns in the phrase        phrase = replace_pronouns(phrase)# Include the phrase in the response        response = response.format(phrase)    return response# Send the messagessend_message("do you remember your last birthday")send_message("do you think humans should be worried about AI")send_message("I want a robot friend")send_message("what if you could be anything you wanted")
output

We learned how to code a simple chatbot like EchoBot and ELIZA. In this series further, we will learn to make more complex chatbots.

Inferring the implications of the world awash with data.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store