Building Chatbots and Conversational AI

Building Chatbots and Conversational AI

A chatbot is the application that ties this whole series together. To hold a conversation, a system has to understand what the user wrote, work out what they want, and produce a sensible reply, drawing on nearly every idea we have covered. This chapter looks at how conversational systems are actually built, from the simple pattern-matchers of decades past to the LLM-powered assistants of today, and where each approach still fits.

Three Generations of Chatbots

Conversational systems have evolved through three broad styles, and all three are still in use.

Rule-based bots match the user's text against hand-written patterns and return scripted replies. The famous 1960s ELIZA worked this way. They are predictable and need no data, which makes them fine for tiny, fixed menus, but they break the moment a user phrases something unexpectedly.

Retrieval-based bots pick the best response from a fixed set rather than inventing one. They understand the message, then select the most appropriate canned reply. Because every possible answer is written and approved in advance, they are reliable and safe, which is why they remain popular for customer support. The trade-off is that they can only say what is already in their library.

Generative bots write their replies word by word. Early versions used the encoder-decoder sequence models from the attention chapter; modern ones are built on the large language models from the previous chapter. They are flexible and fluent, capable of handling phrasing no one anticipated, at the cost of being harder to control and prone to the hallucination problem we discussed.

How a Task-Oriented Bot Understands You

A bot built to get something done, booking a table, checking an order, has a classic structure that reuses the core tasks from earlier in this series. It starts with intent classification: deciding what the user wants, which is just text classification where the labels are intents like "book_table" or "check_order." Then comes entity extraction, or slot filling: pulling out the details the intent needs, such as the date, party size, or order number, which is exactly named entity recognition. A dialogue manager tracks the state of the conversation and decides what to do next, including asking for any missing slots. Finally the bot generates or selects a response. Seen this way, a chatbot is largely an orchestration of skills you already have.

A Minimal Intent-Based Bot

Here is the understanding core in miniature: classify the intent, then respond accordingly. In a real system the classifier would be a trained model, but the control flow is the point.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline

# Tiny intent training set
examples = ["book a table", "reserve a table for two", "what are your hours",
            "when do you open", "cancel my booking", "cancel reservation"]
intents  = ["book", "book", "hours", "hours", "cancel", "cancel"]

bot = make_pipeline(TfidfVectorizer(), LogisticRegression())
bot.fit(examples, intents)

responses = {
    "book":   "Sure, what date and time would you like?",
    "hours":  "We're open 9am to 11pm every day.",
    "cancel": "No problem, what's your booking reference?",
}

def reply(message):
    intent = bot.predict([message])[0]
    return responses.get(intent, "Sorry, I didn't catch that. Could you rephrase?")

print(reply("I'd like to reserve a table"))   # Sure, what date and time would you like?
print(reply("what time do you close"))         # We're open 9am to 11pm every day.

That fallback line in the last branch matters more than it looks: deciding what to do when the bot is unsure is one of the hardest parts of a real conversational system.

The Modern LLM Approach

Today you can skip much of that machinery by handing the conversation to a large language model. You give it a system instruction describing its role, pass the running conversation history so it has context, and let it generate each reply. This produces remarkably natural conversation with little setup. The catch is the one from the previous chapter: an LLM left to its own memory will confidently invent facts. The standard fix for any bot that must be accurate about your specific information, your products, policies, or documents, is to retrieve the relevant material and give it to the model along with the question, so its answer is grounded in real sources. That is retrieval-augmented generation, and building exactly this kind of grounded assistant is what the RAG field manual walks through end to end. For most serious chatbots being built now, an LLM grounded with retrieval is the architecture to reach for.

What Makes a Chatbot Good

• Handling the unexpected gracefully, with sensible fallbacks instead of nonsense.
• Keeping track of context across turns, so it remembers what was said a moment ago.
• Knowing its limits and handing off to a human when needed.
• Staying accurate, which for fact-based bots means grounding answers in real data rather than trusting the model's memory.

Key Takeaways

• Chatbots come in rule-based, retrieval-based, and generative styles, all still used today.
• Task-oriented bots combine intent classification (text classification) and slot filling (NER) with a dialogue manager.
• Retrieval bots are safe and predictable; generative bots are flexible but harder to control.
• Modern bots use LLMs with a system prompt and conversation history.
• Grounding an LLM bot with retrieval (RAG) is the standard way to keep it accurate.

What's Next

A model that works in a notebook is not yet a product. The final chapter covers what it takes to put an NLP model into production: serving it behind an API, the practical concerns of latency and cost, and how to keep it healthy once real users arrive.