A model that works in a notebook is a prototype, not a product. The last step, and the one tutorials most often skip, is deployment: making your model available to real users, reliably, quickly, and affordably. This closing chapter covers what that takes, from saving a trained model to serving it behind an API, handling the practical concerns of speed and cost, and keeping it healthy once traffic arrives.
Deployment starts with persistence. You do not retrain on every request; you train once, save the result, and load it where it is needed. A classical scikit-learn pipeline like the classifiers from earlier is saved with joblib, and a Hugging Face model is saved with its save_pretrained method, which writes both the model weights and its tokenizer so the exact same preprocessing travels with it. That last point is the production echo of a theme from the preprocessing chapter: the text transformation at serving time must be identical to the one used at training time, or predictions quietly degrade.
The standard way to expose a model is to wrap it in a small web service that accepts text and returns a prediction, usually as JSON over HTTP. FastAPI is a popular, lightweight choice in Python. The model is loaded once when the service starts, not on every request, and then each incoming call runs inference and returns the result.
from fastapi import FastAPI
from pydantic import BaseModel
from transformers import pipeline
app = FastAPI()
classifier = pipeline("sentiment-analysis") # loaded once at startup
class Request(BaseModel):
text: str
@app.post("/predict")
def predict(req: Request):
result = classifier(req.text)[0]
return {"label": result["label"], "score": float(result["score"])}
# Run with: uvicorn app:app --host 0.0.0.0 --port 8000
# Then POST {"text": "I love this product"} to /predictThat is the whole core of a production inference service. Everything else is making it fast, cheap, and dependable.
Once real traffic hits, three things dominate. Latency is how long a single prediction takes, and it matters most for anything user-facing. Throughput is how many requests you can serve at once, and batching several inputs together is the main lever for improving it. Cost and model size are tied together: large Transformer models are accurate but heavy, often needing a GPU to be fast, which is expensive, while a smaller distilled model or a classical TF-IDF classifier can run cheaply on a CPU and may be more than good enough. A recurring, healthy question in deployment is whether you actually need the biggest model, since the answer is often no.
You have two broad paths. You can self-host, running the model on your own servers, which gives full control and keeps data in-house but means you manage the infrastructure, scaling, and GPUs. Or you can call a hosted API from a model provider, which removes all the operational burden and is fast to start with, at the cost of per-request fees, sending data to a third party, and dependence on their availability. Smaller, well-understood models are often worth self-hosting; the very largest generative models are commonly accessed through an API. Many production systems mix both.
Deployment is not the finish line, because a model's world keeps changing. Log inputs and predictions so you can see what is actually happening. Watch for data drift, where real-world input gradually diverges from your training data and accuracy slips, which is a signal to retrain. Monitor latency and error rates like any other service, cache results for repeated queries, and keep a fallback for when the model is unavailable. A model in production is a living system that needs the same care as the rest of your application.
That completes the journey, from a raw string of text all the way to a deployed model serving predictions. Along the way you have built the full NLP toolkit: cleaning and tokenizing text, representing it as numbers, the core tasks of classification and entity recognition, the attention mechanism, the large language models that now lead the field, and the applications and deployment that turn all of it into something real.
The most natural next step, especially given how often hallucination and frozen knowledge came up, is to learn how to build accurate, grounded applications on top of large language models. The RAG field manual takes the embeddings, Transformers, and retrieval ideas from this series straight into production systems. And if you want to deepen the foundations underneath everything here, the Neural Networks and Deep Learning series is the companion that explains the architectures in full.
You now have a complete, practical picture of modern natural language processing. The best way to make it stick is to build something: take one task from this series, a sentiment classifier, an entity extractor, or a small grounded chatbot, and put it into a working application. That is where understanding turns into skill.
Sign in to join the discussion and post comments.
Sign in