Rag

Llm Applications

Chatbots

Langchain

Mastering RAG: Build Smarter AI with LangChain and LangGraph in 2025

Hadiuzzaman

6 min readApr 2, 2025

Introduction

Have you ever asked an AI a question only to get an outdated or vague response? I’ve been there — frustrated with chatbots that can’t keep up with 2025’s fast-paced world. That’s when I discovered Retrieval-Augmented Generation (RAG), a game-changing approach that blends real-time data retrieval with advanced language generation. In this guide, I’ll walk you through how I built a RAG system using LangChain and LangGraph, transforming my AI from clueless to cutting-edge. Whether you’re a developer or simply curious about AI, you’ll learn why RAG is a must-know technique in 2025 — and how you can build one yourself.

RAG isn’t just a buzzword; it’s a lifeline for achieving accuracy in AI. Traditional language models rely on static, pre-trained knowledge, but RAG fetches fresh information from external sources, making it perfect for today’s dynamic information landscape. Let’s dive into what makes it tick and how you can harness its power with tools like LangChain and LangGraph.

What is RAG?

Imagine asking a company’s HR chatbot, “How many vacation days do I have left?” With RAG, the chatbot doesn’t simply guess — it retrieves your organization’s latest leave policy and crafts a precise answer. That’s the essence of Retrieval-Augmented Generation: pairing a Retriever (which searches a knowledge base) with a Generator (an LLM that crafts responses). The result? Answers that are accurate, up-to-date, and rich in context.

Of course, RAG isn’t without its challenges. For example, if an AI misinterprets a document or retrieves outdated information, it might generate inaccurate responses. However, the benefits generally outweigh these risks:

Accuracy: Fresh, domain-specific data is far superior to relying solely on static training sets.
Flexibility: There’s no need for retraining — simply update your knowledge base.
Relevance: Perfect for niche topics, real-time updates, or breaking news.

In 2025, RAG is evolving rapidly. Innovations like Long RAG (capable of handling 25,000+ tokens) and Adaptive RAG (which learns from user feedback) are pushing the boundaries, making it a hot topic among AI practitioners.

Why LangChain and LangGraph?

So why choose LangChain and LangGraph over alternatives like Haystack? LangChain simplifies the integration of LLMs with external data — be it documents, APIs, or databases — using an intuitive framework. LangGraph, on the other hand, orchestrates complex workflows (think cyclic agent behaviors) that modern AI demands. I chose these tools because they’re both beginner-friendly and powerful enough for seasoned developers, offering a gentler learning curve compared to some alternatives.

LangChain handles the heavy lifting of data retrieval, while LangGraph structures the process — much like a conductor guiding an orchestra. Together, they form a robust foundation for building reliable and scalable RAG systems. They’re also keeping pace with 2025’s trends by integrating with debugging tools like LangSmith.

Choosing the Right LLM

The heart of your RAG system is its language model. In 2025, while GPT-4 feels a bit dated, newer options are already making waves. Here are my top picks (inspired by Fuel Your Digital, 2025):

OpenAI’s GPT-5 with RAG Mode: Exceptionally accurate and customizable, albeit at a premium price.
Google’s Gemini AI: Offers real-time web access for fresh context, though privacy remains a concern.
Anthropic’s Claude 3.5: Known for its simplicity and collaborative nature, but less adept at real-time data integration.

For my build, I opted for GPT-5 because its RAG Mode significantly boosts retrieval precision — just keep in mind that it comes with a higher price tag. Your choice should be guided by your budget and specific use case, so testing multiple models might be the best approach.

Step-by-Step Guide: Building Your RAG System

Let’s get hands-on with a step-by-step walkthrough of how to build a RAG system in 2025.

Prerequisites

Before you start, ensure you have the following:

Python 3.8 or later
Libraries: langchain, langgraph, chromadb, openai
(Optional) langsmith for tracing and debugging

Install them using:

pip install --quiet --upgrade langchain-text-splitters langchain-community langgraph
pip install -qU langchain-openai
pip install -qU langchain-chroma

Step 1: Set Up the Environment

Begin by importing necessary libraries and configuring your API key. Also, enable LangSmith tracing if you plan to debug your process.

import getpass
import os
from langchain_openai import OpenAIEmbeddings
from langchain.chat_models import init_chat_model
from langchain_chroma import Chroma
import bs4
from langchain import hub
from langchain_community.document_loaders import WebBaseLoader
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langgraph.graph import START, StateGraph
from typing_extensions import List, TypedDict


if not os.environ.get("OPENAI_API_KEY"):
  os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter API key for OpenAI: ")

llm = init_chat_model("gpt-4o-mini", model_provider="openai")
embeddings = OpenAIEmbeddings(model="text-embedding-3-large")

vector_store = Chroma(
    collection_name="example_collection",
    embedding_function=embeddings,
    persist_directory="./chroma_langchain_db",  
)

Security Note: Always ensure your API keys are stored securely and never hard-code them in public repositories.

Step 2: Load and Index Documents

Next, load your documents, split them into manageable chunks, and store them in a vector database for efficient retrieval.

# Load and chunk contents of the blog
loader = WebBaseLoader(
    web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
    bs_kwargs=dict(
        parse_only=bs4.SoupStrainer(
            class_=("post-content", "post-title", "post-header")
        )
    ),
)
docs = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
all_splits = text_splitter.split_documents(docs)

# Index chunks
vector_store.add_documents(documents=all_splits)

Vector Store Options:

ChromaDB: Easy, open-source Limited scalability
Pinecone: High performance, managed Higher cost and setup required
Weaviate: Scalable, open-source Steeper learning curve

For simplicity, I used ChromaDB. However, for larger projects, you might consider Pinecone or Weaviate.

Step 3: Define the RAG Workflow with LangGraph

LangGraph helps you structure the RAG process by connecting document retrieval to response generation.


# Define prompt for question-answering
prompt = hub.pull("rlm/rag-prompt")


# Define state for application
class State(TypedDict):
    question: str
    context: List[Document]
    answer: str


# Define application steps
def retrieve(state: State):
    retrieved_docs = vector_store.similarity_search(state["question"])
    return {"context": retrieved_docs}


def generate(state: State):
    docs_content = "\n\n".join(doc.page_content for doc in state["context"])
    messages = prompt.invoke({"question": state["question"], "context": docs_content})
    response = llm.invoke(messages)
    return {"answer": response.content}


# Compile application and test
graph_builder = StateGraph(State).add_sequence([retrieve, generate])
graph_builder.add_edge(START, "retrieve")
graph = graph_builder.compile()

Step 4: Test It Out

Now, let’s test the system with a simple query:

response = graph.invoke({"question": "What is Task Decomposition?"})
print(response["answer"])

When I ran this test, the system delivered accurate, context-rich answers quickly — far surpassing the performance of a standalone LLM.

Real-World Use Case

RAG is not just theoretical. Consider Grab, a leading tech giant in Southeast Asia: they use RAG to automatically summarize lengthy reports, cutting down task time by 3–4 hours and boosting efficiency by 20% (Evidently AI, 2025). Imagine a customer support bot that instantly pulls up relevant customer data to resolve issues — that’s RAG in action. It’s no wonder businesses are adopting this approach, and why I’m so passionate about it.

Tips for Scaling

To transition your prototype into a production-ready system, consider these strategies:

Caching: Store retrieved documents to reduce latency for repeat queries.
Hybrid Search: Combine semantic search with traditional keyword search for more precise results (Databricks, 2025).
Monitoring & Evaluation: Track metrics such as response relevance and latency to continuously fine-tune performance.
Security Enhancements: Ensure your system secures sensitive data and API keys through best practices.

These adjustments can transform your initial prototype into a scalable, high-performance AI solution.

Conclusion

My journey from grappling with outdated AI responses to mastering RAG with LangChain and LangGraph has been transformative. We’ve built a system that is accurate, adaptable, and ready to meet the challenges of 2025. Looking ahead, innovations like long-context RAG (handling 25,000+ tokens) and web-integrated models like Gemini are set to redefine the landscape even further (Galileo AI, 2025).

Are you ready to try it? Build your RAG system today, experiment with these cutting-edge tools, and share your experience on Medium. Have you already implemented RAG in your projects? Drop a comment below — I’d love to hear your story. 🚀

Explore more and contact me

Thank you for reading! If you found this guide useful, consider exploring more of my content or connecting with me online:

Connect on LinkedIn
🔗 LinkedIn Profile
Explore My Projects on GitHub
🔗 GitHub Profile
💼 Available for Freelance Work
I’m open to freelance projects in mobile app development and machine learning & AI. Feel free to reach out!

Feel free to reach out or follow me for more content on AI, software development, and tech insights! Happy coding! 🚀