Orchestration with LangChain Core Components

and Memory

Business Scenario

Welcome!

Previously, connected the retriever and generator using the three RAG steps, designed and tested a grounded prompt with fallback instructions and citations, and built SmartCart AI v3 with follow-up search and code-verified product cards.

Today, your manager has assigned you a task to rebuild SmartCart AI v3 using LangChain prompt templates, chains, output parsers, and an integrated retriever. You will also add saved memory for each shopper so Kart can remember past conversations even after restarting, while keeping its answers accurate and consistent.

Git Pull

git pull origin branchName

Pre-Lab Preparation

Dataset :

Topic : Orchestration with Langchain Core Components & Memory

1) Langchain Core Components

2) Retrievers and Chains

3) Memory in Langchain

Task 1: Setup — Bring In the Pieces from Earlier Labs

import os

needed = ["SmartCart_Return_Policy.pdf", "SmartCart_Delivery_FAQ.docx", "smartcart_products.csv"]

for name in needed:
    if os.path.exists(name):
        print("OK      ", name)
    else:
        print("MISSING ", name, "-- upload it with the folder icon on the left")

Upload data files

Output

Install the libraries

!pip install -q langchain langchain-groq langchain-ollama langchain-community \
    langchain-text-splitters langchain-chroma langchain-huggingface \
    sentence-transformers pypdf docx2txt beautifulsoup4

from google.colab import userdata
os.environ["GROQ_API_KEY"] = userdata.get("GROQ_API_KEY")
print("Key loaded:", os.environ["GROQ_API_KEY"][:4] + "...")

Load the API Key

Define RAG Data Layer (copied from Lab 15)

os.environ["USER_AGENT"] = "SmartCartLab/1.0"

from langchain_community.document_loaders import (
    PyPDFLoader, Docx2txtLoader, WebBaseLoader, CSVLoader)
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma

DB_FOLDER = "smartcart_db"
SOURCES = [
    ("SmartCart_Return_Policy.pdf", "policy"),
    ("SmartCart_Delivery_FAQ.docx", "faq"),
    ("https://en.wikipedia.org/wiki/Cash_on_delivery", "web"),
    ("smartcart_products.csv", "product"),]
embedder = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

os.environ["USER_AGENT"] = "SmartCartLab/1.0"

from langchain_community.document_loaders import (
    PyPDFLoader, Docx2txtLoader, WebBaseLoader, CSVLoader)
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma

DB_FOLDER = "smartcart_db"
SOURCES = [
    ("SmartCart_Return_Policy.pdf", "policy"),
    ("SmartCart_Delivery_FAQ.docx", "faq"),
    ("https://en.wikipedia.org/wiki/Cash_on_delivery", "web"),
    ("smartcart_products.csv", "product"),]
embedder = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

def load_documents():
    """Stage 1 -- load every source and tag it with its doc_type."""
    docs = []
    for source, doc_type in SOURCES:
        if source.endswith(".pdf"):
            loader = PyPDFLoader(source)
        elif source.endswith(".docx"):
            loader = Docx2txtLoader(source)
        elif source.endswith(".csv"):
            loader = CSVLoader(source)
        else:
            loader = WebBaseLoader(source)
        for doc in loader.load():
            doc.metadata["doc_type"] = doc_type
            docs.append(doc)
    return docs

def chunk_documents(docs):
    """Stage 2 -- split documents into overlapping chunks of up to 300 characters."""
    splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=50)
    return splitter.split_documents(docs)

def build_vector_store(chunks):
    """Stage 3 -- empty the database, then embed and store all chunks."""
    store = Chroma(collection_name="smartcart", embedding_function=embedder,
                   persist_directory=DB_FOLDER)
    store.reset_collection()        # start empty -- safe to run again and again
    store.add_documents(chunks)
    return store

os.environ["USER_AGENT"] = "SmartCartLab/1.0"

from langchain_community.document_loaders import (
    PyPDFLoader, Docx2txtLoader, WebBaseLoader, CSVLoader)
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma

DB_FOLDER = "smartcart_db"
SOURCES = [
    ("SmartCart_Return_Policy.pdf", "policy"),
    ("SmartCart_Delivery_FAQ.docx", "faq"),
    ("https://en.wikipedia.org/wiki/Cash_on_delivery", "web"),
    ("smartcart_products.csv", "product"),]
embedder = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

def load_documents():
    """Stage 1 -- load every source and tag it with its doc_type."""
    docs = []
    for source, doc_type in SOURCES:
        if source.endswith(".pdf"):
            loader = PyPDFLoader(source)
        elif source.endswith(".docx"):
            loader = Docx2txtLoader(source)
        elif source.endswith(".csv"):
            loader = CSVLoader(source)
        else:
            loader = WebBaseLoader(source)
        for doc in loader.load():
            doc.metadata["doc_type"] = doc_type
            docs.append(doc)
    return docs

def chunk_documents(docs):
    """Stage 2 -- split documents into overlapping chunks of up to 300 characters."""
    splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=50)
    return splitter.split_documents(docs)

def build_vector_store(chunks):
    """Stage 3 -- empty the database, then embed and store all chunks."""
    store = Chroma(collection_name="smartcart", embedding_function=embedder,
                   persist_directory=DB_FOLDER)
    store.reset_collection()        # start empty -- safe to run again and again
    store.add_documents(chunks)
    return store

def get_retriever(k=3, doc_type=None):
    """Stage 4 -- open the saved database and return a retriever."""
    store = Chroma(collection_name="smartcart", embedding_function=embedder,
                   persist_directory=DB_FOLDER)
    search_kwargs = {"k": k}
    if doc_type:
        search_kwargs["filter"] = {"doc_type": doc_type}
    return store.as_retriever(search_kwargs=search_kwargs)

Build the Database (copied from Lab 15)

docs = load_documents()
print("Loaded ", len(docs), "documents")

chunks = chunk_documents(docs)
print("Created", len(chunks), "chunks")

build_vector_store(chunks)
print("Stored in", DB_FOLDER)

Output

Define get_llm() from Lab 13, now with json_mode switch

from langchain_groq import ChatGroq
from langchain_ollama import ChatOllama

CLOUD_MODEL = "qwen/qwen3.8-27b"     # GroqCloud, same model since Lab 7
LOCAL_MODEL = "qwen2.5:3b"           # Ollama, Lab 13

def get_llm(backend="cloud", json_mode=True):
    """Return the chat model for 'cloud' (GroqCloud) or 'local' (Ollama)."""
    if backend == "local":
        if json_mode:
            return ChatOllama(model=LOCAL_MODEL, temperature=0.3, num_predict=800, format="json")
        return ChatOllama(model=LOCAL_MODEL, temperature=0.3, num_predict=800)
    extra = {"response_format": {"type": "json_object"}} if json_mode else {}
    return ChatGroq(model=CLOUD_MODEL, temperature=0.3, max_tokens=800,
                    model_kwargs=extra, reasoning_effort="none")

Define input checks, from lab 10

EXIT_WORDS = ["quit", "exit", "bye"]
MAX_CHARS = 500
FRIENDLY_ERROR = "Sorry, I'm having trouble right now. Please try again in a moment."

def check_input(text):
    """Return (cleaned_text, problem). problem is "" when the input is fine."""
    cleaned = " ".join(text.split())
    if cleaned == "":
        return cleaned, "Please type what you are shopping for."
    if len(cleaned) > MAX_CHARS:
        return cleaned, f"Please keep your request under {MAX_CHARS} characters."
    return cleaned, ""

Create retriever & format_context(), (copied from Lab 16, task 2)

retriever = get_retriever(k=4)

def format_context(docs):
    """Number each chunk and label where it came from."""
    blocks = []
    for number, doc in enumerate(docs, start=1):
        source = os.path.basename(doc.metadata.get("source", "unknown"))
        blocks.append(f"[{number}] ({doc.metadata['doc_type']} | {source})\n{doc.page_content}")
    return "\n\n".join(blocks)

Task 2 : Prompt Templates

Build a template and fill it

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are Kart, SmartCart's friendly shopping assistant. 
    Keep answers under 60 words."),
    ("human", "{question}"),])

filled = prompt.invoke({"question": "gym earbuds under Rs. 2,000"})
for message in filled.to_messages():
    print(message.type, "->", message.content)
print("Placeholders:", prompt.input_variables)

Task 3 : LLMs & Output Parsers in a Chain

A text chain

from langchain_core.output_parsers import StrOutputParser, JsonOutputParser

text_llm = get_llm("cloud", json_mode=False)
chain = prompt | text_llm | StrOutputParser()

print(chain.invoke({"question": "gym earbuds under Rs. 2,000"}), "\n")

answers = chain.batch([{"question": "healthy office snacks"},
                       {"question": "formal shirt for office"}])
for answer in answers:
    print("-", answer[:70])

Output :

A JSON chain.

card_prompt = ChatPromptTemplate.from_messages([
    ("system", 'Reply in JSON: {{"products": [{{"name": "text", "price_rs": number}}]}}'),
    ("human", "{question}"),])
card_chain = card_prompt | get_llm("cloud") | JsonOutputParser()
data = card_chain.invoke({"question": "2 gym earbuds under Rs. 2,000"})
print(type(data))
print(data)

Output :

Task 4 : Retriever and Chain Abstractions

Build the RAG chain with grounding rules from Lab 16

from langchain_core.runnables import RunnablePassthrough

rag_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are Kart, SmartCart's shopping assistant. Answer using ONLY the context. "
               "If the context does not contain the answer, reply: "
               "\"Sorry, I couldn't find that in SmartCart's information.\" "
               "Only mention products from the context, with their exact price. "
               "Cite chunk numbers like [2]. Keep the answer under 80 words."),
    ("human", "Context:\n{context}\n\nQuestion: {question}"),])

rag_chain = (
    {"context": retriever | format_context, "question": RunnablePassthrough()}
    | rag_prompt
    | text_llm
    | StrOutputParser())

print(rag_chain.invoke("How many days do I have to return a shirt?"))

Output :

Check the chain gives the same quality as Lab 16

import pandas as pd

tests = [
    ("How many days do I have to return a shirt?", "15 days"),
    ("What is the delivery fee on a Rs. 300 order?", "40"),
    ("What is the maximum order value for cash on delivery?", "10,000"),
    ("How long does a UPI refund take?", "2 business days"),
    ("How much is the Logitech M185 mouse?", "699"),
    ("What is the warranty on the Redmi Note 13 5G?", "couldn't find"),
    ("Do you sell the iPhone 15?", "couldn't find"),]

answers = rag_chain.batch([question for question, expected in tests])

rows = []
for (question, expected), answer in zip(tests, answers):
    answer = answer.lower().replace("’", "'")
    rows.append({"question": question, "lc_chain": "Pass" if expected.lower() in
    answer else "Fail"})
print(pd.DataFrame(rows).to_string(index=False))

Output :

Task 5 : Conversational Memory in Langchain

Add memory to a chat chain and talk as 2 different shoppers

from langchain_core.prompts import MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.chat_history import InMemoryChatMessageHistory

chat_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are Kart, SmartCart's friendly shopping assistant. 
    Keep answers under 60 words."),
    MessagesPlaceholder("history"),
    ("human", "{question}"),])
chat_chain = chat_prompt | text_llm | StrOutputParser()

sessions = {}

def get_history(session_id):
    """Return this shopper's history, creating it the first time."""
    if session_id not in sessions:
        sessions[session_id] = InMemoryChatMessageHistory()
    return sessions[session_id]

chat_with_memory = RunnableWithMessageHistory(
    chat_chain, get_history,
    input_messages_key="question",
    history_messages_key="history",)
priya = {"configurable": {"session_id": "priya"}}
rahul = {"configurable": {"session_id": "rahul"}}

from langchain_core.prompts import MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.chat_history import InMemoryChatMessageHistory

chat_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are Kart, SmartCart's friendly shopping assistant. 
    Keep answers under 60 words."),
    MessagesPlaceholder("history"),
    ("human", "{question}"),])
chat_chain = chat_prompt | text_llm | StrOutputParser()

sessions = {}

def get_history(session_id):
    """Return this shopper's history, creating it the first time."""
    if session_id not in sessions:
        sessions[session_id] = InMemoryChatMessageHistory()
    return sessions[session_id]

chat_with_memory = RunnableWithMessageHistory(
    chat_chain, get_history,
    input_messages_key="question",
    history_messages_key="history",)
priya = {"configurable": {"session_id": "priya"}}
rahul = {"configurable": {"session_id": "rahul"}}

print("[priya]", chat_with_memory.invoke({"question": "running shoes for women under 3000"}, 
config=priya))
print("[priya]", chat_with_memory.invoke({"question": "make the second one cheaper"}, 
config=priya))
print("[rahul]", chat_with_memory.invoke({"question": "make the second one cheaper"}, 
config=rahul))
print("Messages saved for priya:", len(sessions["priya"].messages))

Output

Task 6 : Combine Memory with Retrieval

Build and test the rewrite chain

from langchain_core.messages import HumanMessage, AIMessage

rewrite_prompt = ChatPromptTemplate.from_messages([
    ("system", "Rewrite the shopper's latest message as one complete search question. "
               "Use the chat history to replace words like 'it' or 'that one'. "
               "Return only the question."),
    MessagesPlaceholder("history"),
    ("human", "{question}"),])
rewrite_chain = rewrite_prompt | text_llm | StrOutputParser()

from langchain_core.messages import HumanMessage, AIMessage

rewrite_prompt = ChatPromptTemplate.from_messages([
    ("system", "Rewrite the shopper's latest message as one complete search question. "
               "Use the chat history to replace words like 'it' or 'that one'. "
               "Return only the question."),
    MessagesPlaceholder("history"),
    ("human", "{question}"),])
rewrite_chain = rewrite_prompt | text_llm | StrOutputParser()

demo_history = [HumanMessage("How much is the boAt Airdopes 141?"),
                AIMessage("The boAt Airdopes 141 costs Rs. 1,299.")]
print(rewrite_chain.invoke({"history": demo_history, "question": 
"Is it returnable after opening?"}))

Output :

Join rewrite → retrieve → answer and add memory

memory_rag_prompt = ChatPromptTemplate.from_messages([
    rag_prompt.messages[0],                      # the same grounded system message as Task 4
    MessagesPlaceholder("history"),
    ("human", "Context:\n{context}\n\nQuestion: {question}"),])

def find_context(inputs):
    """Search with the rewritten question."""
    print("   (searching for:", inputs["standalone"] + ")")
    return format_context(retriever.invoke(inputs["standalone"]))

memory_rag_chain = (
    RunnablePassthrough.assign(standalone=rewrite_chain)    # adds "standalone"
    | RunnablePassthrough.assign(context=find_context)      # adds "context"
    | memory_rag_prompt
    | text_llm
    | StrOutputParser())

memory_rag_prompt = ChatPromptTemplate.from_messages([
    rag_prompt.messages[0],                      # the same grounded system message as Task 4
    MessagesPlaceholder("history"),
    ("human", "Context:\n{context}\n\nQuestion: {question}"),])

def find_context(inputs):
    """Search with the rewritten question."""
    print("   (searching for:", inputs["standalone"] + ")")
    return format_context(retriever.invoke(inputs["standalone"]))

memory_rag_chain = (
    RunnablePassthrough.assign(standalone=rewrite_chain)    # adds "standalone"
    | RunnablePassthrough.assign(context=find_context)      # adds "context"
    | memory_rag_prompt
    | text_llm
    | StrOutputParser())

kart_demo = RunnableWithMessageHistory(
    memory_rag_chain, get_history,
    input_messages_key="question", history_messages_key="history",)

anita = {"configurable": {"session_id": "anita"}}
for question in ["How much is the boAt Airdopes 141?", "Is it returnable after opening?"]:
    print("You :", question)
    print("Kart:", kart_demo.invoke({"question": question}, config=anita), "\n")

Output :

Task 7 : Rebuild SmartCart v3 with Persistent Memory

Define v3's chain and its saved memory

from langchain_community.chat_message_histories import FileChatMessageHistory

MEMORY_SIZE = 10
HISTORY_FOLDER = "chat_history"
json_llm = get_llm("cloud")                 # writes the final JSON answer

answer_prompt = ChatPromptTemplate.from_messages([
    ("system", """You are Kart, SmartCart's friendly shopping assistant for shoppers in India.

Answer using ONLY the numbered context from SmartCart's catalogue and documents.
Rules:
1. If the context does not contain the answer, say: "Sorry, I couldn't find that in SmartCart's 
information."
2. Only recommend products that appear in the context, with their exact name and price.
3. Only help with shopping. Reply in the shopper's style (English or Hinglish).
4. Use the earlier conversation to understand words like "it" or "the first one".
5. When the shopper compares products, think step by step before giving a verdict.

Always reply in JSON with exactly these keys:
{{"steps": ["reasoning steps, or an empty list"],
  "reply": "a short friendly answer, under 80 words",
  "products": [{{"name": "text", "price_rs": number, "category": "text", "reason": 
  "one short sentence"}}],
  "sources": ["file names from the context labels you used"]}}"""),
    MessagesPlaceholder("history"),
    ("human", "Context:\n{context}\n\nQuestion: {question}"),])

def keep_recent(inputs):
    """Send only the last MEMORY_SIZE messages to the model."""
    return inputs["history"][-MEMORY_SIZE:]

from langchain_community.chat_message_histories import FileChatMessageHistory

MEMORY_SIZE = 10
HISTORY_FOLDER = "chat_history"
json_llm = get_llm("cloud")                 # writes the final JSON answer

answer_prompt = ChatPromptTemplate.from_messages([
    ("system", """You are Kart, SmartCart's friendly shopping assistant for shoppers in India.

Answer using ONLY the numbered context from SmartCart's catalogue and documents.
Rules:
1. If the context does not contain the answer, say: "Sorry, I couldn't find that in SmartCart's 
information."
2. Only recommend products that appear in the context, with their exact name and price.
3. Only help with shopping. Reply in the shopper's style (English or Hinglish).
4. Use the earlier conversation to understand words like "it" or "the first one".
5. When the shopper compares products, think step by step before giving a verdict.

Always reply in JSON with exactly these keys:
{{"steps": ["reasoning steps, or an empty list"],
  "reply": "a short friendly answer, under 80 words",
  "products": [{{"name": "text", "price_rs": number, "category": "text", "reason": 
  "one short sentence"}}],
  "sources": ["file names from the context labels you used"]}}"""),
    MessagesPlaceholder("history"),
    ("human", "Context:\n{context}\n\nQuestion: {question}"),])

def keep_recent(inputs):
    """Send only the last MEMORY_SIZE messages to the model."""
    return inputs["history"][-MEMORY_SIZE:]
    
def find_kart_context(inputs):
    """Search with the rewritten, complete question."""
    return format_context(retriever.invoke(inputs["standalone"]))

kart_chain = (
    RunnablePassthrough.assign(history=keep_recent)
    | RunnablePassthrough.assign(standalone=rewrite_chain)
    | RunnablePassthrough.assign(context=find_kart_context)
    | answer_prompt
    | json_llm
    | StrOutputParser())

def get_saved_history(session_id):
    """Return the history saved in chat_history/<session_id>.json."""
    os.makedirs(HISTORY_FOLDER, exist_ok=True)
    return FileChatMessageHistory(f"{HISTORY_FOLDER}/{session_id}.json")

kart = RunnableWithMessageHistory(
    kart_chain, get_saved_history,
    input_messages_key="question",
    history_messages_key="history",
) | JsonOutputParser()

def ask_kart(question, session_id):
    """Return Kart's answer as a dictionary for this shopper."""
    return kart.invoke({"question": question},
                       config={"configurable": {"session_id": session_id}})

Define the chat window

def show(data):
    """Print the reasoning steps, answer, product cards and sources."""
    for step in data.get("steps", []):
        print("   (thinking)", step)
    print("Kart:", data.get("reply", ""))
    for p in data.get("products", []):
        print(f"   [{p['category']}] {p['name']} – Rs. {p['price_rs']:,} – {p['reason']}")
    if data.get("sources"):
        print("   Sources:", ", ".join(data["sources"]))
def chat():
    """Ask for the shopper's name, then chat until they type an exit word."""
    name = input("Your name: ").strip().lower().replace(" ", "_") or "guest"
    saved = len(get_saved_history(name).messages)
    print(f"SmartCart AI v3 -- hi {name}! {saved} saved messages. 'quit' to exit, 
    'new' to forget our chat")
    while True:
        user_input = input("\nYou: ")
        command = user_input.strip().lower()
        if command in EXIT_WORDS:
            print("Kart: Thanks for shopping with SmartCart. See you next time!")
            break
        if command == "new":
            get_saved_history(name).clear()
            print("Kart: Okay, I've forgotten our chat. What are you shopping for?")
            continue
        cleaned, problem = check_input(user_input)
        if problem:
            print("Kart:", problem)
            continue
        try:
            show(ask_kart(cleaned, name))
        except Exception as error:
            print("[log]", type(error).__name__, "-", str(error)[:80])
            print("Kart:", FRIENDLY_ERROR)

Chat as "priya". The cell waits for you to type: enter name, one request, then quit

chat()

Output

Prove that the memory survives the restart

Click on Runtime - Restart Session, this wipes variables but keeps files

Click on cell of chat(), then click on Runtime - Run Before, run the cell again

Output :

Look at what was saved

!head -c 300 chat_history/priya.json

Output

Great job!

You have successfully built prompt templates and chains using LangChain components, integrated retrieval into a RAG chain, added per-shopper memory, and enabled follow-up question rewriting before retrieval. You rebuilt SmartCart AI v3 with persistent memory that survives restarts.

Checkpoint

Git Push

Next-Lab Preparation

Topic : Agentic AI Tool Calling

1) AI Agents

2) Tool Calling

3) The ReAct Loop

git push origin branchName

GenAI - 17

GenAI - 17

Content ITV PRO

GenAI - 17

GenAI - 17

Content ITV PRO

More from Content ITV