A Guide to Advanced Prompting with LangChain

Large Language Models (LLMs) offer incredible capabilities for text generation, summarization, and understanding. These capabilities can automate complex workflows and deliver exceptional user experiences through applications that integrate with them. However, harnessing the full potential of LLMs often comes down to one crucial skill: effective prompting. LLMs are powerful, but getting them to do exactly what you want consistently can be a nuanced art. This is where LangChain comes in. LangChain is an open-source framework designed to simplify the development of applications powered by LLMs. It provides a rich set of tools and abstractions that make working with these complex models more manageable. And at the heart of controlling LLM behavior lies the prompt. In this post, we'll dive into several powerful prompting techniques you can leverage with LangChain, inspired by practical examples, to build more intelligent and reliable LLM applications. We'll cover everything from reusable templates to managing conversations and guiding complex reasoning.

What We'll Cover

• Prompt Templates
• Human, AI, and System Messages
• Few-Shot Prompting
• Chain of Thought Prompting
• Basic Chatbot Conversation Management
• The Modern Prompting Lifecycle with LangSmith and LangGraph

Prompt Templates: Your Blueprint for Reusable LLM Behavior

Think of Prompt Templates as the formatting strings of the LLM world. They are structures that allow for dynamic input into a predefined prompt, enabling you to capture reusable LLM functionality.

Why are they so useful?

• Reusability: Define a task once (like translating text) and reuse it with various inputs.
• Consistency: Ensure your LLM receives instructions in a standardized format.
• Clarity: Make your prompts easier to read, manage, and debug.

Here's how to create a simple one in LangChain:

from langchain_core.prompts import ChatPromptTemplate

english_to_spanish_template = ChatPromptTemplate.from_template(
    "Translate the following from English to Spanish. Provide only the translated text: '{english_statement}'"
)

prompt = english_to_spanish_template.invoke({"english_statement": "Today is a good day."})

# This 'prompt' object is now ready to be passed to an LLM.

In this snippet, {english_statement} is a placeholder that gets filled when you .invoke() the template with a dictionary.

Decoding Conversations: Human, AI, and System Messages

Chat models are designed for dialogue. They expect a sequence of messages, each with a defined role, to understand the context of a conversation. LangChain provides clear ways to represent these roles.

Core Message Types:

• HumanMessage: Represents input from the user.
• AIMessage: Represents output from the LLM.
• SystemMessage: Sets the overall context, persona, or "rules of engagement" for the AI throughout the entire conversation. Think of it as the director's note to the AI actor.

Notably, LangChain offers an abstraction over the default APIs offered by LLM providers. For example, Anthropic and Open AI have different ways of providing “system” messages. By using LangChain, we are able to abstract those aspects away.

from langchain_core.messages import AIMessage
from langchain_core.prompts import ChatPromptTemplate

pirate_template = ChatPromptTemplate.from_messages([
    ("system", "You are a pirate. Your name is Sam. You always talk like a pirate."),
    ("human", "{prompt}")
])

# Example for Few-Shot Prompting
prompt_with_examples = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant that translates English to French."),
    ("human", "Hello"),
    AIMessage(content="Bonjour"), # Using AIMessage for clarity
    ("human", "{user_input}")
])

Key Takeaway: Understanding and using these message types is fundamental for building any interactive, context-aware, or persona-driven LLM application.

Supercharge Your Prompts with Few-Shot Prompting

Few-Shot Prompting is a powerful technique where you provide the LLM with a few examples (the "shots") of the desired input-output behavior directly within the prompt. This helps the LLM "learn" the task or the desired format in context without needing lengthy instructions; you show the model what you mean, rather than just telling it. This method leads to more precise, customized, and often more accurate outputs.

from langchain_core.prompts import FewShotChatMessagePromptTemplate, ChatPromptTemplate
# Define a few-shot prompt for extracting technical specs from product blurbs
examples = [
    {
        "input": "Google Nest Wifi, network speed up to 1200Mpbs, 2.4GHz and 5GHz frequencies, WP3 protocol",
        "output": """{
  "product":"Google Nest Wifi",
  "speed":"1200Mpbs",
  "frequencies": ["2.4GHz", "5GHz"],
  "protocol":"WP3"
}"""
    },
    {
        "input": "Apple AirPods Pro (2nd gen), Bluetooth 5.3, ANC, MagSafe case, IP54",
        "output": """{
  "product": "Apple AirPods Pro (2nd gen)",
  "bluetooth": "5.3",
  "features": ["ANC", "MagSafe case"],
  "ip_rating": "IP54"
}"""
    },
]

example_prompt = ChatPromptTemplate.from_messages(
    [
        ("human", "INPUT:\n{input}\n\nReturn ONLY valid JSON."),
        ("ai", "{output}"),
    ]
)

few_shot_prompt = FewShotChatMessagePromptTemplate(
    examples=examples,
    example_prompt=example_prompt,
)

final_prompt = ChatPromptTemplate.from_messages(
    [
        ("system",
         "You extract technical specifications from short product blurbs. "
         "Return ONLY strict, minified JSON with consistent keys. "
         "If a field is missing, omit it (do not invent)."),
        ("human", "Here are examples of the exact format to follow:"),
        few_shot_prompt,
        ("human", "Now extract from this:\n{input}\n\nReturn ONLY JSON."),
    ]
)

Unlocking Complex Reasoning with Chain of Thought (CoT)

LLMs can sometimes "jump to conclusions" when faced with problems that require step-by-step reasoning (e.g., math problems, logic puzzles). Chain of Thought (CoT) Prompting is a technique that encourages the LLM to "think step by step" by explicitly prompting it to generate intermediate reasoning before arriving at a final answer.

Implementing Zero-Shot CoT in LangChain

• Few-Shot CoT: Provide examples that explicitly demonstrate the step-by-step reasoning process. You would structure these examples using HumanMessage and AIMessage pairs.
• Zero-Shot CoT: A surprisingly effective and simpler approach for some models is to just append a phrase like "Let's think step by step" to your problem description.

zero_shot_cot_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are an expert at solving logic puzzles."),
    ("human", "{problem_description} Let's think step by step.")
])

Key Takeaway: CoT prompting can significantly enhance an LLM's ability to tackle complex reasoning tasks by guiding it through a more deliberative, step-by-step generation process.

Building a Chatbot: Rudimentary Conversation Management

At their core, LLMs are stateless. To build a chatbot that remembers the conversation, we must explicitly provide the conversation history with each new user message. LangChain facilitates this through Message Placeholders. You can include a special ('placeholder', '{variable_name}') message type in your template, which allows you to dynamically inject a list of past messages.

chat_template = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly and helpful AI assistant."),
    ('placeholder', '{chat_history}'), # This will hold past Human/AI messages
    ('human', '{current_user_input}')  # The user's latest message
])

Your application logic would then be responsible for maintaining a chat_history list, appending the new user input and the AI's response after each turn, and passing the updated list back into the prompt for the next turn.

Key Takeaway: Managing and re-injecting conversation history using placeholder messages is the fundamental mechanism for building chatbots that can hold coherent, multi-turn conversations.

The Modern Prompting Lifecycle: From Draft to Production

The techniques above are the building blocks. However, modern LLM development requires more than just writing a good initial prompt. It involves a full lifecycle of testing, evaluation, and deployment.

1. Observability and Evaluation with LangSmith

Effective prompt engineering is an iterative process. How do you know if your new system message or few-shot examples are actually improving performance? This is where LangSmith becomes essential. LangSmith is a platform for debugging, testing, evaluating, and monitoring your LLM applications. With LangSmith, you can: Trace Execution: See exactly how your prompts are being processed and what inputs are being sent to the LLM at each step. Create Datasets: Collect interesting or problematic examples from your traces to build evaluation datasets. Run Evaluations: A/B test different prompt versions against your datasets to objectively measure their impact on quality, tone, and correctness.

1. Building Stateful Agents with LangGraph

What happens when your task is too complex for a single prompt? You might need an "agent" that can use tools, reason in loops, and make decisions. LangGraph is an extension of LangChain for building stateful, multi-actor applications. While a deep dive is beyond this post, it's important to know that each node in a LangGraph cycle is often powered by the prompting techniques we've discussed. You might have one node with a prompt designed for planning, another for tool use, and a third for generating a final answer. LangGraph helps orchestrate these prompted steps into a robust, cyclical workflow.

1. Prompting for Advanced Tool Use

Modern LLMs are increasingly capable of "function calling" or "tool use," where they can decide to call external functions (like a calculator, a database query, or a search API). Your prompts are key to making this work reliably. This involves: 1. Providing clear descriptions of your tools. 1. Using system messages or few-shot examples to guide the LLM on when and how to use a specific tool.

LLM Powered Applications with BridgePhase

There are countless real world use cases for natural language based automation. At BridgePhase, we design, build, and deliver LLM powered applications to create exceptional user experiences, streamline natural language based workflows, and create efficiencies that enable organizations to do more with less. We’re experts at managing the full lifecycle for complex systems, purpose built applications, and the underlying infrastructure that enables intelligent automation for critical missions. As the state of the art for LLM powered applications evolves, our team serves as a trusted partner helping organizations navigate complexity to create innovative solutions while ensuring security, performance, and reliability.

Thoughts?

What prompting techniques have you found most useful in your LLM applications? Any tips or tricks you’d like to share? Connect with us on LinkedIn to start a conversation!