Challenges Faced by Llms in Multi-Turn Conversations

Ever noticed that your AI starts strong, but after a few back-and-forths, it spirals into nonsense? It turns out that it’s not your imagination; it’s science. New research from Microsoft + Salesforce tested 15 leading LLMs (including #GPT-4, #Claude, #Gemini) across multi-turn tasks. 𝐓𝐡𝐞 𝐫𝐞𝐬𝐮𝐥𝐭𝐬?  1. Performance dropped by an average of 39%. 2. Same task. Same info. Just given step by step instead of all at once. 3. Every single model got worse. 𝐇𝐞𝐫𝐞’𝐬 𝐰𝐡𝐲 𝐋𝐋𝐌𝐬 𝐠𝐞𝐭 𝐥𝐨𝐬𝐭 𝐢𝐧 𝐜𝐨𝐧𝐯𝐞𝐫𝐬𝐚𝐭𝐢𝐨𝐧: -> Premature answers → They guess before they have full context. -> Answer bloat → Responses get longer, carrying over flawed logic. -> Loss of middle context → They remember the start and end, but forget what’s in between. -> Verbal drift → More words = more assumptions = more confusion. The scary part isn’t just the decline.  𝐈𝐭’𝐬 𝐭𝐡𝐞 𝐮𝐧𝐫𝐞𝐥𝐢𝐚𝐛𝐢𝐥𝐢𝐭𝐲: It’s the unreliability: – In single-turn tasks, results are fairly consistent. – In multi-turn, the same prompt can succeed one time and fail the next. 𝐓𝐡𝐢𝐬 𝐞𝐱𝐩𝐥𝐚𝐢𝐧𝐬 𝐰𝐡𝐲: -> Your AI-generated UI starts strong but drifts into chaos. -> Conversations often end with users restarting from scratch. -> Temperature settings don’t fix the problem — it’s deeper than randomness. 𝐖𝐡𝐚𝐭 𝐜𝐚𝐧 𝐰𝐞 𝐝𝐨 (𝐟𝐨𝐫 𝐧𝐨𝐰)? – Give more context upfront instead of “drip-feeding” instructions. – Reset conversations when quality drops. – Use summaries to re-establish shared context. 𝐓𝐡𝐞 𝐛𝐢𝐠 𝐭𝐚𝐤𝐞𝐚𝐰𝐚𝐲: The next frontier of AI isn’t just “smarter models.” It’s models that can stay coherent and consistent across extended interactions.

🧠 LLMs still get lost in conversation. You should pay attention to this, specially when building AI Agents! A new paper just dropped, and it uncovers something many of us suspected: LLMs perform way worse when instructions are revealed gradually in multi-turn conversations. 💬 While LLMs excel when you give them everything up front (single-turn), performance drops by an average of 39% when the same task is spread across several conversational turns. Even GPT-4 and Gemini 2.5 stumble. Why? Because in multi-turn chats, models: ❌ Make premature assumptions ❌ Try to “wrap up” too soon ❌ Get stuck on their own past mistakes ❌ Struggle to recover when they go off-track The authors call this the “𝗟𝗼𝘀𝘁 𝗶𝗻 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻” effect, and it explains why LLMs sometimes seem great in demos, but frustrating in real-world use. 🔍 If you’re building agentic AI products, this is a wake-up call. Most evaluation benchmarks don’t reflect how users actually interact with messy, evolving, often underspecified prompts. 📄 Paper link in comments.

🧠 If you're building apps with LLMs, this paper is a must-read. Researchers at Microsoft and Salesforce recently released LLMs Get Lost in Multi-Turn Conversation — and the findings resonate with our experience at Vellum. They ran 200,000+ simulations across 15 top models, comparing performance on the same task in two modes: - Single-turn (user provides a well-specified prompt upfront) - Multi-turn (user reveals task requirements gradually — like real users do) The result? ✅ 90% avg accuracy in single-turn 💬 65% avg accuracy in multi-turn 🔻 -39% performance drop across the board 😬 Unreliability more than doubled Even the best models get lost when the task unfolds over multiple messages. They latch onto early assumptions, generate bloated answers, and fail to adapt when more info arrives. For application builders, this changes how we think about evaluation and reliability: - One-shot prompt benchmarks ≠ user reality - Multi-turn behavior needs to be a first-class test case - Agents and wrappers won’t fix everything — the underlying model still gets confused This paper validates something we've seen in the wild: the moment users interact conversationally, reliability tanks — unless you're deliberate about managing context, fallback strategies, and prompt structure. 📌 If you’re building on LLMs, read this. Test differently. Optimize for the real-world path, not the happy path.

LLMs Get Lost in Multi-Turn Conversations: New Research Reveals Major Reliability Gap Just read a fascinating new paper from Microsoft and Salesforce Research revealing a critical flaw in today's LLMs: they dramatically underperform in multi-turn conversations compared to single-turn interactions. 📊 Key findings: 🔗 LLMs suffer an average 39% performance drop in multi-turn settings across six generation tasks 🔗 This occurs even in conversations with as few as two turns 🔗 The problem affects ALL tested models, including the most advanced ones (Claude 3.7, GPT-4.1, Gemini 2.5) 🔍 The researchers call this the "lost in conversation" phenomenon - when LLMs take a wrong turn in conversation, they get lost and don't recover. This is caused by: 🔗 Making assumptions too early 🔗 Prematurely generating final solutions 🔗 Relying too heavily on previous (incorrect) answers 🔗 Producing overly verbose responses 💬 Why conversation-level evaluation matters: Traditional LLM benchmarks focus on single-turn performance, creating a dangerous blind spot. Real-world AI interactions are conversational by nature, and this research shows that even the most capable models struggle with maintaining context and adapting to new information over multiple turns. Without robust conversation-level evaluation, we risk deploying systems that perform brilliantly in lab tests but frustrate users in practice. 🔎 At Coval, this is exactly what we focus on: evaluating LLMs in realistic conversational scenarios rather than isolated prompts. By measuring how models handle the natural flow of information across turns, we can identify reliability issues before they impact users and guide development toward truly conversational AI. This research highlights a critical gap between how we evaluate LLMs (single-turn) versus how we use them in practice (multi-turn). As we build AI assistants and agents, addressing this reliability issue becomes essential.

Oof. Salesforce tested the world’s top Agents on real enterprise CRM tasks - and most of them flopped. “Even leading LLM agents achieve modest overall success…around 58% in single-turn scenarios, degrading to approximately 35% in multi-turn settings.” “All evaluated models demonstrate near-zero confidentiality awareness.” This is Salesforce’s own research - testing Agents inside real CRM sandbox environments using Sales Cloud, Service Cloud, and CPQ data. Tasks like: - Approve a quote. - Route a lead. - Extract insights from a sales call. - Enforce policy compliance. To be honest? The results are brutal: 𝐀𝐠𝐞𝐧𝐭𝐬 𝐟𝐚𝐢𝐥 𝐚𝐭 𝐜𝐥𝐚𝐫𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧. They don’t ask “what do you mean?” - they guess. 𝐓𝐡𝐞𝐲 𝐥𝐞𝐚𝐤 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐭𝐢𝐚𝐥 𝐝𝐚𝐭𝐚. Unless prompted not to, they’ll share private info. Add guardrails, and they get safer but dumber. 𝐓𝐡𝐞𝐲’𝐫𝐞 𝐠𝐫𝐞𝐚𝐭 𝐚𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬, 𝐛𝐚𝐝 𝐚𝐭 𝐫𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠. Structured tasks like case routing? 80%+ success. Textual reasoning like summarising a call? <35%. 𝐌𝐮𝐥𝐭𝐢-𝐭𝐮𝐫𝐧 = 𝐦𝐨𝐫𝐞 𝐟𝐚𝐢𝐥𝐮𝐫𝐞, 𝐧𝐨𝐭 𝐦𝐨𝐫𝐞 𝐢𝐧𝐬𝐢𝐠𝐡𝐭. Success drops from 58% to 35% when the Agent has to ask follow-ups. Most don’t. ++++++++++++ The great thing about honest benchmarking papers is they give us a “dragons be here” map and blueprint for what breaks and what works. Strategic Implications for Teams: Building Agents That Actually Work 𝐒𝐭𝐚𝐫𝐭 𝐒𝐦𝐚𝐥𝐥, 𝐒𝐭𝐚𝐫𝐭 𝐒𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞𝐝 Workflow first. Reasoning later. LLMs shine on defined, rule-based tasks. Route a lead? Yes. Mine insights from messy calls? Not yet. 𝐂𝐥𝐚𝐫𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧 ≠ 𝐈𝐧𝐭𝐮𝐢𝐭𝐢𝐨𝐧 Most Agents still don’t ask - they assume. Without scaffolds for gathering missing info, you’re automating risk. 𝐂𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐭𝐢𝐚𝐥𝐢𝐭𝐲 𝐈𝐬 𝐍𝐨𝐭 𝐚 𝐆𝐢𝐯𝐞𝐧 “All models showed near-zero confidentiality awareness.” Train it in or risk a breach. Safer prompts often reduce performance. That’s the alignment tradeoff. 𝐌𝐮𝐥𝐭𝐢-𝐭𝐮𝐫𝐧 𝐈𝐬𝐧’𝐭 𝐅𝐫𝐞𝐞 More turns = more places to fail. If the Agent can’t manage context and clarify precisely, every extra step is just compounding confusion. 𝐑𝐞𝐚𝐥𝐢𝐬𝐦 > 𝐃𝐞𝐦𝐨𝐬 Salesforce used 25 real CRM objects and 4K+ test cases. If your Agent only works on clean data, it’s not ready. Real mess is where things break and where real value lives. 𝐂𝐨𝐬𝐭-𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐈𝐬𝐧’𝐭 𝐋𝐢𝐧𝐞𝐚𝐫 The best model isn’t always the biggest or the priciest. Measure value per action, not token count. Cheap and dumb is still expensive downstream. 𝐀𝐠𝐞𝐧𝐭 ≠ 𝐏𝐫𝐨𝐝𝐮𝐜𝐭 A chatbot calling APIs isn’t a product. You need ownership, fallback logic, escalation paths. Otherwise, you’ve shipped a cool demo - not a working system. If your team is building or buying AI agents and you want expert help making sure they actually deliver - let’s talk. H/T Pradeep Sanyal

The biggest limitation in today’s AI agents is not their fluency. It is memory. Most LLM-based systems forget what happened in the last session, cannot improve over time, and fail to reason across multiple steps. This makes them unreliable in real workflows. They respond well in the moment but do not build lasting context, retain task history, or learn from repeated use. A recent paper, “Rethinking Memory in AI,” introduces four categories of memory, each tied to specific operations AI agents need to perform reliably: 𝗟𝗼𝗻𝗴-𝘁𝗲𝗿𝗺 𝗺𝗲𝗺𝗼𝗿𝘆 focuses on building persistent knowledge. This includes consolidation of recent interactions into summaries, indexing for efficient access, updating older content when facts change, and forgetting irrelevant or outdated data. These operations allow agents to evolve with users, retain institutional knowledge, and maintain coherence across long timelines. 𝗟𝗼𝗻𝗴-𝗰𝗼𝗻𝘁𝗲𝘅𝘁 𝗺𝗲𝗺𝗼𝗿𝘆 refers to techniques that help models manage large context windows during inference. These include pruning attention key-value caches, selecting which past tokens to retain, and compressing history so that models can focus on what matters. These strategies are essential for agents handling extended documents or multi-turn dialogues. 𝗣𝗮𝗿𝗮𝗺𝗲𝘁𝗿𝗶𝗰 𝗺𝗼𝗱𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 addresses how knowledge inside a model’s weights can be edited, updated, or removed. This includes fine-grained editing methods, adapter tuning, meta-learning, and unlearning. In continual learning, agents must integrate new knowledge without forgetting old capabilities. These capabilities allow models to adapt quickly without full retraining or versioning. 𝗠𝘂𝗹𝘁𝗶-𝘀𝗼𝘂𝗿𝗰𝗲 𝗺𝗲𝗺𝗼𝗿𝘆 focuses on how agents coordinate knowledge across formats and systems. It includes reasoning over multiple documents, merging structured and unstructured data, and aligning information across modalities like text and images. This is especially relevant in enterprise settings, where context is fragmented across tools and sources. Looking ahead, the future of memory in AI will focus on: • 𝗦𝗽𝗮𝘁𝗶𝗼-𝘁𝗲𝗺𝗽𝗼𝗿𝗮𝗹 𝗺𝗲𝗺𝗼𝗿𝘆: Agents will track when and where information was learned to reason more accurately and manage relevance over time. • 𝗨𝗻𝗶𝗳𝗶𝗲𝗱 𝗺𝗲𝗺𝗼𝗿𝘆: Parametric (in-model) and non-parametric (external) memory will be integrated, allowing agents to fluidly switch between what they “know” and what they retrieve. • 𝗟𝗶𝗳𝗲𝗹𝗼𝗻𝗴 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴: Agents will be expected to learn continuously from interaction without retraining, while avoiding catastrophic forgetting. • 𝗠𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝗺𝗲𝗺𝗼𝗿𝘆: In environments with multiple agents, memory will need to be sharable, consistent, and dynamically synchronized across agents. Memory is not just infrastructure. It defines how your agents reason, adapt, and persist!

LLMs Get Lost in Multi-turn Conversation The cat is out of the bag. Pay attention, devs. This is one of the most common issues when building with LLMs today. Glad there is now paper to share insights. Here are my notes: The paper investigates how LLMs perform in realistic, multi-turn conversational settings where user instructions are often underspecified and clarified over several turns. I keep telling devs to spend time preparing those initial instructions. Prompt engineering is important. The authors conduct large-scale simulations across 15 top LLMs (including GPT-4.1, Gemini 2.5 Pro, Claude 3.7 Sonnet, DeepSeek-R1, and others) over six generation tasks (code, math, SQL, API calls, data-to-text, and document summarization). Severe Performance Drop in Multi-Turn Settings All tested LLMs show significantly worse performance in multi-turn, underspecified conversations compared to single-turn, fully-specified instructions. The average performance drop is 39% across six tasks, even for SoTA models. For example, models with >90% accuracy in single-turn settings often drop to ~60% in multi-turn settings. Degradation Is Due to Unreliability, Not Just Aptitude The performance loss decomposes into a modest decrease in best-case capability (aptitude, -15%) and a dramatic increase in unreliability (+112%). In multi-turn settings, the gap between the best and worst response widens substantially, meaning LLMs become much less consistent and predictable. High-performing models in single-turn settings are just as unreliable as smaller models in multi-turn dialogues. Don't ignore testing and evaluating in multi-turn settings. Main reasons LLMs get "lost" - Make premature and often incorrect assumptions early in the conversation. - Attempt full solutions before having all necessary information, leading to “bloated” or off-target answers. - Over-rely on their previous (possibly incorrect) answers, compounding errors as the conversation progresses. - Produce overly verbose outputs, which can further muddle context and confuse subsequent turns. - Pay disproportionate attention to the first and last turns, neglecting information revealed in the middle turns (“loss-in-the-middle” effect). Practical Recommendations: - Users are better off consolidating all requirements into a single prompt rather than clarifying over multiple turns. - If a conversation goes off-track, starting a new session with a consolidated summary leads to better outcomes. - System builders and model developers are urged to prioritize reliability in multi-turn contexts, not just raw capability. This is especially true if you are building complex agentic systems where the impact of these issues is more prevalent. - LLMs are really weird. And all this weirdness is creeping up into the latest models too but it more subtle ways. Be careful out there, devs.

New study from Microsoft finds LLM accuracy to drop very quickly. LLMs ace single-shot prompts - but get lost in two-turn conversations. Whether you’re asking an AI to write code or summarize documents, you’ll naturally refine instructions over several messages. Yet benchmarks still focus on one-and-done prompts. The paper from Microsoft & Salesforce introduces a “sharded conversation” simulator that feeds one requirement per turn across six generation tasks - from coding and math to SQL, API calls, table captions and summaries - and tests 15 leading models. By measuring best-case “aptitude” and stability “reliability,” the authors show an average 39% performance drop driven by doubled unreliability. Even agent-style recaps or deterministic decoding recoup only a fraction of the loss. By spotlighting multi-turn fragility, they highlight a new path for LLM evaluation: conversation consistency - not just raw accuracy. ↓ 𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡