Query decontextualization for multi-turn chat
The problem
Follow-up questions break naive vector search because they lean on words spoken earlier in the chat. A user opens your chat widget and types two messages:
Turn 1: "What's your pricing?"Bot: "Starter is $39/mo, Growth $99/mo..."Turn 2: "Does it include WhatsApp?"The vector embedding of the second message has no signal about what "it" refers to. Searching the vector index for "Does it include WhatsApp?" retrieves chunks about WhatsApp generally (number setup, message templates, API docs), not chunks about your pricing plans.
Result: the LLM answers correctly only by luck (because the conversation history is in its context), or it cites the wrong page, or it says "I don't see WhatsApp in our pricing" because the relevant pricing chunk never made it into the top-K.
This is the single largest accuracy bug in naive RAG pipelines.
How AskVault fixes it
AskVault rewrites the user's message into a standalone query before the vector search runs, using the last few turns as context. See how vector databases work for background on what that search step is actually doing.
Original: "does it include WhatsApp?"Decontextualized: "Does AskVault's Starter Plan pricing include WhatsApp?"The rewritten query is what hits the vector index. The pricing chunks come back. The agent answers correctly with the right source citation, whether the visitor is comparing the Starter Plan, the Growth Plan, the Business Plan, or the custom-priced Enterprise Plan.
When the rewrite happens (and when it skips)
| Condition | Rewriter behavior |
|---|---|
| First turn of a conversation | Skipped; message is already standalone |
| Message is a greeting/thanks ("hi", "ok", "thanks") | Skipped |
| Message is one or two words | Skipped; no referential signal to resolve |
| Multi-turn follow-up with prior history | Rewritten into a standalone question |
It runs automatically on every workspace; there is nothing to configure.
What this means for you
The rewrite is a quick extra step that is separate from your chosen chat model. This means:
- The rewrite is not metered against your monthly message allowance.
- It adds a small delay on follow-up turns only.
- If the rewrite can't be done for any reason, the original question is used unchanged. The chat turn never fails because of decontextualization.
What you'll see in your data
Retrieval precision on follow-up turns rises when a workspace has active multi-turn conversations. The size of the gain depends on how your knowledge base is shaped. Along with that:
- Source citation accuracy improves: the agent attributes answers to the page that actually contained the answer instead of an adjacent page that happened to be lexically close. This ties into how reranking orders the retrieved chunks before the answer is written.
- Latency increases slightly on follow-up turns only. First turns are unaffected, since there's no history to fold in.
Configuration
There is nothing to configure. Query decontextualization runs automatically on every workspace, and is skipped on first turns, greetings, and very short messages (see the table above). It folds in a small window of recent turns as context, tuned so it resolves references like "it" and "that plan" without over-compressing the conversation. It runs the same way on the Free plan (which has no time limit) and on paid plans. The rewrite itself is not charged as a message. Only the answer counts against your plan's monthly message allowance, which the pricing plan comparison lists by plan.
Why this matters as chat gets more conversational
As chat surfaces shift from one-shot Q&A to genuine multi-turn dialogue, the percentage of queries that are follow-ups climbs. A pipeline that handles first questions well but stumbles on follow-ups feels broken to users.
Query decontextualization is one of the low-cost RAG fixes that a serious chat system runs by default. AskVault has it on for every plan, including the Free plan. On the API, send the same session_id on every turn of one user's conversation in the query request so AskVault has the prior turns to fold into the rewrite.