POST /v1/query/stream, Server-Sent Events reference

Written by Joseph Russo, Principal Architect, Networks and Tech, AskVault · Reviewed by Allison Chan

Last updated: June 18, 2026 · 5 min read

TL;DR

Stream a chat response token by token over Server-Sent Events, so the first words show before the full answer is ready. Same request body as POST /v1/query. Uses your plan's message allowance; the API needs Starter or above.

Endpoint

POST https://api.askvault.co/v1/query/stream

Same request schema as POST /v1/query. The response is a Server-Sent Events stream instead of synchronous JSON.

Authentication: Authorization: Bearer ak_xxx. See authentication.

When to use streaming

Two cases where streaming wins over synchronous:

1. Live Chat UI. The customer sees the bot "typing" word by word. Perceived latency drops dramatically even though total latency is the same.
2. Long answers. Streaming sends tokens as they're generated, so a client with a short timeout still sees progress instead of waiting on one response, with no single boundary to hit.

For backend automation (a nightly job processing emails), synchronous is simpler. For any user-facing real-time UI, streaming.

Minimal example

curl

curl -N -X POST https://api.askvault.co/v1/query/stream \
-H "Authorization: Bearer ak_xxx" \
-H "Content-Type: application/json" \
-d '{"query": "How does pricing work?"}'

Python

import os, requests

with requests.post(
  "https://api.askvault.co/v1/query/stream",
  headers={"Authorization": f"Bearer {os.environ['ASKVAULT_API_KEY']}"},
  json={"query": "How does pricing work?"},
  stream=True,
  timeout=60,
) as r:
  for line in r.iter_lines():
      if line and line.startswith(b"data: "):
          print(line[6:].decode())

Node.js

const response = await fetch("https://api.askvault.co/v1/query/stream", {
method: "POST",
headers: {
  "Authorization": `Bearer ${process.env.ASKVAULT_API_KEY}`,
  "Content-Type": "application/json",
},
body: JSON.stringify({ query: "How does pricing work?" }),
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split("\n");
buffer = lines.pop();
for (const line of lines) {
  if (line.startsWith("data: ")) {
    const event = JSON.parse(line.slice(6));
    if (event.error) { console.error("\nError:", event.error); continue; }
    if (event.suppressed) continue; // a person is handling this conversation
    if (!event.done) process.stdout.write(event.token || "");
    else console.log("\nSources:", (event.sources ?? []).map(s => s.document_name));
  }
}
}

The curl -N flag disables output buffering so you see tokens as they arrive.

Event format

Every data: line is a JSON object. Read the done boolean to distinguish token events from the final event.

Token events (done: false)

Partial answer text. Many of these arrive as the LLM generates.

{"token": "To ", "done": false}

Concatenate token values in order to assemble the full answer.

Status events

Before tokens arrive, the server emits status events describing what the pipeline is doing. These are for UI display (animated progress indicator) and can be ignored in backend consumers.

{"event": "status", "stage": "understanding", "text": "Understanding your question…"}

Stages: understanding, search, read, write.

Final event (done: true)

Marks the end of the answer and, for a normal answer, carries the full sources array. Error frames and suppressed frames (sent when a person is handling the conversation) also have done: true but no sources, so read sources ?? [].

{
  "token": "",
  "done": true,
  "sources": [
    {
      "document_name": "Refund policy",
      "chunk_text": "Refunds are available within 30 days...",
      "url": "https://acme.co/policies/refunds",
      "relevance_score": 0.94
    }
  ],
  "tokens_used": 187,
  "conversation_id": "conv_xxx",
  "message_id": "msg_xxx"
}

A few extra frames, such as {"event": "chips"} with suggested follow-up questions, can arrive after done. Keep reading until the stream closes.

Error events

Stream-level error. Always has done: true.

{"error": "Upstream LLM provider returned 503", "done": true}

Retry with backoff for infrastructure errors; don't retry for workspace-not-found or invalid-key errors.

Latency profile

Events arrive in this order:

1. The request is validated and retrieval runs.
2. status events say what stage the answer is at.
3. token events stream the answer as it is written.
4. A done: true frame carries the sources.
5. Optional frames such as suggested follow-up chips may follow.

Total time depends on the model your workspace uses, the size of your knowledge base and the length of the answer. Streaming matters because the visitor sees the first words as soon as they are written instead of waiting for the whole reply.

Request parameters

Same as POST /v1/query: query (required), top_k, session_id, conversation_id. The workspace is identified by the API key, so there's no workspace_id in the body.

Parsing the stream

Server-Sent Events format:

data: {"token":"Hello","done":false}

data: {"token":" world","done":false}

data: {"token":"","done":true,"sources":[...],"tokens_used":187,...}

Each event has:

• A data: <json> line with the event payload.
• A blank line as the terminator.

Use a fetch-based SSE reader: the browser's built-in EventSource only makes GET requests without custom headers, so it cannot call this POST endpoint with your API key. If you're writing a parser by hand, split on \n\n to find event boundaries, then on \n to find lines within an event. Check event.done rather than event.type to detect stream completion.

Error recovery

For 5xx errors mid-stream:

1. Close the stream connection.
2. Wait 2^attempt * 1 seconds (exponential backoff).
3. Retry the same request with the same session_id to resume context. (Resuming by conversation_id alone is rolling out; an error event does not carry a conversation_id.)
4. Max 5 attempts.

For client-side errors (network drop, browser tab backgrounded): don't auto-retry; show the user the partial response and a "regenerate" button.

Cancellation

To stop reading a stream, close the connection on the client.

How a cancelled stream is counted against your limits is not something to rely on either way; plan as if it counts.

Limits

• Plan availability. Same as synchronous: Starter through Enterprise. The REST API (streaming and non-streaming) requires a Starter plan or above; Free doesn't include API access.
• Rate limits. Per the rate limits page. Streaming queries count the same as synchronous.
• Concurrent streams per key. Not separately capped: governed by the same per-minute/per-day request limits as any other call. See rate limits to raise a key's ceiling.

Common pitfalls

Connection hangs after the first event. Output buffering in your HTTP Client. Disable it (curl -N, requests.iter_lines(decode_unicode=False), etc.).

Some tokens missing. SSE parsers occasionally drop events when chunks span TCP Packet boundaries. Use a battle-tested SSE library, not a hand-rolled parser.

Stream feels slow. Answer time depends mostly on the workspace's model and the answer length. Every query uses your limits, so do not send warm-up queries.

Done event never arrives. Connection dropped. Treat the partial response as final and show a "regenerate" option.

FAQ

Does streaming cost more than synchronous?

No. Same per-query cost.

Can I use streaming in a browser?

Yes, but your API key would be exposed. Proxy through your backend or use the widget channel which authenticates with a public workspace token.

How do I show "typing" indicators in my UI?

Show a typing animation as soon as you send the request. Hide it when the first token event arrives and start appending tokens to the bubble.

Can I get the source citations before tokens arrive?

No. Sources arrive in the final done event after all tokens. If you need to render citations progressively, parse the answer text for inline markers and resolve them against the sources in the done event.

Does cancellation save quota?

Do not count on it. Plan as if an accepted request counts against your limits whether or not you read it to the end.

Related guides

• POST /v1/query reference
• API authentication
• Rate limits per plan
• Error codes
• How to embed an AI chatbot on a React website

Start building with the AskVault API

Generate an API key in the dashboard, then send your first request.

Get an API key → Read the API docs

Was this page helpful? Yes No