Compare models side by side
When to use it
Comparing on your own content beats reading model benchmarks:
- Picking a model. Try your most common questions before you commit.
- Checking a cheaper model. See whether a lighter model still answers your questions well.
- Spot-checking after a change. Re-run a few questions after you edit your content or prompt.
How to use it
- Open the chat playground.
- Choose Compare Models.
- Pick 2 to 4 models from those your plan can use.
- Type a question and run it.
Each model answers from the same retrieved content, with the same temperature and strictness settings you chose in the playground.
What you see
For each model:
- The answer.
- Response time, in milliseconds.
- Tokens used.
- An error, if that model failed, so one failure does not hide the others.
Weighing cost and quality
Models cost different numbers of message credits per answer; the model selector shows each one's cost. For a busy agent, choose the lightest model that answers your questions well. For a quiet one, you can afford a stronger model.
Limits
- Models per comparison. 2 to 4.
- Question length. Up to 4,000 characters.
- One question per run. There is no batch upload.
- Message credits. Each compared model is charged like a normal answer.
Common pitfalls
Deciding from one question. One answer proves little. Try a set of real customer questions, including awkward ones.
Reading too much into one response time. Times vary call to call. Run a question a few times before drawing conclusions.
FAQ
Does compare mode use my message allowance?
Yes. Each model that answers costs the same credits as a normal chat answer with that model.
Can I compare across workspaces?
No. Comparisons run within the current workspace.
Will my live agent change model?
Only if you change it in the model selector afterwards.