A chatbot is the picture people have of AI, so products grow a text box even when the user never wanted a conversation. They wanted a decision. Approve this, label that, route this request, hide this result. The budget for that decision is often about 100 milliseconds, and a wandering model is the wrong shape.
Name the decisionLink to this heading
A decisions API is a boring contract: input in, a small structured answer out, with a latency budget and a cost you can plot. It does not explain itself in paragraphs. That is a feature. The caller can branch. The logs can be counted. A human can sample the mistakes.
A rule engine belongs in the same design. If the rule is stable, do not pay a model to rediscover it on every request. Use the model where the input is messy and the rule would be a novel. Use the rule where the business has already decided.
Latency, money, and trustLink to this heading
A general chat model can classify things. It will also sometimes be slow, expensive, and confidently odd. For a path on the request that users feel, those three are product bugs, not research notes. Put the slow model off to the side, or behind a cache, or not on this path at all.
The question I want a team to ask is narrow. Does this moment need a conversation, or does it need a decision that returns before the user notices the wait? If it is the second one, build the decision. Leave the chatbot for the place where a person actually wanted to talk.