Using it with your LLM

Language models are good at reading between the lines and writing replies. They're not a good place for a rule-bound decision: they can be talked out of your rules, and their confidence isn't calibrated on your cases. Put the decision in your own model, and let the LLM do the talking.

Support bot

Your model picks the action; your LLM writes the reply around it. The bot can't promise a refund your rules don't allow, because the action is decided before the LLM writes a word.

res = decide(model="bank-support@latest", state={"message": msg, **account})
d = res["decision"]

if d["route"] == "ask":
    reply = llm(f"Tell the customer a person will pick this up shortly. Message: {msg}")
    hand_to_person(res["id"])
else:
    reply = llm(
        f"You are a bank's support agent. The next step is decided: {d['action']}. "
        f"Explain it to the customer in two sentences. Don't offer anything else. Message: {msg}"
    )

Guardrails

Before an action an LLM proposes is carried out, check it against your model. Act only if no rule blocks it and your model agrees:

res = decide(model="refunds@latest", state=case)
d = res["decision"]
allowed = llm_action not in d["blocked_actions"] and d["action"] == llm_action
if not allowed:
    log("blocked", llm_action, d["blocked_by_rules"])   # rule ids, in words you wrote

Agent decision layer

Give your agent one tool, decide, for the choices that must follow your rules. It gets calibrated answers and a route instead of guessing.

{
  "name": "decide",
  "description": "Decide the next action for a customer case. Follows the company's rules. Returns the action, a confidence and a route: act, review or ask.",
  "input_schema": {
    "type": "object",
    "properties": { "state": { "type": "object", "description": "The case: message and account fields." } },
    "required": ["state"]
  }
}

Tell the agent: on act, go ahead; on review, go ahead and flag it; on ask, stop and hand over.

Your LLM as the fuzzy judge

Some rules depend on things like tone or urgency. Ask your LLM (or Jev) for that judgment and pass it in as a state field; your model makes the rule-bound decision. Or ask it as a question: a question your model hasn't learned is answered by your day-one backend, and graduates once it has outcomes.

Your LLM answering the unsure queue

The ask cases can go to a stronger model instead of a person. Post its answer to /v1/domains/{domain}/answers, and the next version learns from it.