Classificar mensagens com respostas em JSON

Tradução automática

Esta página foi traduzida automaticamente do inglês e ainda não foi revisada por um falante nativo; correções são bem-vindas no GitHub. Se algo não bater, vale o original em inglês. O código e os resultados são os mesmos do original, importados dos arquivos testados.

Tarefa: arquivar cada mensagem de suporte numa categoria, decidida pelo modelo, numa tabela.

# Recipe: the model reads free text and answers with fields you store.
paths:
  components: ./components
  migrations: ./migrations

datasources:
  db:
    driver: sqlite
    database: ./data/helpdesk.db

llm:
  model: phi3

CREATE TABLE tickets (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    message TEXT NOT NULL,
    category TEXT NOT NULL CHECK (category IN ('billing', 'shipping', 'other')),
    urgent INTEGER NOT NULL DEFAULT 0
);

responseFormat="json" pede JSON ao modelo e o interpreta: ticket é um objeto. Os campos dele são uma entrada como qualquer outra — a página verifica a categoria antes de guardá-la, e as próprias regras dos q:param da ação rodam antes de o modelo sequer ser chamado.

<q:component name="Helpdesk">
  <q:action name="open" method="POST">
    <q:param name="message" required="true" minlength="10" />

    <!-- responseFormat="json": the model is asked for JSON, and the value is
         the parsed object — ticket.category, ticket.urgent. -->
    <q:llm name="ticket" responseFormat="json" temperature="0">
      <q:prompt>Classify this customer message. Answer with JSON only:
{"category": "billing" or "shipping" or "other", "urgent": true or false}
Message: {message}</q:prompt>
    </q:llm>

    <!-- A model's answer is input like any other: a category it made up is
         filed as "other" (the table's CHECK would refuse it). -->
    <q:set name="category" value="{ticket.category if ticket.category in ['billing', 'shipping'] else 'other'}" />

    <q:query name="saved" datasource="db">
      INSERT INTO tickets (message, category, urgent) VALUES (:message, :category, :urgent)
      <q:param name="message" value="{message}" type="string" />
      <q:param name="category" value="{category}" type="string" />
      <q:param name="urgent" value="{ticket.urgent == true}" type="boolean" />
    </q:query>
    <q:redirect url="/" flash="Filed under {category}." />
  </q:action>

  <q:query name="tickets" datasource="db">
    SELECT category, urgent, message FROM tickets ORDER BY id DESC
  </q:query>

  <ui:window title="Helpdesk">
    <q:if condition="flash">
      <ui:alert variant="success">{flash}</ui:alert>
    </q:if>
    <ui:form on-submit="open">
      <ui:input bind="message" rows="3" placeholder="How can we help?" />
      <ui:button variant="primary">Send</ui:button>
    </ui:form>
    <ui:table source="{tickets}" />
  </ui:window>
</q:component>

<!-- Structural checks, never the model's exact words: the same tests run
     against a real model before every release. -->
<q:test name="a message is filed with the model's fields" page="/">
  <test:submit action="open" message="I was charged twice for order 1042, please refund one." />
  <test:expect redirect="/" />
  <test:expect table="tickets" count="1" where="category = 'billing'" />
</q:test>

<q:test name="a message too short never reaches the model" page="/">
  <test:submit action="open" message="help" />
  <test:expect error="message" />
  <test:expect table="tickets" count="0" />
</q:test>

<q:test name="a category the model made up is filed as other" page="/">
  <test:submit action="open" message="Please write me a poem about the sea." />
  <test:expect redirect="/" flash="Filed under other." />
  <test:expect table="tickets" count="1" where="category = 'other'" />
</q:test>

[
  {"when": "charged twice", "replies": ["{\"category\": \"billing\", \"urgent\": true}"]},
  {"when": "a poem about the sea", "replies": ["{\"category\": \"poetry\", \"urgent\": false}"]}
]

tests/tickets.test.q
  PASS  a message is filed with the model's fields
  PASS  a message too short never reaches the model
  PASS  a category the model made up is filed as other
3 passed, 0 failed

Testado: no CI estes testes rodam contra um servidor de modelos substituto, que responde a partir da primeira fonte que recebe; antes de cada versão eles rodam contra um modelo de verdade (tests/live_ai/test_cookbook_ai.py). Por isso eles conferem a estrutura — qual fonte, qual ferramenta, o que a página mostra quando algo falha — e nunca as palavras do modelo.

Veja IA-1.