Request a demo

Feature

Triggers

Triggers let an agent speak up on its own when something happens, instead of waiting to be asked. For example:

  • a museum guide starts talking about the artwork a visitor stops at
  • a field coach points out a tool left in the wrong place
  • a build coach notices a skipped step
  • an agent checks in after a long silence

A trigger watches for one kind of moment: something the user said, something the camera saw, or time passing. It can ask the model to look at the camera frame, the conversation, and reference files, then decide whether the agent should say something. The agent says it in its own voice, without talking over the user.

Add a trigger

Ask Builder, for example: "When the visitor looks at an artwork, tell them one fact about it." Builder writes the trigger into the agent's files.

The Triggers section of the agent's configuration lists every trigger and marks the ones that need a fix. To see a trigger at work, run the agent: the session inspector shows when each trigger acted and why.

Technical

Trigger files

A trigger is a JavaScript module at data/triggers/<name>.trigger.js in the agent's folder. Every valid file there runs in the agent's sessions; there is no list to attach it to. The module exports schemaVersion, a metadata object with title and summary, interests, and evaluate(ctx). configSchema defaults, frameIntervalMs, timerIntervalMs, and timeoutMs are optional.

Events

interests names the events a trigger runs on:

  • text.turn: the user finished a turn; ctx.text.latestUserText holds it. In chat, a speak decision answers instead of the model. In voice, the turn ends when the agent starts its reply, so a speak decision follows that reply, or cuts in with interrupt. A decision is dropped once the user's next turn closes, when they talk over the agent, or after 30 seconds.
  • visual.proactive_candidate: recognition found objects the conversation has not just covered; ctx.visual.recognizedArtworks lists them.
  • visual.frame: a camera frame. Frames reach the server about once a second.
  • timer: a regular tick in a voice session, camera or not, for nudges after silence or periodic check-ins. A session ticks only if the agent has a timer trigger when the session starts.

How often it runs

A visual.frame trigger runs at most every frameIntervalMs (default 5000, from 1000 to 60000). A timer trigger runs every timerIntervalMs (default 60000, from 1000 to 3600000). Both count from the start of the last evaluation, and a trigger still busy when it is due again is skipped.

  export const interests = ["visual.frame"];
export const frameIntervalMs = 2000;

Each trigger keeps its own pace and runs one evaluation at a time per session, alongside the other triggers. A recognition event that arrives while it is busy is evaluated right after. A slow text.turn trigger delays the reply.

timeoutMs is the deadline for one evaluation: 10 seconds by default, from 1 to 60 seconds. Past it, the evaluation and its model call are dropped.

Asking the model

ctx.llm.generate asks the model about the moment. Pass includeCurrentFrame: true for the camera frame, messages for the conversation, data for anything else, and schema (a JSON Schema) to get a structured answer in json:

  const { json } = await ctx.llm.generate({
  system: "You check the build in the camera frame against the current step.",
  files: ["build-guide.md", `manual/step-${step}.png`],
  includeCurrentFrame: true,
  data: { currentStep: step },
  schema: {
    type: "object",
    properties: { shouldSpeak: { type: "boolean" }, about: { type: "string" } },
    required: ["shouldSpeak", "about"],
  },
});
  • files sends up to eight files from the agent's data/ folder, or the organization's files with a shared: prefix. Pictures go as images (scaled down); anything else must be UTF-8 text. Files go first, so files that stay the same between calls can be cached.
  • An evaluation may make up to three calls. The model thinks at a low level by default; thinking: "high" thinks longer.

What the trigger can read

  • ctx.conversation.recentTurns: the last twelve messages.
  • ctx.conversation.recentToolCalls: the agent's last twelve tool calls, each with toolId and arguments. For example, a trigger can read the step from the manual page the agent opened last.
  • A trigger keeps no state between evaluations. Cooldowns are the only thing that lasts.

Decisions

evaluate returns a list of decisions, or [] to stay quiet:

  • { type: "speak", message } says the message as written. { type: "speak", instruction } asks the agent to say or do something in its own words. If both are set, voice uses the instruction and chat uses the message.
  • priority orders decisions across triggers (lower first, default 100). The first speak decision is delivered.
  • cooldowns: [{ key, cooldownMs }] drops later decisions with the same key from the same trigger until the time passes. ctx.cooldowns.has(key) and ctx.cooldowns.set(key, ms) work on the same keys.
  • interrupt: true lets a speak decision cut the agent off (voice only). Otherwise a decision waits until the agent and user are quiet, and is dropped if that does not happen. No decision ever talks over the user or a running tool call.
  • inspector: { title, summary, fields } shows in the session inspector why the trigger acted. { type: "none", inspector } only updates the inspector.

The old text, dedupeKey, and top-level cooldownMs fields are rejected, and the inspector shows which field to use instead.

Where triggers run

Each agent's triggers run in one sandbox process, away from the rest of the server. In production it has no network, a read-only system, and an empty environment. Each evaluation gets a fresh context.

Older agents

Agents saved before triggers became files had a triggerRefs list. It is ignored now: every trigger file in the agent's folder runs, including ones that were switched off, and per-agent config overrides give way to the file's configSchema defaults. Before deploying such an agent, delete the trigger files that should not run and move any override into the file.