Spatial Intelligence Engine

The Spatial Intelligence Engine (SIE) is the part of WorldAgents that understands places. WorldAgents builds and runs agents; SIE tells them about the places they work in.

What SIE does

• Spaces: keeps 3D versions of real places, with their semantic layer of rooms, zones, objects, landmarks, and routes. See Spaces.
• Recognition: looks at camera frames and says which known objects are in view, such as an artwork or a machine.
• Localization: works out where the visitor is in the space.
• Simulations: renders spaces in 3D so you can test agents before field use. See Simulations.

Agents never handle raw 3D files. They get short, current facts instead: which room is active, what was recognized, what is nearby, and how to get somewhere.

Why it matters

One space can serve many agents. A store, museum floor, worksite, or training area mapped once can support a guide, an audit agent, and a training agent.

Spatial context also keeps agents grounded. Instead of guessing from pixels, an agent knows the name of the object in front of the visitor and what the team wrote about it.

Seeing what it found

The session inspector shows what recognition and localization found, with confidence and debug pictures. That detail is for reviewing a session; the agent itself gets only the short version.

Technical

  camera, glasses, simulator, or uploaded frame
  -> WorldAgents session
  -> recognition and localization (optional)
  -> short spatial context
  -> agent reply or tool call

• Recognition runs on sampled frames, alongside the conversation. The session keeps talking while spatial context updates.
• The latest useful result is stored on the session and passed to the agent in short form. Fuller evidence goes to the inspector and logs.
• Spaces hold splats (3D scans), 3D models, and the semantic layer. Agents reach them through space tools, and assistants through space_read and space_write.