Skip to content

AI Control Protocol

Sektor's defining feature: an AI collaborator is a first-class client of the engine, not a bolted-on afterthought. It connects to the same local command boundary the editor uses, so anything you can author, an AI can author, and nothing it does is invisible or irreversible.

How it works

The engine exposes a local named pipe (sektor.control/v1): one UTF-8 JSON command in, one JSON response out. It is local-only by design. Over 100 validated operations cover projects, scenes, transforms, hierarchy, components, terrain, materials, prefabs, lights, navigation, animation, input injection, Play control, saves, packaging, and render captures.

{"op":"scene.snapshot"}
{"op":"scene.create","name":"Ashen Citadel","x":12,"y":0,"z":-8}
{"op":"terrain.sculpt","id":5,"worldCenter":[0,1.5,0],"radius":12,"strength":0.5,"falloff":0.65,"mode":"raise"}
{"op":"render.capture"}
{"op":"history.undo"}

The guarantees

Every mutation flows through the engine's single transaction boundary:

  • Validated. Malformed or illegal requests are rejected with structured errors before anything changes.
  • Undoable. Every AI edit is a normal entry in your history. Ctrl+Z works on AI work.
  • Attributable. Every outcome lands in the activity log with sequence, revision, and result.

The AI is never a privileged bypass. It cannot do anything you could not do, and it cannot do it silently.

Discovery instead of guessing

The engine publishes machine-readable registries, so tools stay in lockstep with reality:

  • inspect.capabilities reports what this engine build supports.
  • inspect.components returns the same registry Studio's Add Component browser uses: every component's fields, types, bounds, enum values, and mutation commands.
  • The Event operation and expression registries define the gameplay language itself.

A well-behaved client reads these first. When a new component ships, the AI discovers it the same morning you do.

Seeing the world

Structured state, not screen scraping:

  • scene.snapshot returns entities, components, hierarchy, derived world transforms, the asset catalog, diagnostics, and editor state in one coherent document.
  • scene.snapshot.wait holds until something actually changes (bounded timeout), so assistants react in tens of milliseconds without polling.
  • render.capture produces a revision-tagged image through the same final frame you see, when pixels genuinely matter.
  • During Play, runtime telemetry exposes input state, character grounding, audio instances, navigation paths, random stream cursors, and runtime.stateHash, a fingerprint of the whole simulation.

A real session

What an AI collaborator actually does with "make the courtyard feel like dusk and prove the guard still patrols":

{"op":"world.environment.set","skyColor":[0.08,0.16,0.3],"horizonColor":[0.48,0.68,0.94],
 "fogColor":[0.5,0.62,0.78],"fogDensity":0.015,"ambientIntensity":0.55,"exposure":1.2,
 "atmosphereEnabled":true,"sunDiskIntensity":6}
{"op":"scene.light.point.create","x":14,"y":3,"z":-6}
{"op":"component.light.set","id":41,"kind":"point","color":[1,0.6,0.25],"intensity":9,"range":12,"castShadows":true}
{"op":"render.capture"}
{"op":"runtime.play"}
{"op":"inspect.navigation"}
{"op":"runtime.stop"}
{"op":"history.undo"}

Every step validated, every step visible in your activity log, every step undoable.

Troubleshooting channels

Snapshots carry two complementary channels: activity.entries, a bounded history of the last 200 validated command outcomes, and diagnostics.entries, the current active problem set across import, scripts, audio, renderer streaming, and lighting budgets. Studio's Output workspace renders these exact structures, so you and the AI debug from the same evidence.

Security posture

Local-only named pipe, remote clients rejected. Any future remote bridge requires explicit project-level opt-in, authentication, scoped permissions, audit logging, and user approval for destructive actions. That is a product principle, not a TODO.