Command (Tool Call) Format Under MEAI

Document version: 2.1 | Date: May 2026

This document describes the internal command (tool call) format generated by models when using functions. With native MEAI (Microsoft.Extensions.AI) function calling, this format is managed automatically.

General Tool Call Format (Internal Structure)

When the model decides to invoke a tool, a structure equivalent to the following MEAI format is produced:

{
  "name": "tool_name",
  "arguments": {
    "param1": "value1",
    "param2": "value2"
  }
}

Warning

DO NOT SPECIFY THIS FORMAT IN PROMPTS.
The native MEAI client automatically passes the tool schema (JSON Schema) to the model. If you manually ask the model to return JSON text, it will respond with ordinary text containing JSON instead of triggering a system (native) tool invocation. In prompts, simply say: "Use the tool [name]."

1. Memory Tool (all roles with memory)

Roles: Creator, Analyzer, Programmer, CoreMechanicAI, custom agents with .WithMemory()

Write (overwrite)

{
  "name": "memory",
  "arguments": {
    "action": "write",
    "content": "Wave 7: difficulty increased, EliteBoss spawned at (50,0,50)"
  }
}

Append

{
  "name": "memory",
  "arguments": {
    "action": "append",
    "content": "Craft#3: Steel + Ice Crystal → Frostblade damage:55 ice:20"
  }
}

Clear

{
  "name": "memory",
  "arguments": {
    "action": "clear"
  }
}

Parameters:

Parameter	Type	Required	Description
`action`	string	Yes	`write` / `append` / `clear`
`content`	string	Yes (except for `clear`)	Text to store

Result: "Memory saved" / "Memory appended" / "Memory cleared"

2. Execute Lua Tool (Programmer)

Roles: Programmer (primary), Creator (when needed)

Note

Runtime Lua execution runs on Lua-CSharp, a managed, AOT-safe VM that works on IL2CPP and WebGL. WebGL player Lua execution is supported and on by default (CoreAISettingsAsset.EnableLuaOnWebGl); if disabled or unsupported, SecureLuaEnvironment.IsSupported is false and Lua envelopes return LuaExecutionFailed instead of executing.

Execute Lua code

{
  "name": "execute_lua",
  "arguments": {
    "code": "local dmg = 45 + math.random(10)\nreport('damage: ' .. dmg)"
  }
}

Parameters:

Parameter	Type	Required	Description
`code`	string	Yes	Lua code to run in the sandbox

Available Lua API:

Function	Description
`report(string)`	Send result back
`add(a, b)`	Add two numbers
`coreai_world_spawn({prefab=..., name=..., x?, y?, z?, rx?, ry?, rz?, scale?, scaleX?, scaleY?, scaleZ?, parent?})`	Spawn object; `prefab` and `name` required, transform and parent optional
`coreai_world_change(name, {x?, y?, z?, rx?, ry?, rz?, scale?, scaleX?, scaleY?, scaleZ?, parent?})`	Apply only supplied transform or parent fields
`coreai_world_set_color(name, htmlColor)`	Set renderer colour
`coreai_world_destroy(name)`	Destroy an object
`coreai_world_set_active(name, active)`	Enable/disable an object
`coreai_world_load_scene(sceneName)`	Load a scene
`coreai_world_reload_scene()`	Reload the scene
`coreai_world_play_animation(name, anim)`	Play an animation
`coreai_world_list_animations(name)`	List animations
`coreai_world_show_text(name, text)`	Show text
`coreai_world_apply_force(name, fx, fy, fz)`	Apply force
`coreai_world_list_objects(pattern)`	Search objects
`coreai_component_add(name, type)`	Add a supported component
`coreai_component_remove(name, type)`	Remove a supported component
`coreai_component_set_number(name, type, property, value)`	Set numeric component property
`coreai_component_set_bool(name, type, property, value)`	Set boolean component property
`coreai_component_set_text(name, type, property, value)`	Set text, colour, or enum component property
`coreai_component_set_vector(name, type, property, x, y, z)`	Set vector component property
On success: Output of `report(...)` or `"Lua executed successfully"`
On error: `"[Error] Lua execution failed: <error description>"` or a platform
unsupported failure when `SecureLuaEnvironment.IsSupported` is `false`.

Multi-line Lua example

{
  "name": "execute_lua",
  "arguments": {
    "code": "-- Ambush spawn script\ncoreai_world_spawn({prefab='Enemy', name='ambush_1', x=10, y=0, z=5})\ncoreai_world_spawn({prefab='Enemy', name='ambush_2', x=-10, y=0, z=5})\ncoreai_world_spawn({prefab='EliteBoss', name='ambush_boss', x=0, y=0, z=15, scaleX=2, scaleY=2, scaleZ=2})\nreport('Ambush setup: 2 enemies + 1 boss')"
  }
}

3. World Command Tool (Creator / Designer)

Roles: Creator, Designer AI, custom agents with WorldTool

Spawn - create object

{
  "name": "world_command",
  "arguments": {
    "action": "spawn",
    "prefabKey": "Enemy",
    "targetName": "enemy_wave7_1",
    "x": 10,
    "y": 0,
    "z": 5,
    "fy": 90,
    "scaleX": 2,
    "scaleY": 1,
    "scaleZ": 3,
    "stringValue": "WaveRoot"
  }
}

prefabKey and targetName are required. Position, rotation (fx/fy/fz), uniform scale, per-axis scaleX/scaleY/scaleZ, parent (stringValue), and worldPositionStays are optional. When a parent is set, coordinates are local by default (worldPositionStays: false); true preserves world space. Without a parent, local and world coordinates are identical. scale is a uniform fallback; use per-axis scale for meter-accurate dimensions.

For compound objects, create a named empty root first and parent related pieces beneath it instead of leaving every piece at scene root.

prefabKey may be a registered prefab key or a built-in primitive key (cube, sphere, cylinder, capsule, plane, empty) when AllowWorldPrimitives is enabled. Registered prefabs take precedence.

Change - partial transform/hierarchy edit

{
  "name": "world_command",
  "arguments": {
    "action": "change",
    "targetName": "enemy_wave7_1",
    "x": 12,
    "fy": 180,
    "scaleY": 1.5,
    "stringValue": "none"
  }
}

change applies only supplied fields. Omitted axes and fields stay unchanged. Use stringValue: "none" to detach from the current parent.

Set color

{
  "name": "world_command",
  "arguments": {
    "action": "set_color",
    "targetName": "enemy_wave7_1",
    "stringValue": "#ff3300"
  }
}

Destroy - remove object

{
  "name": "world_command",
  "arguments": {
    "action": "destroy",
    "targetName": "OldBuilding"
  }
}

List Objects - object list

{
  "name": "world_command",
  "arguments": {
    "action": "list_objects"
  }
}

List Objects - name filter

{
  "name": "world_command",
  "arguments": {
    "action": "list_objects",
    "stringValue": "enemy"
  }
}

Load / reload scene

{
  "name": "world_command",
  "arguments": {
    "action": "load_scene",
    "stringValue": "Level_2"
  }
}

{
  "name": "world_command",
  "arguments": {
    "action": "reload_scene"
  }
}

Other runtime actions

{"name":"world_command","arguments":{"action":"set_active","targetName":"SecretDoor"}}
{"name":"world_command","arguments":{"action":"play_animation","targetName":"Boss1","animationName":"rage_attack"}}
{"name":"world_command","arguments":{"action":"stop_animation","targetName":"Boss1"}}
{"name":"world_command","arguments":{"action":"list_animations","targetName":"Boss1"}}
{"name":"world_command","arguments":{"action":"play_sound","targetName":"Alarm","stringValue":"alarm_clip","volume":0.8}}
{"name":"world_command","arguments":{"action":"set_volume","targetName":"Alarm","volume":0.35}}
{"name":"world_command","arguments":{"action":"show_text","targetName":"QuestPanel","textToDisplay":"Quest completed! +500 XP"}}
{"name":"world_command","arguments":{"action":"hide_panel","targetName":"QuestPanel"}}
{"name":"world_command","arguments":{"action":"apply_force","targetName":"Boulder","fx":0,"fy":100,"fz":50}}
{"name":"world_command","arguments":{"action":"set_velocity","targetName":"Boulder","fx":0,"fy":0,"fz":10}}

Full parameter table:

Parameter	Type	Description
`action`	string	Operation type.
`prefabKey`	string	Registered prefab key or primitive key for `spawn`.
`targetName`	string	GameObject name in the scene.
`x`, `y`, `z`	float	Position coordinates or vector components.
`fx`, `fy`, `fz`	float	Rotation degrees, force vector, or velocity vector depending on action.
`scale`	float	Uniform scale fallback.
`scaleX`, `scaleY`, `scaleZ`	float	Per-axis scale values.
`stringValue`	string	Scene name, search query, parent object name, HTML colour, or sound key depending on action.
`animationName`	string	Animation name.
`textToDisplay`	string	Text to display.
`volume`	float	Audio volume.

All public world_command action values:

Action	Description	Required parameters
`spawn`	Create registered prefab or built-in primitive	`prefabKey`, `targetName`
`change`	Apply optional transform/parent changes	`targetName`
`set_color`	Set renderer colour	`targetName`, `stringValue`
`destroy`	Remove	`targetName`
`list_objects`	List objects	optional `stringValue`
`load_scene`	Load scene	`stringValue`
`reload_scene`	Reload	none
`set_active`	Enable/disable	`targetName`
`play_animation`	Animation	`targetName`, `animationName`
`stop_animation`	Stop animation	`targetName`
`list_animations`	List animations	`targetName`
`play_sound`	Play audio	`targetName`, optional `stringValue`, `volume`
`set_volume`	Set audio volume	`targetName`, `volume`
`show_text`	Show text	`targetName`, `textToDisplay`
`hide_panel`	Hide UI panel/object	`targetName`
`apply_force`	Apply force	`targetName`, `fx`, `fy`, `fz`
`set_velocity`	Set Rigidbody velocity	`targetName`, `fx`, `fy`, `fz`

Removed public world actions: move, rotate, set_scale, parent, set_transform, update_score, and spawn_particles.

4. Component Command Tool (Creator / Designer / Programmer via Lua)

Roles: Custom agents with ComponentLlmTool; Programmer through Lua coreai_component_* bindings.

Add component

{
  "name": "component_command",
  "arguments": {
    "action": "add",
    "targetName": "Cube",
    "componentType": "rigidbody"
  }
}

Remove component

{
  "name": "component_command",
  "arguments": {
    "action": "remove",
    "targetName": "Cube",
    "componentType": "rigidbody"
  }
}

Set component property

{
  "name": "component_command",
  "arguments": {
    "action": "set",
    "targetName": "Lamp",
    "componentType": "light",
    "propertyName": "color",
    "stringValue": "#88aa33"
  }
}

List components

{
  "name": "component_command",
  "arguments": {
    "action": "list_components",
    "targetName": "Cube"
  }
}

Envelope shape:

{
  "action": "add|remove|set|list_components",
  "targetName": "GameObjectName",
  "componentType": "rigidbody",
  "propertyName": "mass",
  "stringValue": "",
  "floatValue": 5,
  "boolValue": 0,
  "x": 0,
  "y": 0,
  "z": 0
}

Parameters:

Parameter	Type	Required	Description
`action`	string	Yes	`add`, `remove`, `set`, or `list_components`
`targetName`	string	Yes	Existing `GameObject` name
`componentType`	string	For `add`, `remove`, `set`	Supported component catalog key
`propertyName`	string	For `set`	Supported property key for the selected component
`stringValue`	string	Optional	Text, HTML colour, or enum name
`floatValue`	number	Optional	Numeric value
`boolValue`	number	Optional	Boolean encoded as `0` or `1`
`x`, `y`, `z`	number	Optional	Vector components

Supported componentType values: rigidbody, rigidbody2d, boxcollider, spherecollider, capsulecollider, meshcollider, light, audiosource, camera, linerenderer, trailrenderer, textmesh, meshrenderer, particlesystem.

set auto-adds the component if it is missing. Example properties include rigidbody.mass, rigidbody.useGravity, rigidbody.isKinematic, rigidbody.drag, rigidbody.angularDrag; light.type, light.intensity, light.range, light.color, light.spotAngle; boxcollider.isTrigger, boxcollider.size, boxcollider.center; and audiosource.volume, audiosource.pitch, audiosource.loop.

Lua bindings publish the same envelope shape:

coreai_component_add("Cube", "rigidbody")
coreai_component_remove("Cube", "rigidbody")
coreai_component_set_number("Cube", "rigidbody", "mass", 5)
coreai_component_set_bool("Cube", "rigidbody", "useGravity", true)
coreai_component_set_text("Lamp", "light", "color", "#88aa33")
coreai_component_set_vector("Trigger", "boxcollider", "size", 2, 3, 2)

5. Get Inventory Tool (Merchant / Shopkeeper)

Roles: Merchant NPC, any agent with InventoryTool

Request inventory

{
  "name": "get_inventory",
  "arguments": {}
}

Result (example):

[
  {"name": "Iron Sword", "type": "weapon", "quantity": 3, "price": 50},
  {"name": "Steel Axe", "type": "weapon", "quantity": 1, "price": 100},
  {"name": "Health Potion", "type": "consumable", "quantity": 10, "price": 25},
  {"name": "Flame Blade", "type": "weapon", "quantity": 1, "price": 250}
]

6. Game Config Tool (Creator / Designer)

Roles: Creator, Designer AI

Read configuration

{
  "name": "game_config",
  "arguments": {
    "action": "read"
  }
}

Update configuration

{
  "name": "game_config",
  "arguments": {
    "action": "update",
    "content": "{\"wave_difficulty\": 1.5, \"spawn_rate\": 2.0, \"boss_hp_multiplier\": 1.3}"
  }
}

Parameters:

Parameter	Type	Required	Description
`action`	string	Yes	`read` / `update`
`content`	string	Yes (for `update`)	JSON with new values

7. Scene Tool (all agents)

Roles: All agents (available in Play Mode)

Find objects

{
  "name": "scene_tool",
  "arguments": {
    "action": "find_objects",
    "pattern": "Enemy*"
  }
}

Get hierarchy

{
  "name": "scene_tool",
  "arguments": {
    "action": "get_hierarchy"
  }
}

Get transform

{
  "name": "scene_tool",
  "arguments": {
    "action": "get_transform",
    "targetName": "Player"
  }
}

Set transform (scene tool only)

{
  "name": "scene_tool",
  "arguments": {
    "action": "set_transform",
    "targetName": "Player",
    "position": {"x": 10, "y": 0, "z": 5}
  }
}

scene_tool.set_transform is separate from world_command; use world_command.change for public name-based world edits.

8. Camera Tool (all agents)

Roles: All agents (available in Play Mode)

Capture screenshot

{
  "name": "camera_snapshot",
  "arguments": {}
}

Result: Base64 JPEG string (for vision analysis by the model)

9. Action / Event Tool (custom agents)

Roles: Custom agents via AgentBuilder.WithAction() / WithEventTool()

Format depends on the C# method signature

// C# definition:
.WithAction("heal_player", "Heal the player fully",
    (int amount) => player.Heal(amount))

// JSON invocation by the model:
{
  "name": "heal_player",
  "arguments": {
    "amount": 100
  }
}

// C# definition:
.WithEventTool("trigger_boss_phase2", "Trigger boss phase 2")

// JSON invocation by the model:
{
  "name": "trigger_boss_phase2",
  "arguments": {}
}

10. Matrix: tools available per role

component_command is optional for host-wired agents through ComponentLlmTool. The Programmer role can publish the same component command envelope through the Lua coreai_component_* bindings when Lua is available.

Role	memory	execute_lua	world_command	get_inventory	game_config	scene_tool	camera	custom
Creator	✅ write	optional	✅	❌	✅	✅	✅	✅
Programmer	✅ append	✅	✅ (via Lua)	❌	❌	✅	✅	✅
Analyzer	✅ append	❌	❌	❌	✅ read	✅	✅	✅
CoreMechanicAI	✅ append	❌	❌	❌	✅ read	❌	❌	✅
AINpc	❌	❌	❌	optional	❌	❌	❌	✅
PlainChat	❌	❌	❌	❌	❌	❌	❌	✅
SmartChat	✅ append	❌	❌	❌	❌	❌	❌	✅
Merchant	✅	❌	❌	✅	❌	❌	❌	✅
Custom	optional	optional	optional	optional	optional	optional	optional	optional

Legend: ✅ default | ❌ disabled | optional — may be added via AgentBuilder

11. Tool call error handling

Tool call retry (up to 3 attempts)

Attempt 1: Model → {"tool": "memory", "action": "write"}  ← Invalid format!
  ↓
System: "ERROR: Tool call not recognized. Use this format: {"name": "...", "arguments": {...}}"
  ↓
Attempt 2: Model → {"name": "memory", "arguments": {"action": "write", "content": "..."}}  ✅

Robust parsing

CoreAI automatically handles:

Missing fenced code markers: ```json {"name":...}```
Reasoning tags: <think>I should call...</think>{"name":...}
Legacy format: {"tool": "memory"} is converted to {"name": "memory"}

Related documents:

COMMAND_FLOW_DIAGRAM.md — command flow diagram
TOOL_CALL_SPEC.md — tool calling specification
AGENT_BUILDER.md — agent builder

FilesExpand file tree

JSON_COMMAND_FORMAT.md

Latest commit

History

JSON_COMMAND_FORMAT.md

File metadata and controls

Command (Tool Call) Format Under MEAI

General Tool Call Format (Internal Structure)

1. Memory Tool (all roles with memory)

Write (overwrite)

Append

Clear

2. Execute Lua Tool (Programmer)

Execute Lua code

Multi-line Lua example

3. World Command Tool (Creator / Designer)

Spawn - create object

Change - partial transform/hierarchy edit

Set color

Destroy - remove object

List Objects - object list

List Objects - name filter

Load / reload scene

Other runtime actions

4. Component Command Tool (Creator / Designer / Programmer via Lua)

Add component

Remove component

Set component property

List components

5. Get Inventory Tool (Merchant / Shopkeeper)

Request inventory

6. Game Config Tool (Creator / Designer)

Read configuration

Update configuration

7. Scene Tool (all agents)

Find objects

Get hierarchy

Get transform

Set transform (scene tool only)

8. Camera Tool (all agents)

Capture screenshot

9. Action / Event Tool (custom agents)

Format depends on the C# method signature

10. Matrix: tools available per role

11. Tool call error handling

Tool call retry (up to 3 attempts)

Robust parsing