Test Model Tool Calling in OpenClaw

Do not approve a model from a chat sample alone. Agent reliability depends on what happens before, during, and after a tool call.

Freeze the test conditions

Record the exact provider/model reference, OpenClaw version, runtime version, hardware, context setting, and tool policy. Start each candidate with a fresh session and the same files.

openclaw --version
openclaw models status

Run five tests

1. No-tool answer: ask a factual question that does not need a tool. The model should not call one unnecessarily.
2. Single read: ask it to read one harmless fixture and report a specific value.
3. Structured arguments: provide a tool with required and optional fields; check the emitted arguments, not only the final prose.
4. Recoverable failure: make the first path invalid and provide a valid alternative. The model should inspect the error and retry deliberately.
5. Two-step task: require one read followed by one non-destructive transformation. The model should stop after the requested result.

Use disposable fixtures. Do not give a candidate model broad file access, production credentials, or a real message-sending tool during evaluation.

Score observable behavior

Track first-pass success, malformed calls, unnecessary calls, repeated calls, recovery success, time, and token usage. A smaller model that finishes consistently can be more useful than a stronger chat model that loops after errors.

Test denial behavior

Ask for an action outside the configured allowlist. A safe result is a clear failure or request for approval—not an invented success message.

Promote the model

Set the primary model only after the candidate passes the representative task set:

openclaw models set <provider/model>
openclaw models status

Expected result

The model chooses tools only when needed, produces valid arguments, uses error output to recover, respects denied operations, and terminates without a retry loop.