Test, Trace and Debug MCP Systems
Move beyond “the tool worked once” and build repeatable tests, traces and failure signals.
Catch a refund regression before production
Your refund tool should reject pending orders. Encode that as a contract test so a later refactor cannot silently loosen the rule.
async def test_refund_pending_order_is_rejected(mcp_client):
result = await mcp_client.call_tool(
"request_refund",
{"order_id": "pending-123", "reason": "changed mind"}
)
assert result.is_error
assert "delivered" in result.message.lower()Trace
Record agent run → selected tool → arguments → server result → final response under one correlation ID.
Evaluate
Keep a small suite of representative user requests and compare tool choice, safety and final-answer quality after changes.
Trade-off: deterministic tool tests are cheap and stable; end-to-end agent evaluations are noisier but catch reasoning and orchestration regressions.