Automated MCP Server Testing
Use different test layers for different failure classes.
A practical test pyramid
- Domain unit tests: business services without MCP.
- In-process MCP tests: schemas, tools, resources, prompts.
- Transport tests: stdio process or Streamable HTTP wiring.
- Authorization tests: scopes, tenants, negative access.
- Agent evaluations: does the model select and use the capability correctly?
Keep deterministic tests deterministic
Do not use a live LLM when validating protocol schema or database authorization. Reserve model calls for behavior evaluations where model variability is actually under test.
Version contracts
Pin SDK versions in CI and run compatibility tests before upgrading protocol/SDK dependencies.