In-memory test harness: invoke MCP tools without spinning stdio/HTTP #2

New issue

Closed

opened 2026-05-23 20:53:53 +00:00 by coilysiren · 0 comments

coilysiren commented

2026-05-23 20:53:53 +00:00

Owner

Originally filed by @coilysiren on 2026-05-13T14:23:50Z - https://github.com/coilysiren/cli-mcp/issues/41

FastMCP provides an in-memory client that connects directly to a FastMCP instance without going through stdio or HTTP. This is the single biggest DX win for testing - no subprocess, no transport, just client.call_tool("name", args) against a server object in the same process.

cli-mcp today has unit tests at the projection-mechanics level but no canonical pattern for an integration test that says "build a server, call this tool by name, assert on the result." Today you'd have to spin a real ServeStdio or StreamableHTTPHandler, both of which add framing overhead and make assertions verbose.

Proposal: ship a climcp/climcptest (or similar) helper exposing a direct invoker:

srv, _ := climcp.NewServer(root, opts)
res, err := climcptest.Call(srv, "tool.name", map[string]any{...})
assert.Equal(t, "expected output", res.Content[0].(*mcp.TextContent).Text)

Should hit the same code path as a real stdio invocation (input validation, tool dispatch, output capture) but without the framing layer. The in-process execution architecture (issue #36) makes this almost free - the Action already runs in the test goroutine.

Open questions:

Reuse the SDK's in-memory transport if it exposes one, vs build cli-mcp-specific helper?
Should the helper also exercise IsError + StructuredContent (issue #34) once those land?
Scope: just tool calls, or also ListTools / capability negotiation?

_Originally filed by @coilysiren on 2026-05-13T14:23:50Z - [https://github.com/coilysiren/cli-mcp/issues/41](https://github.com/coilysiren/cli-mcp/issues/41)_ [FastMCP][fastmcp] provides an in-memory client that connects directly to a `FastMCP` instance without going through stdio or HTTP. This is the single biggest DX win for testing - no subprocess, no transport, just `client.call_tool("name", args)` against a server object in the same process. cli-mcp today has unit tests at the projection-mechanics level but no canonical pattern for an integration test that says "build a server, call this tool by name, assert on the result." Today you'd have to spin a real `ServeStdio` or `StreamableHTTPHandler`, both of which add framing overhead and make assertions verbose. Proposal: ship a `climcp/climcptest` (or similar) helper exposing a direct invoker: ```go srv, _ := climcp.NewServer(root, opts) res, err := climcptest.Call(srv, "tool.name", map[string]any{...}) assert.Equal(t, "expected output", res.Content[0].(*mcp.TextContent).Text) ``` Should hit the same code path as a real stdio invocation (input validation, tool dispatch, output capture) but without the framing layer. The in-process execution architecture (issue #36) makes this almost free - the Action already runs in the test goroutine. Open questions: - Reuse the SDK's in-memory transport if it exposes one, vs build cli-mcp-specific helper? - Should the helper also exercise `IsError` + `StructuredContent` (issue #34) once those land? - Scope: just tool calls, or also `ListTools` / capability negotiation? [fastmcp]: https://github.com/jlowin/fastmcp

coilysiren added the

label

2026-05-30 17:09:02 +00:00

coilysiren

2026-06-17 08:24:07 +00:00

closed this issue
added the
burndown-2026-06
label

Commenting is not possible because the repository is archived.

No labels

burndown-2026-06

icebox

No milestone

No project

No assignees

1 participant

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

coilysiren/cli-mcp#2

No description provided.

Rows
Columns