Skip to content

feat: add optional Langfuse tracing for MCP tool calls#37

Open
adalton wants to merge 1 commit intomainfrom
andalton/RHOAIENG-48700-langfuse-integration
Open

feat: add optional Langfuse tracing for MCP tool calls#37
adalton wants to merge 1 commit intomainfrom
andalton/RHOAIENG-48700-langfuse-integration

Conversation

@adalton
Copy link

@adalton adalton commented Mar 5, 2026

Summary

  • Add Langfuse tracing integration to the mcp-acp server (RHOAIENG-48700)
  • Every MCP tool call produces a Langfuse trace with tool name, filtered args, duration, and status
  • HTTP requests to the ACP public API appear as nested child spans with method, path, and status code
  • Silently no-ops when Langfuse credentials are not configured or MCP_ACP_TRACING_ENABLED=false

Changes

File What
pyproject.toml Add langfuse>=2.0.0 dependency
src/mcp_acp/tracing.py New — Langfuse v3 SDK client singleton, trace_tool_call + trace_http_request async context managers, no-op fallbacks, flush/shutdown lifecycle
src/mcp_acp/server.py Wrap call_tool() in trace_tool_call, flush tracing on shutdown
src/mcp_acp/client.py Wrap _request() and _request_text() in trace_http_request
src/mcp_acp/settings.py Add missing _acpctl_config_path() (pre-existing bug fix)
tests/test_tracing.py New — 25 unit tests covering all tracing paths
scripts/test_tracing_e2e.py New — E2E test script against live ACP cluster

Configuration

Opt-in via environment variables (Langfuse SDK reads these automatically):

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_BASE_URL="https://us.cloud.langfuse.com"

Kill switch: MCP_ACP_TRACING_ENABLED=false disables tracing even with credentials present.

Test plan

  • 25 unit tests pass (uv run pytest tests/test_tracing.py)
  • All 53 existing tests pass (uv run pytest tests/)
  • Ruff lint/format clean
  • E2E verified against live kind cluster with Langfuse Cloud — traces visible in dashboard
  • Verified silent no-op when credentials are not set
  • Verified kill switch disables tracing with credentials present
  • Verify CI passes

🤖 Generated with Claude Code

Add observability to the mcp-acp server via Langfuse integration. Every
MCP tool call produces a trace with tool name, filtered args, duration,
and success/error status. HTTP requests to the ACP public API appear as
nested child spans with method, path, and status code.

Tracing is opt-in and silently no-ops when unconfigured:
- Reads LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL
- MCP_ACP_TRACING_ENABLED env var (default: true) serves as kill switch
- SDK init failures are caught and logged, never break tool execution

- tracing.py: Langfuse v3 SDK client singleton, trace_tool_call and
  trace_http_request async context managers, no-op fallbacks, lifecycle
- server.py: wrap call_tool() dispatch in trace_tool_call, flush on exit
- client.py: wrap _request() and _request_text() in trace_http_request
- settings.py: add missing _acpctl_config_path() (pre-existing bug fix)
- test_tracing.py: 25 unit tests covering all tracing paths
- scripts/test_tracing_e2e.py: e2e test against live ACP cluster

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@adalton adalton self-assigned this Mar 5, 2026
@adalton adalton requested a review from jeremyeder March 5, 2026 20:36
@adalton
Copy link
Author

adalton commented Mar 5, 2026

Testing

  ============================================================
  MCP-ACP Tracing End-to-End Test
  ============================================================
    Langfuse credentials: configured
    Tracing enabled:      True
    Kill switch:          MCP_ACP_TRACING_ENABLED=true

  [OK] MCP server started and initialized
  [OK] list_tools: 26 tools available

  --- Tool: acp_whoami ---
    Response: Configuration Status:

  Token Configured: Yes
  Cluster: kind
  Server: http://172.19.0.2:31648
  Project: ambient-code


  --- Tool: acp_list_clusters ---
    Response: Configured Clusters (default: kind):

  - kind [DEFAULT]
    Server: http://172.19.0.2:31648
    Description: Local kind development cluster
    Default Project: ambient-code



  --- Tool: acp_list_sessions ---
    Response: Found 0 session(s)

  Sessions:


  --- Tool: acp_get_session (expect error) ---
  {"method": "GET", "path": "/v1/sessions/nonexistent-session-xyz", "status_code": 404, "error": "Session not found", "event": "api_request_failed", "logger":
  "mcp_acp.client", "level": "warning", "timestamp": "2026-03-05T20:14:42.306721Z"}
  {"tool": "acp_get_session", "elapsed_seconds": 0.01, "error": "Session not found", "event": "tool_validation_error", "logger": "__main__", "level": "warning",
  "timestamp": "2026-03-05T20:14:42.306905Z"}
    Response: Validation Error: Session not found
    Got expected error: True

  --- Tool: acp_create_session (dry_run) ---
    Response: DRY RUN MODE - No changes made

  Would create session with custom prompt

  Manifest:
  {
    "task": "Hello from tracing e2e test",
    "model": "claude-sonnet-4",
    "displayName": "tracing-e2e-test"
  }

  --- Tool: acp_bulk_delete_sessions (expect validation error) ---
  {"tool": "acp_bulk_delete_sessions", "elapsed_seconds": 0.0, "error": "Bulk delete sessions requires confirm=true. Use dry_run=true to preview first.", "event":
  "tool_validation_error", "logger": "__main__", "level": "warning", "timestamp": "2026-03-05T20:14:42.315672Z"}
    Response: Validation Error: Bulk delete sessions requires confirm=true. Use dry_run=true to preview first.
    Got expected validation error: True

  ============================================================
  RESULTS SUMMARY
  ============================================================
    [PASS] list_tools
    [PASS] acp_whoami
    [PASS] acp_list_clusters
    [PASS] acp_list_sessions
    [PASS] acp_get_session_error
    [PASS] acp_create_session_dry_run
    [PASS] acp_bulk_validation_error

    7/7 tests passed

    Check your Langfuse dashboard for traces:
      https://us.cloud.langfuse.com
    Expected traces:
      - acp_whoami
      - acp_list_clusters
      - acp_list_sessions        (with child span: GET /v1/sessions)
      - acp_get_session          (with error status)
      - acp_create_session       (dry_run, no HTTP span)
      - acp_bulk_delete_sessions (validation_error, no HTTP span)
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant