Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions cookbook/_routes.json
Original file line number Diff line number Diff line change
Expand Up @@ -493,5 +493,10 @@
"notebook": "example_data_migration-jp.ipynb",
"docsPath": null,
"isGuide": true
},
{
"notebook": "integration_kiro_cli.ipynb",
"docsPath": "integrations/other/kiro-cli",
"isGuide": false
}
]
216 changes: 216 additions & 0 deletions cookbook/integration_kiro_cli.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!-- NOTEBOOK_METADATA source: \"Jupyter Notebook\" title: \"Trace Kiro CLI Sessions with Langfuse\" sidebarTitle: \"Kiro CLI\" logo: \"/images/integrations/kiro_icon.svg\" description: \"Add LLM observability to Kiro CLI coding sessions with Langfuse. Trace agent prompts, tool calls, and completions automatically via Kiro CLI hooks.\" category: \"Integrations\" -->\n",
"\n",
"# Trace Kiro CLI with Langfuse\n",
"\n",
"This guide shows you how to integrate [Langfuse](https://langfuse.com) tracing with [Kiro CLI](https://kiro.dev/cli/) so every coding session — agent prompts, tool calls, and completion status — is captured automatically using [Kiro CLI hooks](https://kiro.dev/docs/cli/hooks/).\n",
"\n",
"> **What is Kiro CLI?** [Kiro CLI](https://kiro.dev/cli/) is the terminal-based AI coding agent from [Kiro](https://kiro.dev). It runs an AI agent in your shell that can read and write files, run commands, and call MCP tools to help you build software.\n",
"\n",
"> **What is Langfuse?** [Langfuse](https://github.com/langfuse/langfuse) is an open-source LLM engineering platform that helps you trace, debug, and evaluate your LLM applications.\n",
"\n",
"> **Note**: This integration is community-maintained at [`dhanpraja231/kiro-cli-langfuse`](https://github.com/dhanpraja231/kiro-cli-langfuse). It uses Kiro CLI's built-in hook system, so no changes to Kiro itself are required.\n",
"\n",
"## How it works\n",
"\n",
"Kiro CLI exposes five hook events that fire over the lifecycle of an agent session. The integration ships a small Node.js handler that subscribes to all five and forwards them to Langfuse via the JS SDK.\n",
"\n",
"| Kiro CLI hook | What it traces in Langfuse |\n",
"| --- | --- |\n",
"| `agentSpawn` | Agent initialization (event) |\n",
"| `userPromptSubmit` | User prompts and queries (generation) |\n",
"| `preToolUse` | Tool invocation before execution (span start) |\n",
"| `postToolUse` | Tool result and duration (span end) |\n",
"| `stop` | Agent completion with status score (event + score) |\n",
"\n",
"Each conversation becomes one trace, sessions are grouped by working directory (`cwd`), and tools/models are surfaced as tags."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!-- STEPS_START -->\n",
"## Prerequisites\n",
"\n",
"Before you begin, ensure you have:\n",
"\n",
"1. [Kiro CLI](https://kiro.dev/cli/) installed and working in your terminal.\n",
"2. [Node.js](https://nodejs.org/) v18 or newer.\n",
"3. A Langfuse account ([sign up for free](https://cloud.langfuse.com)) or a [self-hosted](https://langfuse.com/self-hosting) Langfuse instance.\n",
"4. Langfuse API keys from your project settings."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1: Add the integration to your project\n",
"\n",
"Clone or download the [`kiro-cli-langfuse`](https://github.com/dhanpraja231/kiro-cli-langfuse) repository, then copy the agent definition and hook handler into the project where you use Kiro CLI:\n",
"\n",
"```bash\n",
"git clone https://github.com/dhanpraja231/kiro-cli-langfuse.git\n",
"\n",
"# from the root of the project you want to instrument\n",
"cp -r kiro-cli-langfuse/.kiro/agents/ .kiro/agents/\n",
"cp -r kiro-cli-langfuse/hooks/ hooks/\n",
"```\n",
"\n",
"This adds two things to your project:\n",
"\n",
"- `.kiro/agents/langfuse-observer.json` — a Kiro CLI agent definition that wires every hook event to the Node.js handler.\n",
"- `hooks/` — the hook handler, the Langfuse JS SDK wrapper, and per-event handlers."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2: Install dependencies\n",
"\n",
"Install the handler's dependencies (the Langfuse JS SDK and a couple of small helpers):\n",
"\n",
"```bash\n",
"cd hooks && npm install\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3: Configure Langfuse credentials\n",
"\n",
"Copy the example env file and fill in your Langfuse keys:\n",
"\n",
"```bash\n",
"cp .env.example .env\n",
"```\n",
"\n",
"```bash\n",
"# .env\n",
"LANGFUSE_PUBLIC_KEY=pk-lf-...\n",
"LANGFUSE_SECRET_KEY=sk-lf-...\n",
"\n",
"# 🇪🇺 EU region\n",
"LANGFUSE_BASE_URL=https://cloud.langfuse.com\n",
"# 🇺🇸 US region\n",
"# LANGFUSE_BASE_URL=https://us.cloud.langfuse.com\n",
"```\n",
"\n",
"> **Note**: For self-hosted Langfuse, set `LANGFUSE_BASE_URL` to your own URL (e.g., `http://localhost:3000`)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4: Activate the Langfuse-observer agent\n",
"\n",
"The hooks are defined inside `.kiro/agents/langfuse-observer.json`. The agent inherits `\"tools\": [\"*\"]`, so it keeps full access to file read/write, shell, AWS, and MCP tools while emitting traces in the background.\n",
"\n",
"Switch to it inside Kiro CLI:\n",
"\n",
"```bash\n",
"/agent swap langfuse-observer\n",
"```\n",
"\n",
"Alternatively, merge the `hooks` field into your existing agent config so any agent you use is traced:\n",
"\n",
"```json\n",
"{\n",
" \"name\": \"my-agent\",\n",
" \"hooks\": {\n",
" \"agentSpawn\": [{ \"command\": \"node hooks/hook-handler.js\" }],\n",
" \"userPromptSubmit\": [{ \"command\": \"node hooks/hook-handler.js\" }],\n",
" \"preToolUse\": [{ \"command\": \"node hooks/hook-handler.js\" }],\n",
" \"postToolUse\": [{ \"command\": \"node hooks/hook-handler.js\" }],\n",
" \"stop\": [{ \"command\": \"node hooks/hook-handler.js\" }]\n",
" }\n",
"}\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5 (optional): Filter which tools are traced\n",
"\n",
"By default every tool invocation is captured. To trace only specific tools, add a `matcher` to the `preToolUse`/`postToolUse` hooks in `.kiro/agents/langfuse-observer.json`:\n",
"\n",
"```json\n",
"{\n",
" \"hooks\": {\n",
" \"preToolUse\": [\n",
" {\n",
" \"matcher\": \"fs_write\",\n",
" \"command\": \"node hooks/hook-handler.js\"\n",
" }\n",
" ]\n",
" }\n",
"}\n",
"```\n",
"\n",
"Common matchers:\n",
"\n",
"- `fs_write` / `fs_read` — file write/read tools\n",
"- `execute_bash` — shell commands\n",
"- `use_aws` — AWS CLI tool\n",
"- `@git` — all tools from the git MCP server (use `@git/status` for a single MCP tool)\n",
"- `*` — all tools (default)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 6: View traces in Langfuse\n",
"\n",
"Start a Kiro CLI session as usual and ask the agent to do something. Once you're back in [Langfuse](https://cloud.langfuse.com), you'll see one trace per conversation, with the user prompt as a generation, every tool call as a span, and a completion score on the final `stop` event. Sessions are grouped by working directory so all traces from the same project show up together.\n",
"\n",
"Trace hierarchy:\n",
"\n",
"- **Trace** — one per Kiro CLI conversation\n",
" - **Session** — grouped by `cwd`\n",
" - **Events** — `agentSpawn`, `stop`\n",
" - **Generations** — user prompts\n",
" - **Spans** — tool use (`preToolUse` → `postToolUse`)\n",
" - **Scores** — completion status (0–1)\n",
"\n",
"<!-- TODO: replace with your actual trace screenshot, uploaded to langfuse.com images, and a public example trace link -->\n",
"![Example Kiro CLI trace in Langfuse](https://langfuse.com/images/cookbook/integration-kiro-cli/kiro-cli-example-trace.png)\n",
"\n",
"[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/TODO)\n",
"\n",
"<!-- STEPS_END -->"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!-- MARKDOWN_COMPONENT name: \"LearnMore\" path: \"@/components-mdx/integration-learn-more-js.mdx\" -->"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.11.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
11 changes: 11 additions & 0 deletions public/images/integrations/kiro_icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading