Skip to content

feat: log AI coding agents in user-agent, telemetry, and tracing#550

Open
mwbrooks wants to merge 10 commits into
mainfrom
mwbrooks-track-ai-agent
Open

feat: log AI coding agents in user-agent, telemetry, and tracing#550
mwbrooks wants to merge 10 commits into
mainfrom
mwbrooks-track-ai-agent

Conversation

@mwbrooks
Copy link
Copy Markdown
Member

@mwbrooks mwbrooks commented May 15, 2026

Changelog

When the CLI is invoked by an AI coding agent, the HTTP User-Agent header now includes an AI-Agent suffix identifying the agent. Developers can see the user agent in the --verbose debug logs.

# No AI Agent
$ slack version --verbose
# user_agent: slack-cli/v4.0.1 (os: darwin)

# Pretending we are Claude
$ CLAUDECODE=1 CLAUDE_CODE_ENTRYPOINT=cli slack version --verbose
# user_agent: slack-cli/v4.0.1 (os: darwin) AI-Agent (name=claude-code, entry=cli)

Summary

This pull request adds detection of AI coding agents and surfaces the detected agent across HTTP requests, telemetry, and tracing.

Format:

  • slack-cli/v4.0.1 (os: darwin) AI-Agent (name=claude-code, entry=cli)
  • slack-cli/v4.0.1 (os: darwin) AI-Agent (name=claude-code, entry=desktop)
  • slack-cli/v4.0.1 (os: darwin) AI-Agent (name=codex)

Analytics:

  • We can look up "AI-Agent" to know if the CLI was run by an agent.
  • We can look up the specific agent used by name=.
  • We have the ability to log additional agent information when available (e.g. entry=) but it's very limited today.

Detected AI agents environment variables:

  • CLAUDECODEclaude-code
    • CLAUDE_CODE_ENTRYPOINTcli or desktop or possibly even vscode
  • CODEX_CIcodex
  • GEMINI_CLIgemini-cli
    • This could be gemini if we want, but since the env var is specific the CLI we may want to take advantage of that
  • CLINE_ACTIVEcline
  • CURSOR_AGENTcursor
  • AGENT
    • Generic fallback that's emerging as the common env var for agents

When detected:

  • The User-Agent header on all API requests gets an AI-Agent (name=<name>, [entry=<entrypoint>]) addition
  • Logstash telemetry events include an agent field in the context
  • Jaeger tracing gets an ai_agent span tag
  • Verbose output prints the full user-agent at startup

Testing

# Test Claude Code WITH an entry value
$ CLAUDECODE=1 CLAUDE_CODE_ENTRYPOINT=cli slack version --verbose
# [2026-05-15 15:29:06] user_agent: slack-cli/v4.0.1 (os: darwin) AI-Agent (name=claude-code, entry=cli)
# ...
# [2026-05-15 15:40:26] FlushToLogstash will POST https://dev.slackb.com/events/cli payload: [{"context":{"agent":"claude-code","arch":"arm64,...

# Test Claude Code WITHOUT an entry value
$ CLAUDECODE=1 slack version --verbose
# [2026-05-15 15:29:06] user_agent: slack-cli/v4.0.1 (os: darwin) AI-Agent (name=claude-code)
# ...
# [2026-05-15 15:40:26] FlushToLogstash will POST https://dev.slackb.com/events/cli payload: [{"context":{"agent":"claude-code","arch":"arm64,...

Requirements

mwbrooks added 6 commits May 15, 2026 14:22
When the Slack CLI is invoked by an AI coding agent (Claude Code, Codex,
Gemini CLI, Cline, Cursor, Goose, Amp, or others), detect the agent via
environment variables and append an AI-Agent product token to the HTTP
user-agent header. Also propagates the agent name to telemetry events
and Jaeger tracing spans.

User-Agent format:
  slack-cli/2.38.1 (os: darwin) AI-Agent (name=claude-code, entry=cli)
Removes duplicate detectAIAgentName/detectAgentName helpers from main.go
and tracking.go in favor of useragent.DetectName().
@mwbrooks mwbrooks self-assigned this May 15, 2026
@mwbrooks mwbrooks added code health M-T: Test improvements and anything that improves code health semver:patch Use on pull requests to describe the release version increment labels May 15, 2026
@mwbrooks mwbrooks added this to the Next Release milestone May 15, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

❌ Patch coverage is 85.71429% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.66%. Comparing base (3720e4f) to head (082c836).

Files with missing lines Patch % Lines
internal/useragent/useragent.go 87.09% 4 Missing ⚠️
main.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #550      +/-   ##
==========================================
+ Coverage   71.64%   71.66%   +0.01%     
==========================================
  Files         225      226       +1     
  Lines       19074    19104      +30     
==========================================
+ Hits        13665    13690      +25     
- Misses       4202     4207       +5     
  Partials     1207     1207              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mwbrooks mwbrooks changed the title feat: detect AI coding agents in user-agent, telemetry, and tracing feat: log AI coding agents in user-agent, telemetry, and tracing May 15, 2026
Copy link
Copy Markdown
Member Author

@mwbrooks mwbrooks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments for the kind reviewers! 🖊️

Comment thread cmd/root.go
rootCmd.SetContext(ctx)
// Debug logging
clients.IO.PrintDebug(ctx, "system_id: %s", clients.Config.SystemID)
clients.IO.PrintDebug(ctx, "user_agent: %s", useragent.BuildUserAgent(version.Raw()))
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: This is a new addition to our verbose logs. For a while, I've wanted to see our user-agent which is used for each API request. This may also be helpful when diagnosing user reported issues.

Comment thread main.go
span.SetTag("hashed_hostname", ioutils.GetHostname())
span.SetTag("slack_cli_process", processName)
if agentName := useragent.DetectName(); agentName != "" {
span.SetTag("ai_agent", agentName)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I chose ai_agent instead of agent because it's less ambiguous and more specific that it's an AI "agent".


// EventContext contains information / metadata about the CLI session
type EventContext struct {
AIAgent string `json:"ai_agent,omitempty"`
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I chose ai_agent instead of agent because it's less ambiguous and more specific that it's an AI "agent".

@mwbrooks mwbrooks added changelog Use on updates to be included in the release notes enhancement M-T: A feature request for new functionality and removed code health M-T: Test improvements and anything that improves code health labels May 15, 2026
@mwbrooks mwbrooks marked this pull request as ready for review May 15, 2026 23:12
@mwbrooks mwbrooks requested a review from a team as a code owner May 15, 2026 23:12
Copy link
Copy Markdown
Member

@zimeg zimeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mwbrooks Let's uncover patterns to encourage whenever's right 📊

The comments I leave aren't blocking so please do ignore but some of these are questions I have about user agent itself? Thanks for landing this much.

ua := fmt.Sprintf("slack-cli/%s (os: %s)", cliVersion, runtime.GOOS)
if agent := Detect(); agent != nil {
var parts []string
parts = append(parts, "name="+agent.Name)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
parts = append(parts, "name="+agent.Name)
parts = append(parts, "name: "+agent.Name)

☎️ question: A ":" separator might match the os adjacent but I understand can make queries more difficult. If this was intentional please ignore but I did want to ask?

Comment on lines +64 to +65
// BuildUserAgent constructs the HTTP User-Agent header value for the CLI. If an
// AI agent is detected, an "AI-Agent (name=..., entry=...)" suffix is appended.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📚 quibble: It'd be helpful to have a full example here but no blocker!


// DetectName returns the normalized name of the detected AI agent, or an empty
// string if no agent is detected.
func DetectName() string {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func DetectName() string {
func DetectHarness() string {

🪬 question: Forgive these words but this is an unusual export for the value found I think but I also admit to not being so familiar with practices encouraged here!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌃 ramble: It does seem solid if we're framing this as the... actual user agent though? I'm understanding more I hope!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog Use on updates to be included in the release notes enhancement M-T: A feature request for new functionality semver:patch Use on pull requests to describe the release version increment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants