docs: add Remote Trace Providers guide for evals SDK#634
Merged
Unshure merged 1 commit intostrands-agents:mainfrom Mar 17, 2026
Merged
docs: add Remote Trace Providers guide for evals SDK#634Unshure merged 1 commit intostrands-agents:mainfrom
Unshure merged 1 commit intostrands-agents:mainfrom
Conversation
Add documentation for evaluating remote traces from observability backends (CloudWatch and Langfuse). Includes setup, configuration, and usage examples for running evaluations against production/staging agent traces without re-running them. Update navigation to include the new page.
afarntrog
temporarily deployed
to
manual-approval
GitHub Actions
Inactive
Contributor
Documentation Preview ReadyYour documentation preview has been successfully deployed! Preview URL: https://d3ehv1nix5p99z.cloudfront.net/pr-cms-634/docs/user-guide/quickstart/overview/ Updated at: 2026-03-10T17:43:04.868Z |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Assessment: Request Changes
This is a well-structured documentation guide for evaluating remote traces. The content is clear, includes proper setup instructions, error handling, and a custom provider implementation section.
Review Summary
- Code Example Accuracy: The
CoherenceEvaluatorreferenced on lines 104 and 125 doesn't appear to exist in the documented evaluators. This should be replaced with an existing evaluator (e.g.,HelpfulnessEvaluator,FaithfulnessEvaluator). - Documentation Structure: Well-organized with clear sections for installation, configuration, usage examples, and error handling.
- Navigation: Appropriately placed in the evals-sdk section.
Nice addition to the evals SDK documentation - just needs the evaluator reference fixed.
poshinchen
approved these changes
Mar 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add documentation for evaluating remote traces from observability backends (CloudWatch and Langfuse). Includes setup, configuration, and usage examples for running evaluations against production/staging agent traces without re-running them. Update navigation to include the new page.
Related Issues
Type of Change
Checklist
npm run devBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.