Skip to content

Add environment retrieval to the LLMObs prompt API #4250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
api-clients-generation-pipeline wants to merge 1 commit into from
Closed

Add environment retrieval to the LLMObs prompt API #4250

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 37 additions & 4 deletions .generator/schemas/v2/openapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1080,6 +1080,16 @@ components:
required: true
schema:
type: string
LLMObsPromptEnvironmentQueryParameter:
description: >-
Optional `DD_ENV` value used to resolve the prompt version deployed to the matching Feature Flags environment. This value is not a Feature Flags environment UUID. Using this parameter additionally requires the `feature_flag_config_read` and `feature_flag_environment_config_read` permissions. This parameter cannot be used with `label`.
example: production
in: query
name: environment
required: false
schema:
minLength: 1
type: string
LLMObsPromptIDPathParameter:
description: The customer-provided identifier of the Agent Observability prompt.
example: "customer-support-assistant"
Expand All @@ -1090,7 +1100,7 @@ components:
type: string
LLMObsPromptLabelQueryParameter:
description: >-
**Deprecated.** Optional label of the prompt version to return. Do not use this parameter for new integrations. If omitted, the latest version is returned. If the prompt has no labels, the latest version is returned even when a label is requested. If the prompt has labels but none match the requested label, a 404 response is returned.
**Deprecated.** Optional label of the prompt version to return. Do not use this parameter for new integrations. If omitted, the latest version is returned. If the prompt has no labels, the latest version is returned even when a label is requested. If the prompt has labels but none match the requested label, a 404 response is returned. This parameter cannot be used with `environment`.
in: query
name: label
required: false
Expand Down Expand Up @@ -167110,11 +167120,13 @@ paths:
**Note**: This endpoint is in preview and is subject to change.
If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/).
get:
description: Get the latest version of an Agent Observability prompt by prompt ID.
description: >-
Get an Agent Observability prompt by prompt ID. When `environment` is omitted, this returns the latest version or uses the deprecated `label` behavior and requires `llm_observability_read`. When `environment` is supplied, it must be a nonempty `DD_ENV` value, cannot be combined with `label`, and additionally requires `feature_flag_config_read` and `feature_flag_environment_config_read`. An empty environment or combining it with `label` returns 400, and missing either additional permission returns 403. A missing prompt or deployment returns 404 without falling back to the latest version. An environment resolution failure returns 500.
operationId: GetLLMObsPrompt
parameters:
- $ref: "#/components/parameters/LLMObsPromptIDPathParameter"
- $ref: "#/components/parameters/LLMObsPromptLabelQueryParameter"
- $ref: "#/components/parameters/LLMObsPromptEnvironmentQueryParameter"
responses:
"200":
content:
Expand All @@ -167137,6 +167149,14 @@ paths:
schema:
$ref: "#/components/schemas/LLMObsPromptSDKResponse"
description: OK
"400":
content:
application/json:
schema:
$ref: "#/components/schemas/JSONAPIErrorResponse"
description: |-
Bad Request
The environment is empty, or environment and label are both supplied.
"401":
content:
application/json:
Expand All @@ -167148,15 +167168,28 @@ paths:
application/json:
schema:
$ref: "#/components/schemas/JSONAPIErrorResponse"
description: Forbidden
description: |-
Forbidden
The application key lacks `llm_observability_read`, or environment resolution was requested without both
`feature_flag_config_read` and `feature_flag_environment_config_read`.
"404":
content:
application/json:
schema:
$ref: "#/components/schemas/JSONAPIErrorResponse"
description: Not Found
description: |-
Not Found
The prompt does not exist, or no prompt deployment can be resolved for the supplied environment.
"429":
$ref: "#/components/responses/TooManyRequestsResponse"
"500":
content:
application/json:
schema:
$ref: "#/components/schemas/JSONAPIErrorResponse"
description: |-
Internal Server Error
Environment resolution failed.
security:
- apiKeyAuth: []
appKeyAuth: []
Expand Down
29 changes: 29 additions & 0 deletions examples/v2/agent-observability/GetLLMObsPrompt_2452494832.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// Get an Agent Observability prompt by environment returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;
import com.datadog.api.client.v2.api.AgentObservabilityApi.GetLLMObsPromptOptionalParameters;
import com.datadog.api.client.v2.model.LLMObsPromptSDKResponse;

public class Example {
public static void main(String[] args) {
ApiClient defaultClient = ApiClient.getDefaultApiClient();
defaultClient.setUnstableOperationEnabled("v2.getLLMObsPrompt", true);
AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);

try {
LLMObsPromptSDKResponse result =
apiInstance.getLLMObsPrompt(
"customer-support-assistant",
new GetLLMObsPromptOptionalParameters().environment("production"));
System.out.println(result);
} catch (ApiException e) {
System.err.println("Exception when calling AgentObservabilityApi#getLLMObsPrompt");
System.err.println("Status code: " + e.getCode());
System.err.println("Reason: " + e.getResponseBody());
System.err.println("Response headers: " + e.getResponseHeaders());
e.printStackTrace();
}
}
}
38 changes: 34 additions & 4 deletions src/main/java/com/datadog/api/client/v2/api/AgentObservabilityApi.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5632,6 +5632,7 @@ public ApiResponse<LLMObsPatternsRunStatusResponse> getLLMObsPatternsRunStatusWi
/** Manage optional parameters to getLLMObsPrompt. */
public static class GetLLMObsPromptOptionalParameters {
private String label;
private String environment;

/**
* Set label.
Expand All @@ -5640,13 +5641,28 @@ public static class GetLLMObsPromptOptionalParameters {
* not use this parameter for new integrations. If omitted, the latest version is returned.
* If the prompt has no labels, the latest version is returned even when a label is
* requested. If the prompt has labels but none match the requested label, a 404 response is
* returned. (optional)
* returned. This parameter cannot be used with <code>environment</code>. (optional)
* @return GetLLMObsPromptOptionalParameters
*/
public GetLLMObsPromptOptionalParameters label(String label) {
this.label = label;
return this;
}

/**
* Set environment.
*
* @param environment Optional <code>DD_ENV</code> value used to resolve the prompt version
* deployed to the matching Feature Flags environment. This value is not a Feature Flags
* environment UUID. Using this parameter additionally requires the <code>
* feature_flag_config_read</code> and <code>feature_flag_environment_config_read</code>
* permissions. This parameter cannot be used with <code>label</code>. (optional)
* @return GetLLMObsPromptOptionalParameters
*/
public GetLLMObsPromptOptionalParameters environment(String environment) {
this.environment = environment;
return this;
}
}

/**
Expand Down Expand Up @@ -5712,7 +5728,15 @@ public CompletableFuture<LLMObsPromptSDKResponse> getLLMObsPromptAsync(
}

/**
* Get the latest version of an Agent Observability prompt by prompt ID.
* Get an Agent Observability prompt by prompt ID. When <code>environment</code> is omitted, this
* returns the latest version or uses the deprecated <code>label</code> behavior and requires
* <code>llm_observability_read</code>. When <code>environment</code> is supplied, it must be a
* nonempty <code>DD_ENV</code> value, cannot be combined with <code>label</code>, and
* additionally requires <code>feature_flag_config_read</code> and <code>
* feature_flag_environment_config_read</code>. An empty environment or combining it with <code>
* label</code> returns 400, and missing either additional permission returns 403. A missing
* prompt or deployment returns 404 without falling back to the latest version. An environment
* resolution failure returns 500.
*
* @param promptId The customer-provided identifier of the Agent Observability prompt. (required)
* @param parameters Optional parameters for the request.
Expand All @@ -5723,10 +5747,12 @@ public CompletableFuture<LLMObsPromptSDKResponse> getLLMObsPromptAsync(
* <caption>Response details</caption>
* <tr><td> Status Code </td><td> Description </td><td> Response Headers </td></tr>
* <tr><td> 200 </td><td> OK </td><td> - </td></tr>
* <tr><td> 400 </td><td> Bad Request The environment is empty, or environment and label are both supplied. </td><td> - </td></tr>
* <tr><td> 401 </td><td> Unauthorized </td><td> - </td></tr>
* <tr><td> 403 </td><td> Forbidden </td><td> - </td></tr>
* <tr><td> 404 </td><td> Not Found </td><td> - </td></tr>
* <tr><td> 403 </td><td> Forbidden The application key lacks &#x60;llm_observability_read&#x60;, or environment resolution was requested without both &#x60;feature_flag_config_read&#x60; and &#x60;feature_flag_environment_config_read&#x60;. </td><td> - </td></tr>
* <tr><td> 404 </td><td> Not Found The prompt does not exist, or no prompt deployment can be resolved for the supplied environment. </td><td> - </td></tr>
* <tr><td> 429 </td><td> Too many requests </td><td> - </td></tr>
* <tr><td> 500 </td><td> Internal Server Error Environment resolution failed. </td><td> - </td></tr>
* </table>
*/
public ApiResponse<LLMObsPromptSDKResponse> getLLMObsPromptWithHttpInfo(
Expand All @@ -5746,6 +5772,7 @@ public ApiResponse<LLMObsPromptSDKResponse> getLLMObsPromptWithHttpInfo(
400, "Missing the required parameter 'promptId' when calling getLLMObsPrompt");
}
String label = parameters.label;
String environment = parameters.environment;
// create path and map variables
String localVarPath =
"/api/v2/llm-obs/v1/prompts/{prompt_id}"
Expand All @@ -5755,6 +5782,7 @@ public ApiResponse<LLMObsPromptSDKResponse> getLLMObsPromptWithHttpInfo(
Map<String, String> localVarHeaderParams = new HashMap<String, String>();

localVarQueryParams.addAll(apiClient.parameterToPairs("", "label", label));
localVarQueryParams.addAll(apiClient.parameterToPairs("", "environment", environment));

Invocation.Builder builder =
apiClient.createBuilder(
Expand Down Expand Up @@ -5808,6 +5836,7 @@ public CompletableFuture<ApiResponse<LLMObsPromptSDKResponse>> getLLMObsPromptWi
return result;
}
String label = parameters.label;
String environment = parameters.environment;
// create path and map variables
String localVarPath =
"/api/v2/llm-obs/v1/prompts/{prompt_id}"
Expand All @@ -5817,6 +5846,7 @@ public CompletableFuture<ApiResponse<LLMObsPromptSDKResponse>> getLLMObsPromptWi
Map<String, String> localVarHeaderParams = new HashMap<String, String>();

localVarQueryParams.addAll(apiClient.parameterToPairs("", "label", label));
localVarQueryParams.addAll(apiClient.parameterToPairs("", "environment", environment));

Invocation.Builder builder;
try {
Expand Down
17 changes: 17 additions & 0 deletions src/test/resources/com/datadog/api/client/v2/api/agent_observability.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -766,6 +766,23 @@ Feature: Agent Observability
When the request is sent
Then the response status is 200 OK

@skip @team:DataDog/ml-observability
Scenario: Get an Agent Observability prompt by environment returns "OK" response
Given operation "GetLLMObsPrompt" enabled
And new "GetLLMObsPrompt" request
And request contains "prompt_id" parameter from "REPLACE.ME"
And request contains "environment" parameter with value "production"
When the request is sent
Then the response status is 200 OK

@generated @skip @team:DataDog/ml-observability
Scenario: Get an Agent Observability prompt returns "Bad Request" response
Given operation "GetLLMObsPrompt" enabled
And new "GetLLMObsPrompt" request
And request contains "prompt_id" parameter from "REPLACE.ME"
When the request is sent
Then the response status is 400 Bad Request

@generated @skip @team:DataDog/ml-observability
Scenario: Get an Agent Observability prompt returns "Not Found" response
Given operation "GetLLMObsPrompt" enabled
Expand Down
Loading