Add endpoint preset creation and reuse

Author: Andrey Cheptsov

mkdocs.yml

@@ -297,6 +297,7 @@ nav:
           - Services: docs/concepts/services.md
           - Volumes: docs/concepts/volumes.md
           - More:
+            - Endpoints: docs/concepts/endpoints.md
             - Gateways: docs/concepts/gateways.md
             - Secrets: docs/concepts/secrets.md
             - Projects: docs/concepts/projects.md
@@ -344,6 +345,7 @@ nav:
               - dev-environment: docs/reference/dstack.yml/dev-environment.md
               - task: docs/reference/dstack.yml/task.md
               - service: docs/reference/dstack.yml/service.md
+              - endpoint: docs/reference/dstack.yml/endpoint.md
               - fleet: docs/reference/dstack.yml/fleet.md
               - gateway: docs/reference/dstack.yml/gateway.md
               - volume: docs/reference/dstack.yml/volume.md
@@ -366,6 +368,7 @@ nav:
               - dstack volume: docs/reference/cli/dstack/volume.md
               - dstack gateway: docs/reference/cli/dstack/gateway.md
               - dstack secret: docs/reference/cli/dstack/secret.md
+              - dstack endpoint: docs/reference/cli/dstack/endpoint.md
               - dstack export: docs/reference/cli/dstack/export.md
               - dstack import: docs/reference/cli/dstack/import.md
           - HTTP API:

mkdocs/docs/concepts/endpoints.md

@@ -0,0 +1,154 @@
+---
+title: Endpoints
+description: Creating and reusing optimized model inference recipes
+---
+
+# Endpoints
+
+An endpoint configuration is a new `dstack` feature that lets you use an agent to create presets: validated and optimized model inference endpoint recipes. Once a preset is created, it can be reused to deploy model inference on validated hardware without an agent.
+
+The value of presets comes from combining two fundamental features: agent-driven model inference optimization and the `dstack` [service](services.md) primitive, which can deploy model inference to any cloud, Kubernetes, or on-prem cluster.
+
+> The endpoints feature is experimental and may change.
+
+??? info "Prerequisites"
+    Before using endpoint presets, make sure you’ve [installed](../installation.md) the server and CLI, and created a [fleet](fleets.md).
+
+    Creating an endpoint preset requires the `claude` CLI to be installed on the machine where you create a preset.
+
+## Define an endpoint
+
+Before you can create or reuse an endpoint preset, you first have to define an endpoint configuration. The filename must end with `.dstack.yml`.
+
+<div editor-title="endpoint.dstack.yml">
+
+```yaml
+type: endpoint
+name: qwen25-7b
+
+model:
+  base: Qwen/Qwen2.5-7B-Instruct
+
+env:
+  - HF_TOKEN
+```
+
+</div>
+
+Since `base` is specified, the preset can use any compatible variant of the base model, including a different precision, quantization, or trusted fork.
+
+If you want to deploy an exact model, set `model` directly to the repo of that model:
+
+<div editor-title="endpoint.dstack.yml">
+
+```yaml
+type: endpoint
+name: qwen25-7b
+
+model: Qwen/Qwen2.5-7B-Instruct
+
+env:
+  - HF_TOKEN
+```
+
+</div>
+
+Set `context_length` to require a minimum context length. Placement properties,
+including `fleets`, `backends`, `max_price`, and `spot_policy`, constrain both
+creation and reuse. Environment variables such as `HF_TOKEN` can be passed
+through `env`.
+
+See the [reference](../reference/dstack.yml/endpoint.md) for all supported configuration options.
+
+## Create a preset
+
+To create a preset, pass the configuration file to the `dstack endpoint preset create` command:
+
+<div class="termy">
+
+```shell
+$ dstack endpoint preset create -f endpoint.dstack.yml
+```
+
+</div>
+
+This command executes entirely locally and uses the locally installed `claude` CLI along with `dstack`'s bundled skills. The agent uses a `dstack` task to find the best serving recipe matching the available fleet offers. Once the recipe is found, it submits a `dstack` service for a final benchmark. The validated recipe is saved locally under `~/.dstack/presets`.
+
+??? info "Claude authorization"
+    Preset creation supports two Claude authorization methods. To use an Anthropic API key, set:
+
+    ```shell
+    export DSTACK_AGENT_ANTHROPIC_API_KEY=...
+    ```
+
+    If you are already logged in with `claude`, use the existing authorization instead:
+
+    ```shell
+    export DSTACK_AGENT_CLAUDE_USE_EXISTING_AUTH=1
+    ```
+
+    These options are mutually exclusive.
+
+## List presets
+
+Use `dstack endpoint preset` to list existing presets:
+
+<div class="termy">
+
+```shell
+$ dstack endpoint preset list
+ MODEL                       GPU
+ Qwen/Qwen2.5-7B-Instruct
+    recipe=8f3a12c4          L4:24GB:1..
+```
+
+</div>
+
+Each base model corresponds to one preset. A preset may contain multiple recipes built for specific hardware. Each recipe includes benchmark data.
+
+Pass `-v` to include the full service resources and verified context length.
+
+## Apply a preset
+
+To deploy a preset as a service, pass the endpoint configuration to the `dstack endpoint preset apply` command:
+
+<div class="termy">
+
+```shell
+$ dstack endpoint preset apply -f endpoint.dstack.yml
+```
+
+</div>
+
+If you don't pass `--recipe ID` (or specify it in the endpoint configuration), `dstack` automatically picks one of the recipes that matches the available fleet offers.
+
+If a recipe matches, `dstack` deploys it as a service.
+
+## Delete a preset
+
+You can delete a specific recipe or the entire preset.
+
+<div class="termy">
+
+```shell
+dstack endpoint preset delete --recipe 8f3a12c4
+```
+
+</div>
+
+To delete the entire preset and all its recipes, pass the base model:
+
+<div class="termy">
+
+```shell
+dstack endpoint preset delete Qwen/Qwen2.5-7B-Instruct
+```
+
+</div>
+
+For command options and agent settings, see the
+[`dstack endpoint` CLI reference](../reference/cli/dstack/endpoint.md).
+
+!!! info "What's next?"
+    1. Learn how dstack [services](services.md) work
+    2. Learn how to configure [fleets](fleets.md)

mkdocs/docs/reference/cli/dstack/endpoint.md

@@ -0,0 +1,80 @@
+# dstack endpoint
+
+The `dstack endpoint` commands create, list, apply, and delete local endpoint
+[preset recipes](../../../concepts/endpoints.md).
+
+## dstack endpoint preset list
+
+The `dstack endpoint preset list` command lists locally stored recipes.
+
+##### Usage
+
+<div class="termy">
+
+```shell
+$ dstack endpoint preset list --help
+#GENERATE#
+```
+
+</div>
+
+## dstack endpoint preset create
+
+The `dstack endpoint preset create` command uses an agent to create and save a
+verified recipe from an endpoint configuration.
+
+##### Usage
+
+<div class="termy">
+
+```shell
+$ dstack endpoint preset create --help
+#GENERATE#
+```
+
+</div>
+
+##### Agent settings
+
+Set either `DSTACK_AGENT_ANTHROPIC_API_KEY` or
+`DSTACK_AGENT_CLAUDE_USE_EXISTING_AUTH` before creating a preset.
+
+| Variable | Description |
+| --- | --- |
+| `DSTACK_AGENT_ANTHROPIC_API_KEY` | Anthropic API key used by the agent. |
+| `DSTACK_AGENT_CLAUDE_USE_EXISTING_AUTH` | Use the existing `claude` login. |
+| `DSTACK_AGENT_CLAUDE_PATH` | `claude` executable name or path. Defaults to `claude` from `PATH`. |
+| `DSTACK_AGENT_ANTHROPIC_MODEL` | Claude model used by the agent. |
+| `DSTACK_AGENT_CLAUDE_EFFORT` | Claude effort level: `low`, `medium`, `high`, `xhigh`, or `max`. |
+
+## dstack endpoint preset apply
+
+The `dstack endpoint preset apply` command selects a matching local recipe and
+submits its service.
+
+##### Usage
+
+<div class="termy">
+
+```shell
+$ dstack endpoint preset apply --help
+#GENERATE#
+```
+
+</div>
+
+## dstack endpoint preset delete
+
+The `dstack endpoint preset delete` command deletes one local recipe by ID or
+all recipes for a base model.
+
+##### Usage
+
+<div class="termy">
+
+```shell
+$ dstack endpoint preset delete --help
+#GENERATE#
+```
+
+</div>

mkdocs/docs/reference/dstack.yml/endpoint.md

@@ -0,0 +1,53 @@
+# `endpoint`
+
+The `endpoint` configuration type describes a model request and the constraints
+used to create or apply an [endpoint preset](../../concepts/endpoints.md).
+
+## Root reference
+
+#SCHEMA# dstack._internal.core.models.endpoints.EndpointConfiguration
+    overrides:
+      show_root_heading: false
+      type:
+        required: true
+
+### `model`
+
+=== "Base model"
+
+    Allows the creation agent to select a compatible model variant.
+
+    #SCHEMA# dstack._internal.core.models.endpoints.EndpointModelBase
+        overrides:
+          show_root_heading: false
+
+=== "Exact model"
+
+    Requires an exact model repo or path and optionally sets another
+    client-facing model name.
+
+    #SCHEMA# dstack._internal.core.models.endpoints.EndpointModelRepo
+        overrides:
+          show_root_heading: false
+
+### `retry`
+
+#SCHEMA# dstack._internal.core.models.profiles.ProfileRetry
+    overrides:
+      show_root_heading: false
+
+### `utilization_policy`
+
+#SCHEMA# dstack._internal.core.models.profiles.UtilizationPolicy
+    overrides:
+      show_root_heading: false
+      type:
+        required: true
+
+### `schedule`
+
+#SCHEMA# dstack._internal.core.models.profiles.Schedule
+    overrides:
+      show_root_heading: false
+      type:
+        required: true

pyproject.toml

@@ -62,11 +62,19 @@ artifacts = [
     "src/dstack/_internal/server/statics/**",
 ]
 
+[tool.hatch.build.targets.sdist.force-include]
+"skills/dstack/SKILL.md" = "skills/dstack/SKILL.md"
+"skills/dstack-prototyping/SKILL.md" = "skills/dstack-prototyping/SKILL.md"
+
 [tool.hatch.build.targets.wheel]
 artifacts = [
     "src/dstack/_internal/server/statics/**",
 ]
 
+[tool.hatch.build.targets.wheel.force-include]
+"skills/dstack/SKILL.md" = "dstack/_internal/core/resources/endpoint_agent/skills/dstack/SKILL.md"
+"skills/dstack-prototyping/SKILL.md" = "dstack/_internal/core/resources/endpoint_agent/skills/dstack-prototyping/SKILL.md"
+
 [tool.hatch.metadata.hooks.fancy-pypi-readme]
 content-type = "text/markdown"