How do we invoke non-OpenAI (Mistral) deployments with Microsoft Agent Framework on Azure AI Foundry?

[hexbit2]

Question: How do we invoke non-OpenAI (Mistral) deployments with Microsoft Agent Framework on Azure AI Foundry?

Hi team,

We’ve integrated the Microsoft Agent Framework with Azure AI Foundry, and OpenAI-based deployments (GPT‑4o, GPT‑4o Mini) are working perfectly. However, when we switch to our Mistral deployment the request fails with a 404 DeploymentNotFound error.

---

Context

• Stack: ASP.NET Core + SignalR hub → AgentFactory → Microsoft Agent Framework (AIAgent)
• Goal: Allow runtime model switching (GPT‑4o, GPT‑4o Mini, Mistral, etc.) via configuration only.
• Configuration: AzureOpenAIOptions captures each deployment (endpoint, API key, display name, and optional Model value for non-OpenAI providers).
• Client provisioning: AzureOpenAIClientProvider creates an AzureOpenAIClient per model at startup so we reuse the heavy client and fail fast on bad keys/endpoints.

// appsettings.json (excerpt)
"AzureOpenAI": {
  "Endpoint": "https://{resource}.openai.azure.com",
  "ApiKey": "********",
  "ChatModels": {
    "gpt4mini": {
      "DeploymentName": "gpt-4o-mini",
      "DisplayName": "GPT-4o Mini",
      "Endpoint": "https://{resource}.openai.azure.com",
      "ApiKey": "********"
    },
    "mistral": {
      "DeploymentName": "Mistral-3B",
      "DisplayName": "Mistral 3B",
      "Model": "Mistral-3B",
      "Endpoint": "https://{resource}.services.ai.azure.com",
      "ApiKey": "********"
    }
  },
  "DefaultChatModel": "gpt4mini"
}

// Singleton provisioning at startup
public AzureOpenAIClientProvider(IOptions<AzureOpenAIOptions> options)
{
    foreach (var (modelKey, modelConfig) in options.Value.ChatModels)
    {
        var client = new AzureOpenAIClient(
            new Uri(modelConfig.Endpoint),
            new AzureKeyCredential(modelConfig.ApiKey));

        _clients[modelKey] = client; // cached singleton
    }
}

// Simplified AgentFactory logic
if (!_options.ChatModels.TryGetValue(modelKey, out var modelConfig))
{
    modelKey = _options.DefaultChatModel ?? throw new ArgumentException($"Model '{modelKey}' not found.");
    modelConfig = _options.ChatModels[modelKey];
}

var chatClient = _clientProvider
    .GetClient(modelKey)                        // Cached AzureOpenAIClient
    .GetChatClient(modelConfig.DeploymentName); // e.g., "Mistral-3B"

var agent = chatClient.CreateAIAgent(
    instructions: profile.Instructions,
    name: profile.Key);

var response = await agent.RunAsync(message);

For Mistral we also attempted to set ChatClientAgentOptions.ChatOptions = new ChatOptions { ModelId = modelConfig.Model }, but the same 404 occurs.

Error Details

System.ClientModel.Primitives.ClientResultException: Request failed with status code 404 (DeploymentNotFound)
https://{resource}.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview

Azure portal shows the deployment’s target URI and explicitly states:

Use parameter model = "Mistral-3B" to use this deployment in your request.

What we’ve tried

1. Verified the deployment is live (Provisioning state: Succeeded) and accessible via the Azure AI Studio playground.
2. Confirmed the API key and endpoint (same key works for GPT‑based deployments).
3. Added ChatOptions.ModelId = "Mistral-3B" when creating the agent.
4. Tried invoking the chat client directly (await chatClient.CompleteAsync(...)) with the same result.

Question

• What is the correct way to call non-OpenAI (Mistral) deployments when using the Microsoft Agent Framework + Microsoft.Extensions.AI?
• Is there a specific place we need to set the model query parameter (or another property) so the generated request includes model=Mistral-3B?
• Are there additional requirements or client options we should be aware of for Azure AI Foundry models served from *.services.ai.azure.com?

Any guidance or sample code would be greatly appreciated. Thank you!

[runceel]

I don’t have an environment to test Mistral, so I can’t confirm it, but it seems possible by creating the appropriate IChatClient of Microsoft.Extensions.AI for each model and then calling CreateAIAgent from there.

[hexbit2]

I see, I will try it & report back to you