Skip to content

MockChatClient Prototype - #21

Closed
jeffhandley wants to merge 7 commits into
mainfrom
jeffhandley/mockchatclient-prototype
Closed

MockChatClient Prototype#21
jeffhandley wants to merge 7 commits into
mainfrom
jeffhandley/mockchatclient-prototype

Conversation

@jeffhandley

@jeffhandley jeffhandley commented Jul 24, 2026

Copy link
Copy Markdown
Owner

Replaced by dotnet#7657 after addressing all feedback here.

Summary

Adds deterministic mock AI providers for the project templates and introduces Microsoft.Extensions.AI.Testing for applications and test harnesses that need controlled chat and embedding responses.

  • Removes GitHub Models from aichatweb and aiagent-webapi; --provider is now required with no default.
  • Adds Microsoft.Extensions.AI.Testing:
    • MockChatClient supports seeded non-streaming and streaming responses, exceptions, request predicates, single-use responses, and captured request history.
    • MockEmbeddingGenerator provides repeatable lexical vector behavior for deterministic search scenarios.
  • Adds --provider mock to aichatweb with an offline conversational experience demonstrating suggestions, tool invocation, citations, and semantic search without external credentials.
    • Broad questions are answered directly; progressively specific TrailMaster and survival-kit questions cite the bundled Markdown and PDF documents.
    • Removes the rendered survey UI.
  • Adds --provider mock to aiagent-webapi with seeded writer, editor, and formatter responses for the multi-agent workflow.
  • Refreshes template snapshots and execution coverage, and corrects the testing-library README examples.

Commit structure

  1. Remove GitHub Models from aichatweb and require an explicit provider.
  2. Remove GitHub Models from aiagent-webapi and require an explicit provider.
  3. Introduce Microsoft.Extensions.AI.Testing.
  4. Add the aichatweb mock provider and remove survey UI.
  5. Add the aiagent-webapi mock provider.

Validation

  • MockChatClientTests: 18 passed.
  • aichatweb snapshots: 6 passed.
  • aichatweb execution matrix: 51 passed, 8 intentionally skipped.
  • aiagent-webapi snapshots: 5 passed.
  • aiagent-webapi execution matrix: 15 passed.
  • Fresh generated aichatweb --provider mock browser validation confirmed the seeded greeting and suggestions render without the survey prompt; product/detail questions retain their document citations.
  • Fresh generated aiagent-webapi --provider mock functional validation completed the seeded writer/editor/formatter flow without external credentials.

This prototype PR intentionally remains on the preserved pre-rebase branch. The locally rebased five-commit branch, including final README, snapshot, and survey cleanup, has not been pushed.

jeffhandley and others added 3 commits July 23, 2026 12:29
…aybooks

- Add startup stage-tree overview and per-stage progress rail to the agent's
  operating rules so each session opens with a process snapshot.
- Require concrete next-step guidance after every stage so the user always
  knows the exact prompt or command needed to advance.
- Add explicit run-location discipline: privileged and auth-sensitive commands
  use an editor-in-wait prompt for API keys and a terminal handoff block for
  AzDO/artifact/publish operations.
- Add conditional Stage 7 (support-page) policy: skip for servicing releases
  that touch only existing packages; release-manager authors the dotnet/website
  PR when new packages are introduced; monthly releases review a partner-team PR.
- Prefer the source main PR over the bot-authored backport PR in servicing
  release notes, with correct human author attribution.
- Require a tag preflight check before creating a draft GitHub release.
- Add Step 2 to the servicing preparation flow: survey the release branch and
  open PRs for existing backport activity; inform the user that the
  release-manager agent can efficiently cherry-pick from main as part of the
  servicing release process; pre-highlight identified source PRs as candidates
  in the selection table; classify already-released backports separately and
  surface them in a grounding table rather than omitting them.
- Add release wrap-up rule: present a celebratory closing message with
  per-stage wall-clock timing and an estimate of active user-interaction time.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff

@adamsitnik adamsitnik left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've looked only at selected files and they LGTM. PTAL at my comments

Comment thread src/Libraries/Microsoft.Extensions.AI.Testing/MockChatClient.cs Outdated
Comment thread src/Libraries/Microsoft.Extensions.AI.Testing/MockChatClient.cs Outdated
Comment thread src/Libraries/Microsoft.Extensions.AI.Testing/MockChatClient.cs Outdated
jeffhandley and others added 4 commits July 24, 2026 13:23
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Introduces the 'mock' provider option (--provider mock) to the aichatweb
project template. The mock provider uses Microsoft.Extensions.AI.Testing's
MockChatClient seeded with ~20 canned responses about the TrailMaster X4
GPS Watch, allowing the template to be used without any external AI provider
credentials while still demonstrating the full chat experience with citations,
suggestions, and vector-based semantic search.

Also updates the Aspire test scenario to use --provider=mock instead of
--provider=openai, and adds a test for the plain --provider=openai scenario.
Updates the TemplateSnapshotTestBase to resolve template package paths using
file timestamps rather than wildcard paths, improving cross-platform reliability.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Introduces the 'mock' provider option (--provider mock) to the aiagent-webapi
project template. The mock provider uses Microsoft.Extensions.AI.Testing's
MockChatClient seeded with 3 canned responses for the story writer/editor
workflow (generate a story, edit for tone, format as HTML), allowing the
template to be explored without any external AI provider credentials while
still demonstrating the multi-agent pipeline with writer and editor stages.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
@jeffhandley
jeffhandley force-pushed the jeffhandley/mockchatclient-prototype branch from 408b2da to 52ae066 Compare July 24, 2026 22:16
@jeffhandley

Copy link
Copy Markdown
Owner Author

Replaced by dotnet#7657

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants