Architecture suggestions for large-scale AI assistant

[vladbuinceanu]

Hi everyone,

I’m integrating an AI chat assistant into an ERP platform and I could use some guidance on project structure.

Assistant goals:

• Document and legal text search
• ML tasks (classification, prediction, etc.)
• Human-in-the-loop validation before actions
• Persistent conversation and task context

I tried different examples and orchestrations patterns and while testing workflows and the handoff pattern, I noticed that each message creates a new workflow instance, causing follow-up inputs to route back to the root agent instead of resuming the last active one. I suspect a state management or human-in-the-loop layer is needed, but i don't know if this is the right approach and I don't want to start in the wrong direction.

I also found this orchestration pattern but the docs seem incomplete for now:
https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/orchestrations/magentic?pivots=programming-language-csharp

My questions are:
What’s the recommended architecture for this type of system ?
What are best practices for persisting conversation state and task data ?

Any examples, design patterns, or references would be greatly appreciated. Thanks !