Multi-Agent Automation for Everyday Corporate Workflows 

Background

As an AI-focused technology company, Symphony Solutions made a strategic decision to build an internal corporate AI agent ecosystem not as a one-off automation project, but as a scalable platform for automation across the organization. 

Across teams and roles, employees were spending significant time on repetitive, manual processes with clear patterns well suited for AI-driven automation. These needs varied by function: employees needed faster access to company policies and personal HR data across multiple systems, while People Partners handled recurring 1-on-1 follow-ups, quarterly team health checks, and other repeatable workflows involving information gathering, summarization, and reporting. 

Rather than creating isolated solutions for each use case, Symphony Solutions designed a multi-agent AI platform: an extensible ecosystem where specialized AI agents can be created for different processes and departments, each with its own tools, workflows, and permission scope under one unified architecture. The goal was to support knowledge retrieval and workflow automation at scale, without requiring structural changes to the existing infrastructure. 

Client

Symphony Solutions is a Dutch IT company that stands as a distinguished brand embodying integrity, collaboration, and innovation. These fundamental principles lead the company to go beyond geographical, cultural, and technological borders, becoming a virtual-first company that draws together talent from around the globe and successfully forms cross-functional agile teams. 

Challenges

A growing number of internal processes were identified as suitable for AI-driven automation, spanning knowledge retrieval, onboarding workflows, reporting, 1-on-1 meeting summarization, and quarterly team health checks across functions such as HR, Finance, Marketing, Legal, Operations, and Recruitment. Each process had its own logic, target audience, data sources, and access requirements. 

Building a separate AI solution for each use case would have been costly, fragmented, and difficult to maintain. A unified platform approach was needed to support multiple automation scenarios under one architecture. 

The core challenges were: 

• Diverse automation needs 
  The target processes ranged from simple information retrieval (policy Q&A, personal HR data lookups) to more complex workflows (onboarding automation, meeting summarization, and team health check reporting). A single-agent approach could not reliably cover this breadth. 

• Multiple user groups with distinct roles 
  Different processes served different audiences, including employees, managers, People Partners, and other departmental roles. Each required different capabilities, workflows, and strictly controlled access to data. 

• Varied data sources and integrations 
  Some agents needed to retrieve documents from SharePoint, OneDrive, or Confluence, while others needed to interact with ERP systems, read meeting transcripts, or generate structured reports. The platform had to support diverse integration patterns within one system. 

• Extensibility without rearchitecting 
  The platform needed to support adding new agents for new processes and departments over time without modifying existing agents or disrupting the system. 

• Consistent AI quality across agents 
  Every agent, regardless of purpose, needed to deliver grounded, accurate responses with measurable quality standards and automated regression detection. 

Deliver Production-Ready AI Agents

DISCOVER MORE

Solution

Symphony’s AI team designed and implemented a multi-agent AI system, deployed as a Microsoft Teams chatbot, combining Retrieval-Augmented Generation (RAG) over corporate documentation with secure ERP integration and role-based agent orchestration. 

1. Multi-Agent Architecture with Supervisor Pattern 
Rather than building a monolithic chatbot, the team adopted a Supervisor Agent orchestration pattern using LangGraph. A central routing component (the Agent Supervisor) analyzes each incoming query for intent, checks the user’s role and group membership, and delegates the request to the appropriate specialized agent. 

Multi-Agent Orchestration: Supervisor Routing Pattern 

• Symphony Buddy Agent: Serves all employees with tools for knowledge retrieval from SharePoint, vacation balance lookups, benefits information, a la carte balance inquiries, and laptop net residual value checks. 

• People Partner Agent: Restricted to Growth Department staff (People Partners), providing specialized workflow tools such as creating ERP requests, summarizing 1-on-1 meetings, and preparing data for Team Health Check. 

• Agent X (extensibility placeholder): The architecture supports future expansion, allowing new specialized agents with distinct tool sets to be added without modifying the core orchestration logic. 

This approach was selected over peer-to-peer agent communication because it provides higher control, predictable behavior, debuggable execution paths, and enforceable authorization boundaries. In return for accurate supervisor intent classification, the system gains clear ownership of each capability and straightforward access control. 

2. RAG Pipeline and Knowledge Base 
The knowledge base is built on Qdrant, a vector database running on an Azure VM with persistent storage inside a Virtual Network. A data indexing pipeline, implemented as Azure Function Apps triggered by SharePoint webhooks, keeps the vector store synchronized with corporate documentation. 

When documents are added, updated, or removed in SharePoint, the pipeline extracts text, generates embeddings via Azure OpenAI, and updates Qdrant. When an employee asks a policy-related question, the Symphony Buddy Agent retrieves relevant document sections from Qdrant and generates a grounded answer with references to the source documents. 

The architecture targets a groundedness rate of over 90%, evaluated by LLM-as-a-Judge on production query samples. Knowledge base freshness is maintained with re-indexing occurring within one day of any SharePoint update. 

3. Authentication, Authorization, and Data Privacy 
Security was treated as a first-class architectural concern. The system integrates with Microsoft Entra ID for authentication and implements role-based access control (RBAC) to ensure users can only access data and agent capabilities they are authorized to use. 

Agent routing enforces group membership checks, so People Partner Agent tools are accessible only to Growth Department staff. All communication between services uses JSON/HTTPS, while the Qdrant vector store and LangFuse observability platform are hosted inside an Azure Virtual Network with NSG-protected subnets. 

4. Technology Stack and Deployment 
The backend is built in Python 3.12+ using FastAPI, with LangChain for LLM orchestration and LangGraph for multi-agent workflow management. The Microsoft 365 Agents SDK handles Teams integration, while Azure Bot Service manages channel routing and messaging protocol. 

5. AI Observability and Automated Evaluations 
A key part of the solution was investment in AI quality assurance. LangFuse was implemented as the observability platform, capturing complete traces of agent interactions, including token usage, response latency, and conversation quality metrics. 

Beyond production monitoring, LLM-as-a-Judge evaluations were integrated directly into the CI/CD pipeline. Evaluations run against a curated “golden dataset” stored in LangFuse, with baseline thresholds (e.g., answer correctness ≥ 0.90, faithfulness ≥ 0.95) that changes must meet to pass. 

Regression checks compare current evaluation results against the baseline, and the golden dataset enforces blocking gates: if metrics drop below thresholds, the pipeline fails. Each evaluation run is tagged with PR, commit, branch, and author metadata for full traceability. 

Result

The AI agent (Symphony Buddy) was deployed as an MVP in December 2025. Early feedback from the People Partner team and initial usage data provided the following outcomes. 

Operational Impact 

• People Partners reported that the bot has begun absorbing a meaningful share of repetitive policy and handbook questions, allowing them to redirect time toward strategic, high-value HR work. 

• Employees gained a 24/7 self-service channel for common HR inquiries directly within Microsoft Teams, eliminating the need to search through SharePoint sites or wait for a People Partner response. 

• Personalized employee data (vacation balance, sick leave balance, a la carte benefits budget, loyalty bonus balance) is served directly through the chatbot via live ERP integration, eliminating the most frequent category of manual lookups. 

• The system went from zero automation to a production AI assistant covering the full corporate knowledge base in under 3 months. 

Technical Achievements 

• Fully operational multi-agent system with supervisor orchestration, role-based routing, and extensible agent architecture, deployed and serving production traffic. 

• RAG pipeline over corporate SharePoint with automated webhook-based document indexing and vector store synchronization. 

• Comprehensive AI observability via LangFuse with full request tracing, token usage tracking, and automated LLM-as-a-Judge evaluations integrated into CI/CD. 

LangFuse: 30-Day Usage and Evaluation Volume 

• Infrastructure as Code (Terraform) with containerized deployment on Azure, network isolation via VNet/NSG, and secrets management through Azure Key Vault. 

What’s Next 

The platform roadmap includes expanded People Partner agent capabilities (team health-check summaries, automated 1-on-1 meeting notes, onboarding, offboarding), scheduled evaluation runs to proactively detect knowledge base drift, and continued onboarding of new specialized agents. The multi-agent architecture is domain-agnostic and ready for IT, finance, legal, or any knowledge-heavy function, with each new capability added as a new agent with its own tools and permissions without changes to the core orchestration layer. 

Connect AI to Your Data and Tools

SEE HOW