AI Interactivity (Part II): Multi-Agent Systems and AI Copilots

AI Interactivity (Part II): Multi-agent Systems and AI Copilots

Jackson Chen, Tensility Intern and MBA candidate at Northwestern University Kellogg School of Management
Armando Pauker, Managing Director at Tensility Venture Partners
Wayne Boulais, Managing Director at Tensility Venture Partners

Our focus at Tensility Venture Partners has been on early-stage AI investment for several years. We are particularly drawn to ventures where founders aim to disrupt the status quo, forge new markets, and make the world better through AI. We believe that overarching AI systems are essential enablers to develop sophisticated software solutions that can transform industry norms.

With that in mind, we explored AI Agents and Multimodal Agents, an advanced variant, in the first part of this blog series. Here we shift our focus to collaboration across AI models, spotlighting Multi-agent Systems (MAS) and the increasingly popular concept of AI Copilots. Despite similarities, these two solutions exhibit fundamental differences, particularly in their frameworks, which we'll examine. However, both MAS and AI copilots are adept at addressing complex challenges requiring collaborative efforts across multiple "roles," showcasing their capacity for sophisticated problem-solving.

“Two heads are better than one” - Multi-agent Systems

While an AI Agent, especially a multimodal one, has advantages as noted in part I, there are technical challenges, particularly in response consistency and accuracy. MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) recently proposed a solution[1]: leveraging a network of AI Agents, each contributing unique perspectives to a query. This approach not only enhances the quality of responses but also allows models to refine their outputs by analyzing feedback from their peers. This has sparked a surge in the development and application of a multi-agent system. So, what is MAS and how does it function?

What is a Multi-agent system?

Multi-agent systems are advanced computational constructs where AI agents, working autonomously and independently, collaborate to complete tasks and achieve a common goal (or “objective”, which is set by humans)[2]. Within this setup, each agent is tasked with specific roles (for instance, one agent might act as a "writer" while another serves as a "safeguard" during coding tasks). These agents could be software programs, robots, or any form of computational entities, equipped with the capability to learn, adapt, and make decisions by assessing their surroundings and the actions of fellow agents.

How does a Multi-agent system work?

A MAS framework, specifically its organization and the kind of agent interaction[3], determines how the numerous AI agents work together, while each one tackles distinct tasks independently.

Organization: organization refers to the structural layout and hierarchy within the MAS. It defines roles (e.g., leader, coordinator, worker, etc.), responsibilities, and interaction patterns. There could be different types of organization, such as:

Flat: Agents interact on equal footing without hierarchy.
Hierarchical: Agents are ranked, with higher-level agents guiding lower-tier ones.
Networked: Agents form a web of connections, some more influential than others

Interaction: based on the organization of MAS, AI agents could communicate, collaborate, or compete to achieve their objectives:

Cooperation: Agents collaborate to achieve shared goals, pooling resources and information
Coordination: Agents align their actions to use resources efficiently and avoid conflicts
Negotiation: Agents dialogue to resolve disputes or make collective decisions
Competition: Agents operate in opposition, especially when resources are scarce or goals conflict

Notable MAS frameworks

In September 2023, Microsoft introduced "AutoGen", a MAS framework designed to streamline the management, enhancement, and automation of Large Language Model (LLM) processes[4]. Microsoft's AutoGen illustrates the creation of an intricate multi-agent conversation system by firstly, appointing agents with distinct skills and roles, and secondly, establishing how these agents interact, specifically how they respond to messages from one another. In this system, a leading agent, referred to as the "Commander" (or alternatively, "Coordinator" or "Manager"), handles incoming prompts (such as questions) from users, tailoring these prompts to suit the specific expertise of each agent. The Commander selects the most suitable agent to initially address the user's query, then circulates the question among other agents for their input. Ultimately, these agents collectively reach a "consensus" and provide a unified response to the user.

Another notable MAS framework is MetaGPT[5], an open-source project developed by researchers from DeepWisdom, The Swiss AI Lab, and several top universities worldwide. MetaGPT is an advanced MAS framework designed to generate a range of software development outputs such as user stories, competitive analysis, and more. It simulates the roles of an entire software company, including product managers and engineers, streamlining the development process with predefined standard operating procedures or SOPs (see illustration below). MetaGPT embodies a virtual software development team, automating the creation process of APIs, data structures, etc. from initial concept to final documentation, all initiated by a simple description of the software development requirements.

MAS vs. Ensemble Models

Ensemble Models, akin to multi-agent systems, harness "collective wisdom" but with a distinct focus on enhancing prediction accuracy by combining multiple predictive models, known as base estimators. Unlike MAS, which emphasizes the interaction among intelligent agents to address complex issues, ensemble models aim to consolidate predictions from various models to boost overall prediction accuracy and robustness, effectively mitigating challenges like low accuracy and high variance that single models face[6].

Potential MAS Applications[7]

Multi-agent systems are the next frontier for business innovation, applications include:

Streamlining supply chains: Linked agents facilitate seamless cooperation between supply chain participants like manufacturers, distributors, and retailers. By optimizing specific, collaborative tasks, businesses can eliminate bottlenecks, lower storage costs, and enhance service to customers.
Cybersecurity: MAS offers a dynamic approach to fortifying enterprise defenses. By allocating different cybersecurity tasks—such as threat detection and incident response—to specific agents, MAS can enhance the efficiency and effectiveness of security operations. This division of labor not only accelerates response times but also enables a more nuanced and comprehensive approach to mitigating cyber threats.
Smart energy grid management: The collaborating agents can enhance the operation of smart energy grids, effectively balancing supply and demand, optimizing resource allocation, and ensuring reliability. Agents can monitor and control different, specific aspects of the grid, such as generation, distribution, and consumption.

From Multi Agent Systems to AI Copilots

The AI copilot framework is gaining momentum in the AI field, distinguishing itself from MAS by its operational approach. A short summary of the definitions of MAS and AI copilots are:

MAS: relies on a network of AI agents who can work autonomously (i.e., without human interactions) based on their roles and collaborate with each other to achieve objectives set by humans
AI copilots: utilize conversational interfaces powered by LLMs to facilitate natural, human-like conversational interactions. It is designed to help users with various tasks, often providing guidance, support, and automation in different contexts[8]

The term “copilots” has gained recent public prominence with Microsoft's initiative to integrate the GPT-powered AI assistant, branded as "Copilot," across its extensive range of products. Microsoft has integrated copilots across its Microsoft 365 suite (encompassing Word, PowerPoint, Excel, Teams, and Outlook) and its enterprise SaaS solutions like Power Platform. Similarly, Google has incorporated its Gemini-powered AI assistant, Duet AI, into its productivity tools, including Google Docs, Sheets, and Meet.

Applications of AI Copilots

The primary use of AI copilots lies in boosting human efficiency by allowing collaboration between users and the AI systems. Beyond general productivity tools, AI copilots are branching out into sector-specific applications as well:

General Productivity Enhancement: an AI copilot is capable of enhancing human productivity by responding to questions, providing recommendations, helping draft content naturally, etc. Aside from the Microsoft Copilot and Google Duet AI mentioned earlier, OpenAI’s Assistants API allows users to create an Assistant that can access tools such as Code Interpreter, Retrieval and Function calling to help users solve specific questions.
Customer Relationship Management (CRM): AI copilots can boost sales productivity by automating content insights generation for humans’ reviews. For instance, Salesforce’s Einstein GPT generates personalized content across every Salesforce instance, helping Salesforce customers elevate CRM efficiency with out-of-the-box generative AI.
Quick Service Restaurants (QSRs)[9]: One of Tensility’s portfolio companies, ConverseNow, is changing the way orders are taken in the quick serve restaurant (QSR) industry. Specifically, ConverseNow's Voice AI uses a unique copilot approach to offer its voice-enabled AI order-taking service. The primary flow has the AI agent handle customer orders from start to finish. However, a human representative (rep) is brought into the order taking process when the AI identifies there is a problem completing the order. The order is transferred to a rep who decides what part of the order needs clarification and sends text-based instructions back to the AI agent to ask different questions of the caller. The goal here is to maintain a standard flow so the AI can resume and complete the order. This synergy between AI agents and human representatives creates streamlined order-taking processes that dramatically enhance efficiency while still providing a quality customer experience.

ConverseNow’s Voice AI as a copilot to assist a representative in completing orders

Future developments of AI copilots

Copilots today are based on vertical industry `approaches and solutions that are trained with specific datasets and methods to allow for value added interaction. The software development copilot is an example of this where the value of the copilot depends on all the specific software code that it has been trained on.

Future development will likely evolve to allow the interaction between different instances of copilots. The graphic below imagines a future where a very complex design process, like automotive engineering, can be improved by allowing copilots to access information and insights from many players in the process. An engineer using a copilot to design a bracket for a new car could interact across departments (like costing or manufacturing engineering or part procurement) to verify the feasibility of a given design. Normally these are silos today.

The main challenge making this a reality is creating the environment for collaboration across multiple copilot vendors. The new infrastructure would require open APIs for inter-vendor networking and data sharing.

Conclusion

We are at the dawn of a transformative era with multiple AI system approaches like multi-agent systems and AI copilots. While many AI applications today still rely on single-agent or unimodal agents, the tangible benefits and business value of these collaborative approaches offer a hint at a burgeoning shift towards addressing ever more complex decision-making processes. We anticipate a significant expansion in startups creating innovative applications leveraging these collaborative approaches in the near future.

References

AI Interactivity (Part II): Multi-Agent Systems and AI Copilots