This content originally appeared on DEV Community and was authored by Radosław
Hey Dev community!
I want to share with you my recent open-source project which I am working on - Test Framework for testing Multi-AI Agent systems.
Website: Maia
Framework is written in Python and uses standard pytest approach.
The main features are:
- Multi-Agent Simulation - Simulate conversations and interactions between multiple AI agents
- Extensible Provider Model - Easily integrate with various AI model providers (e.g., LiteLLM, LangChain, CrewAI)
- Built-in Assertions - A suite of assertions to verify agent behavior, including content analysis and participation checks
- Dashboard for visualization - NextJS application to show test results for checking and debugging purpose.
You can use the framework for testing such scenarios like:
- asking various models for the same thing and check the results
- broadcasting a prompt and wait for the completion without user intervention (using not only CrewAI but also other providers!)
- simulate tool calling, so checking if your AI Agent uses your tool in a proper way
- much much more
As an example, please see how easy is to write a test:
class TestConversationSessions(MaiaTest):
def setup_agents(self):
self.create_agent(
name="Alice",
provider=GenericLiteLLMProvider(config={
"model": "ollama/mistral",
"api_base": "http://localhost:11434"
}),
system_message="You are a weather assistant. Only describe the weather.",
)
self.create_agent(
name="Bob",
provider=GenericLiteLLMProvider(config={
"model": "ollama/mistral",
"api_base": "http://localhost:11434"
}),
system_message="You are an assistant who only suggests clothing.",
)
@pytest.mark.asyncio
async def test_agent_to_agent_conversation(self):
session = self.create_session(["Alice", "Bob"])
# Alice initiates conversation with Bob
await session.agent_says("Alice", "Bob", "Given the weather: rainy and 20 degrees Celsius, what clothes should I wear?")
response = await session.agent_responds("Bob")
assert_agent_participated(session, "Bob")
# Bob responds back to Alice
await session.agent_says("Bob", "Alice", f"Based on my info: {response.content}")
response = await session.agent_responds("Alice")
assert_agent_participated(session, "Alice")
Everything is open-source and it provides basic dashboard, where you can see your tests results, including timeline, statuses, durations etc.
You can also see the assertions from the test:
The framework itself is in MVP phase, so more and more features are on the way.
Official website is here: Maia Framework
Github: Maia
PyPI: maia-test-framework
Looking forward for your feedback!
This content originally appeared on DEV Community and was authored by Radosław
Radosław | Sciencx (2025-09-03T12:00:37+00:00) Maia – Multi-AI Agent Test Framework. Retrieved from https://www.scien.cx/2025/09/03/maia-multi-ai-agent-test-framework/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.