Recommended Open-source stack
Open-source stack
Layer |
Recommended Options |
|
LLMs |
LLaMA 4, Qwen 3, Mixtral, DeepSeek |
|
Fast Inference |
Groq, vLLM, Ollama |
|
Agent Framework |
LlamaIndex |
|
Workflow Engine |
LlamaIndex Workflows |
|
API Layer |
FastAPI |
|
Vector DB |
Qdrant, Chroma |
|
Graph DB |
Neo4j, Memgraph |
|
SQL DB |
PostgreSQL |
|
Policy Engine |
OPA (Open Policy Agent) / Cedar |
|
Observability |
Langfuse / OpenTelemetry |
|
Queueing |
Redis / Kafka |
|
Automation |
N8N |
|
Auth |
OAuth2 / Keycloak |
|
Tool interoperability |
MCP (Model Context Protocol) — now under the Agentic AI Foundation / Linux Foundation |
|
Agent memory |
Mem0, Zep, or a self-hosted tiered store (working / episodic / semantic) |
NEW IN 2026
Anthropic donated MCP to the newly formed Agentic AI Foundation under the Linux Foundation in December 2025, with OpenAI and Block as co-founders and AWS, Google, Microsoft, Cloudflare, GitHub, and Bloomberg as supporting members. It is now a vendor-neutral standard rather than a single-vendor protocol, which is why it belongs in the stack table above rather than only in the tool layer section.
Recommended Winning Stack (2026)
Layer |
Choice |
|
API |
FastAPI |
|
Agent Framework |
LlamaIndex |
|
Workflow |
LlamaIndex Workflows |
|
Tool Interoperability |
MCP (2026-07-28 spec, stateless) |
|
Vector Memory |
ChromaDB |
|
Graph Memory |
Neo4j |
|
Agent Memory Layer |
Mem0 / Zep (tiered, benchmarked) |
|
SQL |
PostgreSQL |
|
Models |
Groq / vLLM served OSS models |
|
Policies |
OPA |
|
Observability |
Langfuse + OpenTelemetry |
|
Queue |
Redis |
|
Automation |
N8N |