🧠 Cortex

The AI Infrastructure Stack for Production

Rook — AI Proxy & Router · Huginn — Local Coding Agent

---

What is Cortex • Products • Why Cortex • Quick Start • Architecture • Docs

---

🎯 What is Cortex?

Cortex is a production-ready AI infrastructure stack built in Rust that handles everything between your application and the AI providers.

┌──────────────────────────────────────────────────────────────────────────────┐
│                                                                              │
│    Your App                    Cortex                    AI Providers         │
│  ┌────────┐              ┌────────────┐           ┌─────────────────┐   │
│  │        │              │            │             │                 │   │
│  │  Any   │────────────▶│   Rook     │───────────▶│  OpenAI         │   │
│  │  SDK   │   OpenAI    │   Proxy    │             │  Anthropic      │   │
│  │        │   Compatible│            │             │  Gemini         │   │
│  │        │◀────────────│  + Cache   │◀───────────│  Ollama         │   │
│  └────────┘   Response   │  + Audit   │  Fallback   │  Groq           │   │
│                          └────────────┘             └─────────────────┘   │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

The Two Products

Product	Purpose	What it does
🛡️ Rook	AI Proxy	Routes requests, handles fallbacks, caches responses, logs everything
🤖 Huginn	Coding Agent	Your local AI coding partner with native tool access

---

📦 Products

🛡️ Rook — AI Proxy & Router

"Never let an AI outage break your app again."

Rook is an OpenAI-compatible proxy that sits between your app and AI providers.

Request
    │
    ▼
┌─────────────────────────────────────────────┐
│              Rook Proxy                      │
│                                             │
│  1. Authenticate API key                   │
│  2. Log request (audit)                    │
│  3. Check cache (return if hit)            │
│  4. Route to primary provider               │
│     │                                      │
│     ├── Success → Cache → Response         │
│     │                                      │
│     └── Failure → Try next provider         │
│              │                              │
│              └── All fail → Error response  │
└─────────────────────────────────────────────┘

Key Features

Feature	Description
🔌 OpenAI Compatible	Drop-in replacement for any OpenAI client
🔄 Automatic Fallbacks	Define combo chains: gpt-4 → claude-3 → gemini-pro
🏷️ Model Aliasing	Use gpt-4-turbo → maps to actual model in config
⚡ Smart Caching	TTL-based response caching to cut costs
📊 Full Audit Trail	Every request/response logged to SQLite
🔑 API Key Management	Create, revoke, rotate keys from dashboard
🚀 Streaming	SSE streaming with tool call support
🌐 Web Dashboard	Vue 3 SPA embedded in binary

Providers Supported

┌────────────────────────────────────────────────────────────────┐
│                                                                │
│   OpenAI          Anthropic          Google Gemini              │
│   ├─ GPT-4o      ├─ Claude 3.5      ├─ Gemini 1.5 Pro        │
│   ├─ GPT-4       ├─ Claude 3       └─ Gemini 1.5 Flash       │
│   └─ GPT-3.5     └─ Claude 2                                   │
│                                                                │
│   Groq            Ollama              Custom (OpenAI-like)     │
│   ├─ Llama 3.1   ├─ Llama 3         ├─ LM Studio             │
│   └─ Mixtral     ├─ Mistral         └─ Local endpoints       │
│                  └─ CodeLlama                                │
│                                                                │
└────────────────────────────────────────────────────────────────┘

📖 Docs · API Reference · Dashboard · Configuration

---

🤖 Huginn — Local Coding Agent

"Your AI coding partner that respects your privacy."

Huginn is a Rust-powered coding assistant that runs locally with access to your codebase.

┌─────────────────────────────────────────────────────────────────────────┐
│                                                                          │
│   You: "Refactor the auth module to use async/await"                    │
│                                                                          │
│   Huginn Processing:                                                     │
│                                                                          │
│   ┌─────────────┐     ┌──────────────┐     ┌─────────────────────┐   │
│   │  Parse      │────▶│  LLM Call   │────▶│  Execute Tools      │   │
│   │  Intent     │     │  (Provider) │     │                     │   │
│   └─────────────┘     └──────────────┘     ├─────────────────────┤   │
│                                              │ Read auth/*.rs     │   │
│                                              │ Edit files         │   │
│                                              │ Run cargo check    │   │
│                                              └─────────────────────┘   │
│                                                                          │
│   Response: Here's the refactored auth module...                         │
│                                                                          │
└─────────────────────────────────────────────────────────────────────────┘

Key Features

Feature	Description
💻 Native TUI	Beautiful terminal interface with Ratatui
🔒 Privacy First	With local providers, code stays local; remote providers receive prompts and file context
🛠️ Native Tools	File ops, terminal, git — all built-in
🔌 MCP Integration	Connect to any MCP stdio server
💾 Session Memory	SQLite-backed conversation history
📋 Scoped Context	Per-repo AGENTS.md for instructions

📖 Docs · TUI Guide · MCP Setup

---

✨ Why Cortex?

The Problem

Building with AI is fragile:

Pain Point	Impact
Provider goes down	Your app breaks
Need to switch models	Code changes everywhere
Audit requirements	No visibility
Cost spikes	Budget surprises
Multiple providers	SDK hell

The Solution

┌─────────────────────────────────────────────────────────────────────┐
│                         With Cortex                                │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   ❌ Provider down?     ──▶  ✅ Automatic failover                  │
│   ❌ Hard-coded models  ──▶  ✅ Model aliases                       │
│   ❌ No audit trail     ──▶  ✅ Full request logging                │
│   ❌ SDK complexity     ──▶  ✅ Single OpenAI-compatible API        │
│   ❌ Cost overruns     ──▶  ✅ Response caching                     │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Performance

Metric	Value
Response Time	<5ms overhead (proxy latency)
Cache Hit	~50ms (vs 500ms+ API call)
Memory Footprint	~20MB (Rook)
Binary Size	~15MB (static, musl)

---

🚀 Quick Start

Prerequisites

┌────────────────────────────────────────────────────────────────┐
│  Tool        │  Version  │  Install                           │
├──────────────┼───────────┼────────────────────────────────────┤
│  Rust        │  1.81+   │  curl --proto '=https' -sSf \       │
│              │          │    https://sh.rustup.rs | sh        │
│  Node.js     │  24+     │  nvm install 24                     │
│  pnpm        │  latest  │  npm i -g pnpm                      │
│  Docker      │  latest  │  docker.com/get-started             │
└────────────────────────────────────────────────────────────────┘

Install Rook

# Clone the repo
git clone https://github.com/dallay/cortex.git
cd cortex

# Install dashboard dependencies
pnpm install --dir apps/rook/dashboard

# Build Rook (with embedded dashboard)
cargo build --release -p rook

# Run it
ROOK_CONFIG=./rook.toml cargo run --release -p rook

Configure Rook

# Create config file
cat > rook.toml << 'EOF'
[server]
host = "127.0.0.1"
port = 3773

[cache]
enabled = true
ttl_secs = 3600
EOF

Note: Providers are configured via the REST API at /api/providers after startup. See Configuration for full configuration options.

Test It

# Health check
curl http://localhost:3773/health

# OpenAI-compatible chat completions (requires API key authentication)
curl http://localhost:3773/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Or disable auth for testing (development only):
# Add to rook.toml:
# [auth.api_keys]
# enabled = false

Open Dashboard

# Dashboard is embedded at /dashboard/
# macOS:
open http://localhost:3773/dashboard/
# Linux:
xdg-open http://localhost:3773/dashboard/
# Or just copy the URL to your browser

# Login with admin credentials (set on first run)

---

🏗️ Architecture

Project Structure

cortex/
│
├── apps/
│   ├── rook/                    # 🛡️ Rook binary
│   │   ├── src/
│   │   │   ├── main.rs          # Entry point
│   │   │   ├── server.rs        # Axum server
│   │   │   └── routes/          # HTTP handlers
│   │   └── dashboard/           # 📊 Vue SPA (embedded)
│   │       ├── src/views/       # Page components
│   │       ├── src/components/ # UI components
│   │       ├── src/stores/     # Pinia state
│   │       └── e2e/            # Playwright tests
│   │
│   └── huginn/                 # 🤖 Huginn binary
│       └── src/
│
├── crates/
│   │
│   ├── domain/                  # 💼 Business logic (zero deps)
│   │   ├── shared-kernel/       # Core types & traits
│   │   └── rook-core/          # Domain models & ports
│   │
│   ├── application/            # 🎻 Use cases
│   │   └── rook-usecases/      # Business rules
│   │
│   ├── infrastructure/         # 🔌 External adapters
│   │   ├── providers-openai/   # OpenAI client
│   │   ├── providers-anthropic/# Anthropic client
│   │   ├── providers-gemini/   # Google Gemini client
│   │   ├── providers-ollama/   # Ollama client
│   │   ├── providers-groq/     # Groq client
│   │   ├── auth-sqlite/        # API key management
│   │   ├── audit-sqlite/       # Request logging
│   │   ├── cache-memory/       # TTL cache
│   │   └── transport-axum/     # HTTP transport
│   │
│   └── huginn/                 # 🤖 Huginn runtime
│       └── runtime/
│
├── docs/                       # 📖 Documentation
│   ├── rook/                   # Rook docs
│   │   ├── architecture.md
│   │   ├── api.md
│   │   ├── configuration.md
│   │   ├── providers.md
│   │   └── deployment.md
│   └── huginn/                 # Huginn docs
│
└── dev/                        # 🐳 Testing
    ├── compose.yml             # Smoke test
    └── compose.e2e.yml         # Full E2E

Tech Stack

Layer	Technology	Why
Language	Rust 1.81+	Memory safety, performance
HTTP Server	Axum	Ergonomic, async
Runtime	Tokio	Proven async runtime
Database	SQLite	Zero-config, portable
Frontend	Vue 3 + TypeScript	Reactive, typed
Build	Vite	Fast HMR
TUI	Ratatui	Native, cross-platform
Testing	Vitest + Playwright	Fast + E2E

---

🛠️ Development

Setup

# Install dependencies
just setup

Common Commands

# Format, lint, test
just fmt
just clippy
just test

# Run locally
just run              # Rook server
just run-dashboard    # Vite dev server
just huginn-run       # Huginn

# E2E testing
just test-e2e         # Full E2E suite
just dev-up           # Smoke test container

# Full CI locally
just ci-local

Code Quality

# Rust only
cargo fmt --all
cargo clippy --all -- -D warnings

# Dashboard only
cd apps/rook/dashboard
pnpm typecheck && pnpm build && pnpm exec vitest run

📖 Development Guide · Testing Guide · Contributing

---

📚 Documentation

Guide	Description
Rook Architecture	How Rook works internally
API Reference	All HTTP endpoints
Configuration	Complete config reference
Providers	Provider setup & quirks
Deployment	Production deployment
Huginn Docs	Coding agent documentation
Dev Testing	Local testing with Docker

---

📊 Project Stats

Metric	Value
🦀 Rust Version	1.81+ (MSRV)
📦 Crates	15+ packages
🌐 Providers	5 (OpenAI, Anthropic, Gemini, Ollama, Groq)
📄 License	MIT
⭐ Stars	GitHub

---

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for:

• Development setup
• Code style guidelines
• Pull request process
• Issue templates

We follow the Rust Code of Conduct.

---

📄 License

MIT License. See LICENSE.

---

👥 Community

Channel	Purpose
💬 Discussions	Questions & ideas
🐛 Issues	Bug reports
🌐 DeepWiki	AI-powered docs

---

Built with Rust · Designed for Production · Ready for Your AI Stack