FreeLLMAPI – Open-Source AI Model Aggregation with OpenAI-Compatible API

1. What Is FreeLLMAPI

FreeLLMAPI is an open-source OpenAI-compatible proxy that aggregates free tiers from ~14 AI providers into a single API endpoint—about 1.3 billion tokens/month of free inference combined. Built with TypeScript/Express, it offers automatic failover, smart routing, encrypted keys, and usage tracking for local prototyping and personal experiments. Standard /v1/chat/completions and /v1/models endpoints work with the official OpenAI SDK, LangChain, LlamaIndex, etc.—change base_url and go.

Technical positioning and domain: AI infrastructure—API proxy and aggregation for developers juggling multiple free quotas and inconsistent interfaces. A lightweight, self-hosted gateway normalizing scattered free resources.

Development background: Created by Tashfeen Ahmed to fix "scattered free credits, messy management, incompatible APIs" during prototyping. TypeScript/Express plus React/Vite dashboard for ops visibility.

Core value: Lowers the bar to try AI—one place for many free keys without per-platform adapters. Failover and routing improve uptime and stability.

Technical characteristics: AES-256-GCM for upstream keys at rest; per-(platform, model, key) RPM/RPD/TPM/TPD tracking picks healthy, non-exhausted credentials automatically.

2. Key Features

• OpenAI-compatible proxy: Standard /v1/chat/completions and /v1/models—drop-in for OpenAI SDK, LangChain, LlamaIndex via base_url only.

• Multi-provider free aggregation: Google Gemini, Groq, Cerebras, SambaNova, NVIDIA NIM, Mistral, OpenRouter, GitHub Models, HuggingFace, Cohere, Cloudflare, Z.ai, Moonshot Kimi, MiniMax, etc.—~1.3B tokens/month free combined.

• Automatic failover: On 429/5xx or timeout, cooldown and retry up to 20 times down the priority chain.

• Smart rate and quota tracking: RPM, RPD, TPM, TPD per (platform, model, key)—always route to healthy keys within limits.

• Sticky session locking: Multi-turn chats lock the same model for 30 minutes—avoids mid-conversation model switches and quality/context breaks.

• AES-256-GCM key encryption: Upstream keys encrypted in SQLite; decrypted in memory only for requests, then cleared.

• Unified bearer token: Clients use one freellmapi-… token—no upstream keys in app code.

• SSE streaming: Full stream: true Server-Sent Events for token-by-token output.

• Function calling: OpenAI-style tools/tool_choice for agent workflows.

• Dashboard and Playground: React/Vite UI—drag fallback priority, test models, view 24h/7d/30d latency, success rate, token stats.

3. How to Use

1. Requirements: Node.js 16+; Windows/macOS/Linux. ~40MB RSS idle; runs on Raspberry Pi ARM.

2. Install: git clone https://github.com/tashfeenahmed/freellmapi.git, cd freellmapi, npm install.

3. Configure: cp .env.example .env; generate 32-byte AES key: openssl rand -hex 32.

4. Start: npm run dev—API default :3001, dashboard :5173.

5. Add upstream keys: Open http://localhost:5173 → Keys → paste Gemini, Groq, etc.; auto validation.

6. Fallback chain: Drag provider priority on Fallback Chain page.

7. Unified token: Generate freellmapi-… bearer in admin panel.

8. SDK: Set base_url="http://localhost:3001/v1", model="auto" or specific model name.

9. cURL: POST to http://localhost:3001/v1/chat/completions with Authorization: Bearer freellmapi-….

10. Daemon: PM2/systemd for long-running on ARM, e.g. pm2 start npm --name "freellmapi" -- run dev.

4. Pros and Cons

5. Comparison with Similar Tools

Selection advice: Personal devs maximizing free tiers for prototypes—FreeLLMAPI. Enterprise paid models and multi-tenant—LiteLLM. CN teams needing domestic models and billing—One API.

6. Editor's Review

FreeLLMAPI innovates by bundling free AI credits into one standard API—solving quota sprawl and adapter fatigue for solo builders. TypeScript/Express fits event-driven concurrency; AES-256-GCM is a thoughtful security touch rare at this scale.

~1.3B free tokens/month covers most experiments. Failover, rate tracking, dashboard, and Playground lower ops friction.

Best for individuals, students, and researchers—not enterprises lacking tenancy and cost controls.

Future growth needs more providers (paid enterprise), multi-tenant features, finer cost tracking, and richer docs/plugins.

— Excellent for personal free-tier aggregation; weaker on enterprise features and ecosystem scale.

7. Use Cases

• Local prototyping: Stack free tiers behind one OpenAI endpoint—iterate without paid API bills.

• Multi-platform free routing: Gemini, Groq, GitHub Models, Z.ai, etc.—high volume on a budget (generation, code, translation).

• Low-power edge gateway: ~40MB RSS on Pi/ARM via PM2—home LAN AI proxy for assistants and smart home.

• OpenAI SDK learning: Standard endpoints to learn LangChain/LlamaIndex with base_url swap—no OpenAI spend.

• Model shootouts: One interface + Playground to compare free providers on quality, latency, and style.

8. FAQ

Q: Is FreeLLMAPI completely free?
A: The proxy is free OSS—you bring each platform's free API keys. FreeLLMAPI adds no credits; usage debits each provider's free quota.

Q: How to get upstream keys?
A: Register on each provider (Gemini, Groq, Cerebras, SambaNova, NIM, Mistral, OpenRouter, GitHub Models, HuggingFace, Cohere, Cloudflare, Z.ai, Kimi, MiniMax, etc.) and read their free-tier terms.

Q: Custom providers?
A: ~14 built-in; extend via config (endpoint, models, auth)—docs are thin; needs technical comfort.

Q: Key security?
A: AES-256-GCM in SQLite; decrypt in RAM per request only. Clients use freellmapi-… bearer, not raw upstream keys.

Q: Streaming and tools?
A: Yes—SSE stream: true and OpenAI-style tools/tool_choice.

Q: Production-ready?
A: Better for local experiments—no multi-tenant, cost tracking, or enterprise auth. Production often uses LiteLLM or One API.

9. Project Links

• GitHub: https://github.com/tashfeenahmed/freellmapi