Building a Free-Tier Multi-Provider LLM Gateway with LiteLLM: A Complete Setup Guide
Overview This guide walks through setting up a self-hosted LiteLLM proxy that unifies multiple free-tier LLM providers (NVIDIA NIM and OpenRouter) behind a single OpenAI-compatible API endpoint. It also covers automated model-availability probing and integrating the resulting gateway with an AI coding CLI. No API keys, account names, personal identifiers, or real secrets are included anywhere in this guide. All example values are placeholders — replace them with your own credentials. Why build this Free-tier LLM providers often enforce low per-minute rate limits (e.g., 20-40 requests/minute). Model catalogs change frequently — models get added, deprecated, or temporarily rate-limited. A single unified gateway lets any OpenAI-compatible client (coding CLIs, IDE extensions, custom scripts) fail over automatically across multiple providers/models without code changes. Part 1: Probing Model Availability Before wiring a provider into a gateway, verify which models are actually ...