VSLLama: A Multi-Provider, Local-First AI Coding Assistant for VS Code

If you have ever wanted a VS Code copilot that you fully control — one that talks to your local Ollama or llama.cpp models just as easily as it talks to OpenAI, Anthropic, Gemini, Perplexity, or NVIDIA NIM — that is exactly the itch VSLLama was built to scratch.

VSLLama is a VS Code extension that turns your editor into a multi-provider AI coding assistant, with local-first inference treated as a first-class citizen rather than an afterthought. It is not tied to a single vendor, a single model, or a single API key. It brings together a chat panel, inline code completions, the ability to actually execute actions in your workspace, and support for the Model Context Protocol (MCP), all inside VS Code.

Why build another AI coding extension?

Most AI coding tools force a choice: use a cloud model and give up control, or run local models and lose the polished in-editor experience. VSLLama refuses that trade-off. Underneath, it uses a clean provider abstraction, so switching between a local Ollama model running on your own GPU and a hosted model from Anthropic or Gemini is a simple configuration change, not a rewrite. Eight providers are supported out of the box: Ollama, Ollama Cloud, llama.cpp, OpenAI, Anthropic, Gemini, NVIDIA NIM, and Perplexity.

For a developer who spends time benchmarking quantized GGUF models and tuning CUDA inference, that flexibility matters. You can prototype against a fast cloud model, then flip to a local llama.cpp server to test the same prompt on your own hardware, without leaving the chat panel.

What actually happens inside the editor

VSLLama ships as a chat experience built directly into the editor sidebar, alongside an inline completion engine that suggests code as you type, powered by whichever model you have configured. Every message you send is routed to the right internal service, which builds the prompt, manages the streaming response, and decides when a tool call needs to run.

That tool layer is where things get interesting. The assistant can read files, propose edits, and execute actions inside your workspace, but only through a plan-and-approve workflow, so nothing touches your codebase without your explicit sign-off. Conversation history persists across restarts, and every provider connection is wrapped by rate limiting to keep API usage predictable.

Built-in editor actions

Beyond free-form chat, VSLLama offers commands you would reach for constantly:

• Explain selected code
• Fix bugs in a selection
• Generate unit tests
• Add documentation in JSDoc/Javadoc style
• Generate code from a plain description
• Auto-generate a conventional-commits git commit message from your staged diff, applied directly into the Source Control input box

Each of these is just a prompt fired at whichever AI client you have active, which means the quality of "explain this code" scales with the model you point it at — a small local model for quick sanity checks, or a frontier hosted model when you need deeper reasoning.

MCP: giving the assistant real tools

The part that separates VSLLama from a simple chat wrapper is its support for the Model Context Protocol. VSLLama can connect to any number of MCP servers over different transport types, cache their available tools, and let the AI actually call them — file indexers, search helpers, custom domain tools, whatever you have running. Idle connections are automatically closed after a period of inactivity to avoid leaking processes, and the tool list refreshes in the background without ever blocking the chat UI. If you are already building your own MCP servers, VSLLama becomes the front end you plug them into.

Security and session handling done properly

API keys are never written to disk in plain text — credential storage is handed off to VS Code's native secret store. Tool calls and file operations pass through sanitization, and every AI connection is rate-limited so a runaway loop cannot blow through your API quota. It is a small detail, but for anyone juggling five or six provider API keys at once, it is the difference between a toy and something you trust with real work.

Where it is headed

The current codebase already spans tens of thousands of lines, and there is an active roadmap layered on top: smarter response caching, token-aware context truncation with automatic model fallback, more resilient streaming with pause and resume, a proper file-explorer sidebar, theming, and full test coverage. The direction is clear — keep the multi-provider, local-first philosophy, but make the reliability and UX match what you would expect from a polished, professional tool.

VSLLama is, at its core, an extension for developers who do not want to be locked into one AI vendor's roadmap. Whether you are running llama.cpp on a home GPU rig or calling out to a hosted model for a hard refactor, it aims to be the same chat panel, the same commands, and the same MCP tool ecosystem — just pointed at a different brain.