Bring Your Own Model: Why Your AI Copilot Shouldn't Lock You Into One LLM Provider

When you add an AI copilot to a platform, you are making two decisions that compound: which model to use today, and how locked in you are to that choice tomorrow. Most enterprise platforms that ship an embedded AI assistant make the second decision for you by not giving you any choice on the first.

Zerq Copilot treats model selection as an infrastructure configuration concern — not a product decision made on your behalf. You configure your preferred provider once, per environment. The rest of the platform does not care which provider you chose.

The problem with single-provider AI assistants

Enterprise software teams have a well-founded instinct for avoiding vendor lock-in in their infrastructure. When a single vendor controls your database, your message queue, and your compute layer, the negotiating position on pricing and the cost of switching both change unfavorably.

AI infrastructure is following the same pattern — faster, because the market is moving faster. Teams that embedded a single-provider AI assistant in 2024 are now discovering:

Model capability is shifting rapidly. The best model for a platform operations task in Q1 might not be the best choice in Q3. The ability to switch without a product upgrade or a contract renegotiation is materially different from being locked in.

Data residency requirements vary by environment. A commercial SaaS environment might use a US-based LLM endpoint. A deployment in a European regulated environment might require a regional Azure OpenAI endpoint. An air-gapped government deployment might require a self-hosted Ollama instance running a locally downloaded model. If the copilot is hardcoded to one provider, none of these deployments are possible without forking the product.

Your existing AI procurement may already cover this. Many enterprises have enterprise agreements with one or two LLM providers — agreements that include data processing terms, compliance certifications, and negotiated pricing. If the AI copilot on your API platform cannot use those agreements, you are either double-paying or running AI infrastructure outside your procurement policy.

Credentials in the browser are a security antipattern. Some embedded AI features are implemented by passing an API key to the browser so the browser can call the LLM directly. This is fast to implement. It is also a credential exposure problem: the key is visible in browser dev tools, in network logs, and in any browser extension with page access. If the key has any capability beyond generating text for the UI, the surface is wider than it appears.

How Zerq Copilot handles model configuration

Zerq Copilot uses a server-side model configuration approach. You configure an AI_MODELS block in your environment settings. The platform reads this at startup and routes all LLM calls through the server, never the browser.

The browser sends a prompt. The server receives it, calls the configured LLM provider using the server-side key, and returns the response. The LLM credential never appears in browser network traffic.

The configuration supports:

OpenAI — GPT-4o, o1, or any available model
Anthropic — Claude Sonnet, Haiku, or Opus
Google Gemini — Gemini 1.5 Pro, Flash, or other available models
Groq — for high-throughput inference use cases
Azure OpenAI — for enterprise agreements, regional endpoints, and compliance requirements
Amazon Bedrock — for AWS-native deployments and teams with Bedrock in their infrastructure stack
Ollama — for air-gapped environments, local development, or teams running self-hosted models
OpenRouter — for unified access to a wide model catalog
Any OpenAI-compatible endpoint — for custom deployments, fine-tuned models, or inference infrastructure you manage directly

Swapping providers is an environment configuration change, not a product upgrade. The conversational interface and the Management MCP integration layer are provider-agnostic — they do not change when you change the LLM behind them.

The air-gap case: a distinct requirement

For government, defence, and regulated financial services deployments running in air-gapped or network-restricted environments, the model provider question has a binary answer: the LLM must run locally.

A copilot that is hardcoded to an internet-connected provider cannot be deployed in these environments. The choice is not "which cloud LLM?" — the choice is "can this run on Ollama with a locally-hosted model?"

Zerq Copilot's Ollama support is not an afterthought for developer convenience. It is the mechanism that makes Copilot deployable in environments where no outbound internet connection is available. The same Management MCP integration, the same RBAC enforcement, the same audit trail — running against a model that never makes an outbound call.

The compliance case: data processing boundaries

For regulated industries, the question is less about internet connectivity and more about where data goes and whose infrastructure processes it.

A prompt sent to a copilot for operations tasks might include API endpoint names, error messages, traffic metrics, or partial request payloads. Depending on your data classification, those contents might be regulated. If the LLM call goes to a consumer-tier cloud API, you may be processing regulated data outside your data processing agreements.

The Azure OpenAI and Amazon Bedrock options exist specifically for this case: they give regulated enterprises access to capable models through cloud infrastructure that is already part of their compliance posture — with the data processing terms, geographic boundaries, and audit certifications that enterprise compliance programs require.

One configuration policy, two Copilot experiences

Zerq Copilot has two experiences: Management Copilot for platform operators and Gateway Copilot for developer portal consumers. Both use the same server-side AI_MODELS configuration.

This means model selection is a single policy decision for each environment — not a per-experience configuration with different providers for different user types. Whatever model and provider you have approved for your platform's AI usage applies uniformly to both experiences. One procurement decision. One data processing agreement. One security review.

When to think about this

If you are evaluating an AI copilot for your API platform, the model provider question is worth asking early:

Can you use the LLM providers you already have enterprise agreements with?
Can the copilot run in an air-gapped environment if your deployment requires it?
Does the implementation keep LLM credentials on the server, or do they appear in browser network traffic?
If the best available model changes in twelve months, what is the process to switch?

These are not questions about model quality. They are questions about whether the AI infrastructure on your platform is subject to the same governance as the rest of your infrastructure — or whether it is a special exception that your security and compliance teams have to maintain separately.

Zerq Copilot ships with the Zerq enterprise platform. Model configuration is per-environment and provider-agnostic. See the Copilot product page for the full provider list and configuration options, or request a demo to discuss your specific model provider requirements.