Open-Weight Models

An open-weight model is one whose author publishes the trained weights for anyone to download. You can call it through a hosted provider, like any other model, or run the same weights on your own hardware.

What open-weight means

The weights are the parameters a model learns during training. With a closed model, such as GPT-5.5 or Claude, only the author and its partners can run the weights, so you can only reach the model through their APIs. With an open-weight model, the author releases the weights under a license, and any provider or person can serve them. Open-weight isn’t the same as open source. Authors often publish the weights without the training data or training code. Licenses also differ: some, like Apache 2.0 and MIT, allow almost any use, while others, like the Llama Community License, add conditions. Check the license on the author’s model card before you use a model commercially or fine-tune it.

What it means in the AI Gateway

You send requests to open-weight models the same way you send them to closed models. Being open-weight changes where the model can run:

More than one provider can serve it. When several providers in the model catalog serve the same model, the gateway can fail over between them. Use a provider:model prefix to keep requests on one provider.
You can run it yourself. Serve the weights with a tool like Ollama or vLLM, then add it as a custom provider. Your apps call it through the same gateway endpoint and access keys as your hosted models.
You can mix self-hosted and hosted copies. List your own server first and a hosted provider as a fallback, so requests keep working when your server is down.

For example, this request tries Qwen3-32B on your own Ollama server first, then on Groq:

{
  "model": "ollama:qwen3:32b",
  "models": ["groq:qwen/qwen3-32b"],
  "messages": [{"role": "user", "content": "Hello"}]
}