LangChain API Key & Base URL: Set Up Any LLM

Wiring an LLM into LangChain comes down to two settings: the API key (usually an environment variable) and, for anything that is not stock OpenAI, a base URL that points the client at your provider. Get those two right and every downstream chain, agent and tool works unchanged. This guide shows the three patterns — init_chat_model for provider-native setups, ChatOpenAI with a custom base_url for any OpenAI-compatible endpoint, and pointing a single key at a gateway so you can swap models without touching code — plus the one caveat LangChain's own docs raise about third-party endpoints.

The two settings that matter

To run any LLM in LangChain you configure exactly two things: an API key and — for anything that is not default OpenAI — a base URL. Everything else in a LangChain app (chains, agents, tools, structured output) is model-agnostic once the chat model is constructed. There are three ways to construct that model:

• ChatOpenAI — the concrete class for OpenAI and any OpenAI-compatible endpoint. Set base_url to point it anywhere.
• init_chat_model — a helper that returns the right chat model for a provider:model string, so you can pick the provider at runtime.
• A gateway — one OpenAI-compatible endpoint and one key that fronts many providers, so switching model means changing a string.

The rest of this guide is those three patterns, each with the exact import and the source it comes from.

Set the API key (environment variable)

The idiomatic way is an environment variable read by the integration package — for ChatOpenAI that variable is OPENAI_API_KEY. LangChain's docs show a pattern that prompts for the key only if it is not already set:

import os, getpass

if not os.environ.get("OPENAI_API_KEY"):
    os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")

You do not have to use the env var. The key can be passed directly as the api_key parameter — LangChain's own example instantiates ChatOpenAI(model="gpt-5-mini", base_url="...", api_key="your-azure-api-key"). In production, prefer the env var (or a secrets manager) over a literal in source. Each provider package reads its own variable — the OpenAI package reads OPENAI_API_KEY, the Anthropic package reads its own, and so on.

Point at a custom base URL

To send ChatOpenAI at any OpenAI-compatible provider instead of OpenAI, set the base_url kwarg. Import comes from the langchain_openai package, and the class, key and endpoint all go on one constructor call:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="claude-opus-4.8",                 # the provider's model id
    base_url="https://www.datallmlab.com/v1",  # OpenAI-compatible endpoint
    api_key=os.environ["DATALLMLAB_API_KEY"],  # or set OPENAI_API_KEY
    temperature=0.7,
)

resp = llm.invoke("Explain a token bucket in one sentence.")
print(resp.content)

LangChain accepts openai_api_base as an alias for base_url. When both a kwarg and an env var are present, resolution runs in a fixed order:

So an explicit base_url always wins over any environment variable — handy when one process talks to several endpoints. For the wider picture of what "OpenAI-compatible" guarantees (and where it stops), see what an OpenAI-compatible API actually is.

init_chat_model for any provider

When you want to choose the provider at runtime rather than hard-code a class, use init_chat_model. It takes a model string — optionally prefixed with a provider in provider:model form — and returns the correct chat model:

from langchain.chat_models import init_chat_model

# provider:model string form
model = init_chat_model("openai:o1")

# or a bare model id (provider inferred)
model = init_chat_model("claude-sonnet-4-6")

# with inline provider kwargs forwarded to the client
model = init_chat_model(
    "openai:gpt-5-mini",
    temperature=0.7,
    base_url="https://www.datallmlab.com/v1",
    api_key="sk-...",
    max_tokens=1000,
    max_retries=6,
)

Those inline kwargs — temperature, api_key, base_url, max_tokens, max_retries — are forwarded to the underlying client, which is why init_chat_model can drive a gateway just as well as ChatOpenAI can. Provide the provider either in the string (openai:, azure_openai:) or via the separate model_provider parameter.

Gemini and other native providers

For provider-native setups — where you use the vendor's own package rather than an OpenAI-compatible shim — pass the provider to init_chat_model and set that provider's key. Documented model_provider values include openai, anthropic, azure_openai, google_genai, bedrock_converse, huggingface and openrouter. For Gemini, that is google_genai:

from langchain.chat_models import init_chat_model

# set the provider's own key env var first, e.g. GOOGLE_API_KEY
model = init_chat_model(
    "gemini-2.5-flash",
    model_provider="google_genai",
    temperature=0.7,
)

The exact Gemini model ids and the name of Google's key env var live in the provider integration docs and change over time — check the LangChain Python API reference for the current values rather than trusting a hard-coded string here. As of July 2026 the LangChain Python docs moved from python.langchain.com/docs/... to docs.langchain.com/oss/python/... (308 redirects), and the API reference to reference.langchain.com/python/....

One key for every model via a gateway

The single-key-many-models pattern is just the base-URL trick pointed at a gateway: set base_url once, and switching model becomes changing a string. Because the endpoint is OpenAI-compatible, LangChain does not need to know or care which underlying provider serves the request:

from langchain_openai import ChatOpenAI

def chat(model_id):
    return ChatOpenAI(
        model=model_id,
        base_url="https://www.datallmlab.com/v1",  # one endpoint
        api_key=os.environ["DATALLMLAB_API_KEY"],  # one key
    )

opus   = chat("claude-opus-4.8")     # Anthropic, no Anthropic key
oss    = chat("gpt-oss-120b")         # open weight
glm    = chat("glm-5.2")              # frontier open

This is standard OpenAI-compatible configuration — not a bespoke LangChain integration. Any tool that exposes a custom base URL and an API key can point at the same endpoint the same way. It is why a gateway removes provider lock-in from the LangChain layer: your chains reference model strings, and the routing lives behind one key. For what a gateway adds beyond convenience — failover, spend caps, unified logging — see what an LLM gateway is and the routing and failover guide.

The ChatOpenAI caveat to know

LangChain's official ChatOpenAI docs state the class targets the official OpenAI API spec only, and recommend a provider-specific package for third parties that extend the format. The docs name OpenRouter, LiteLLM, vLLM and DeepSeek as examples, and the only base_url example they give is Azure OpenAI. The practical implications:

• Core chat works. Messages, streaming, tool calling and structured output over an OpenAI-compatible endpoint behave as expected.
• Non-standard fields may be dropped. If a provider returns extra response fields outside the OpenAI schema, ChatOpenAI may not surface them — that is the trade-off the caveat is flagging.
• For a plain OpenAI-compatible gateway, this pattern is the standard one. A gateway that speaks the OpenAI schema faithfully is exactly the case ChatOpenAI handles well; the caveat bites hardest on providers that bolt custom fields onto the format.

If you depend on a provider's proprietary response extensions, reach for that provider's dedicated LangChain package instead. For everything schema-standard, base_url plus api_key is the route.

Provider setup at a glance

The same two settings, expressed per construction pattern. A synthesized reference for which import, key and endpoint each path needs:

Import paths, env var names, resolution order and model_provider values are from LangChain's official ChatOpenAI and overview docs (verified July 2026). Model ids shown are DataLLM Lab gateway examples; browse the full model list or pricing for current ids.

FAQ