How to Connect Any LLM to Dify (OpenAI-Compatible)

Dify is not limited to OpenAI's own models. Through the first-party OpenAI-API-compatible plugin it can connect to any service that speaks the OpenAI standard — a local runtime like LM Studio or llama.cpp, or a gateway that fronts hundreds of models behind one key. The whole integration comes down to four fields: the model Type, Name, API Key, and URL (the base endpoint). This guide walks the exact path in the workspace model-provider settings, shows what to put in each field, explains how Dify actually consumes the model's output once it's wired up, and gives you a troubleshooting table for the errors that trip people up.

The short answer

Install the OpenAI-API-compatible plugin, open the model-provider settings, click Add Model, and enter four things: Type, Name, API Key, and URL (the base endpoint). That is the entire integration for any service that speaks the OpenAI standard — whether it's a local runtime like LM Studio or llama.cpp, a self-hosted model, or a gateway fronting hundreds of models. Dify's first-party plugin (langgenius/openai_api_compatible) exists precisely so you are not locked to OpenAI's own models.

Once the credentials validate, the model becomes selectable everywhere in Dify — apps, chatflows, agents, and workflow LLM nodes — and that selection is how its output gets consumed. The rest of this guide is the detail: where each field comes from, the exact base-URL format, and the errors to watch for.

Install the OpenAI-API-compatible plugin

Model providers live in the workspace under Integrations > Model Provider — and OpenAI-compatible support ships as a plugin you install there. In the Dify workspace, model providers are managed at Integrations > Model Provider (they can also be browsed via the Marketplace). Find and install the official OpenAI-API-compatible plugin — langgenius/openai_api_compatible — which lets Dify integrate with services such as LM Studio, llama.cpp, and any other OpenAI-standard endpoint.

One access note before you start: only the workspace owner and admins can manage model providers, and any connected API key applies workspace-wide — the model is shared across everyone in the workspace, not scoped to your user. If you don't see the provider settings, you likely need an admin to add the model or grant access.

The four fields you fill in

Adding a model through the plugin comes down to Type, Name, API Key, and URL — everything else is optional tuning. When you click Add Model on the provider card, Dify asks for the core parameters below; the optional fields cover completion/context/token limits and streaming and vision behaviour.

The plugin supports five categories in total — LLM, reranking, text embedding, STT, and TTS — so the same OpenAI-compatible provider can serve more than just chat completions if your endpoint exposes those routes.

Setup, step by step

Six steps take you from a blank workspace to a working model. This is the synthesized end-to-end flow; the exact button labels can shift between Dify versions, so treat the labels as guidance and match them to what your UI shows.

For the URL, one detail saves the most support tickets: enter the base URL, not the full completion path. That means the host with a /v1 suffix (for example https://host/v1) rather than https://host/v1/chat/completions — Dify appends the route itself. Pointing the field at the full path is the single most common misconfiguration. (Base-URL format verified against an independent guide, July 2026; the official docs describe the field generically as the endpoint URL.)

# Example values on the Add Model form (OpenAI-API-compatible)
Type    : LLM
Name    : claude-opus-4.8       # the model id your endpoint expects
API Key : sk-...                # your provider / gateway key
URL     : https://www.datallmlab.com/v1   # base URL, NOT /chat/completions

How Dify handles the API output

You don't wire the output up separately — once a model is configured, it becomes selectable across the platform, and selecting it is how its output is consumed. A configured model shows up in apps, chatflows, agents, and workflow LLM nodes. In a workflow, you drop an LLM node, choose your OpenAI-compatible model, and the node returns the completion as its output — which you then reference in downstream nodes (a Code node, an Answer node, a conditional branch, and so on).

So the practical answer to "how do I handle the request output from the API" in Dify is: pick the model in the component, and read the LLM node's output variable downstream. Dify normalizes the OpenAI-compatible response for you — you work with the node's text output, not raw JSON. If you enabled streaming on the model, the node streams tokens; if not, it returns the full completion. For structured downstream logic you parse the output text in a Code node like any other variable.

Default Models and fallbacks

Dify has a system-level fallback layer so components that don't name a model still work. Via the Default Models control (top-right of the workspace model settings), you can assign system defaults that kick in when an app or node doesn't specify its own model:

If you're standing up a fresh workspace, set at least the System Reasoning and Embedding defaults after adding your provider — otherwise RAG and agent features that rely on them will error out for lack of a model.

Troubleshooting model-config errors

Almost every failed Dify model connection is one of five things. This table maps the symptom to the likely cause and the fix. Because Dify validates credentials before activating a model, most of these surface at the moment you click save.

For the deeper HTTP-level view — what a 401 vs a 429 vs a 5xx actually means when your endpoint returns one — see our LLM API error-code reference. The context_size requirement for embedding/rerank types is corroborated by the Dify GitHub repo; underlying credential config there includes fields such as api_key, endpoint_url, context_size, max_tokens_to_sample, mode, and vision_support.

One key across many models

Because the custom-provider path is just base URL + API key, a gateway that exposes an OpenAI-compatible API drops straight into Dify — no special integration required. If you find yourself adding a provider card per vendor — one for Anthropic, one for a local model, one for an open-weights host — a gateway collapses that into a single provider. You point the URL field at the gateway's base endpoint and use one key; the gateway handles which upstream model actually serves each request, plus routing and failover.

DataLLM Lab is an OpenAI-compatible endpoint at https://www.datallmlab.com/v1, so you configure it exactly like any other custom provider in Dify — this is standard OpenAI-compatible setup, not a Dify-specific plugin. Add it once and the same trick works in any tool that accepts a custom base_url, from LangChain to Cursor. Browse the full model catalogue or check pricing to see what one key reaches.

FAQ