Vision

Vision-capable chat models accept images as part of the message content. Works in both OpenAI and Anthropic shapes, with the model having to actually support it (not all do).

OpenAI shape

from openai import OpenAI

client = OpenAI(
    api_key="sk-mel-<YOUR_API_KEY>",
    base_url="https://api.melious.ai/v1",
)

response = client.chat.completions.create(
    model="<VISION_MODEL_ID>",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://example.com/hamburg-harbor.jpg"},
                },
            ],
        },
    ],
)
print(response.choices[0].message.content)

Base64 data URI

When you have the bytes in memory (upload, screenshot, generated image), pass a data URI:

import base64

with open("harbor.jpg", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="<VISION_MODEL_ID>",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe what you see."},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{b64}"},
                },
            ],
        },
    ],
)

Multiple images in one turn

messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Which of these is in Hamburg?"},
            {"type": "image_url", "image_url": {"url": "https://example.com/a.jpg"}},
            {"type": "image_url", "image_url": {"url": "https://example.com/b.jpg"}},
        ],
    },
]

Anthropic shape

from anthropic import Anthropic
import base64

client = Anthropic(
    api_key="sk-mel-<YOUR_API_KEY>",
    base_url="https://api.melious.ai",
)

with open("harbor.jpg", "rb") as f:
    b64 = base64.standard_b64encode(f.read()).decode()

response = client.messages.create(
    model="qwen3.5-122b-a10b",   # a model with "image" in _meta.input_modalities — claude-* names map to a text-only model by default
    max_tokens=512,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": "image/jpeg",
                        "data": b64,
                    },
                },
                {"type": "text", "text": "Describe what you see."},
            ],
        },
    ],
)
text = "".join(block.text for block in response.content if block.type == "text")
print(text)

URL-source images also work:

{"type": "image", "source": {"type": "url", "url": "https://example.com/harbor.jpg"}}

Privacy: URLs are re-encoded server-side on Chat Completions

When you pass an external URL to /v1/chat/completions, Melious fetches the image and converts it to base64 before handing it to the provider. The provider never sees your URL — they only see the bytes. If the fetch fails (bad URL, auth required, 404), you get a 400 with type: invalid_request_error, code: invalid_request_error.

On /v1/messages, a source.type: "url" block is passed to the provider as you sent it. Send source.type: "base64" there if the URL is sensitive.

This matters because some external CDNs track request origin and some clients ship user-identifying tokens in URLs. Re-encoding breaks both — more on the data-handling posture in Privacy and data residency.

Picking a vision model

Check that "image" is in _meta.input_modalities on GET /v1/models?include_meta=true, or filter for vision at melious.ai/hub/models. We don't list the vision families here — the catalog changes faster than this page, and the hub filter is the source of truth.

Passing images to a model that doesn't list "image" in _meta.input_modalities doesn't fail — we drop the image blocks, send the text, and bill the request. A message that was only images arrives as [Image removed — this model does not support vision]. Check _meta.input_modalities before you send.

Limits

Per-image tokens are counted at the "high detail" rate, significant for long-context windows.
URL images: on Chat Completions we fetch the file ourselves — it must be ≤ 20 MB and respond within 10 s, or the request fails with 400.
Inline images: the whole request body is capped at 25 MB, base64 included.
Animated formats: we pass the bytes through unchanged. What the model does with a GIF is up to the model.

If you need layout reasoning (bounding boxes, document structure, form fields), consider Document conversion first — it produces structured markdown that a text model can then reason about, usually cheaper than visual reasoning directly.

Chat completions and Messages for full endpoint details • Models for capability discovery.

On this page