Skip to content
QuantCDN

Chat inference via streaming endpoint (true HTTP streaming) with multimodal support

POST
/api/v3/organizations/{organisation}/ai/chat/stream

Streams responses from the AI streaming subdomain using Server-Sent Events (SSE). Tokens are streamed in real-time as they are generated. * * Execution Modes: * - Streaming Mode (default): Real-time SSE token-by-token responses * - Async Mode: Set async: true for long-running tasks with polling (202 response) * * Async/Durable Mode (async: true): * - Returns immediately with requestId and pollUrl (HTTP 202) * - Uses AWS Lambda Durable Functions for long-running inference * - Supports client-executed tools via waiting_callback state * - Poll /ai/chat/executions/{requestId} for status * - Submit client tool results via /ai/chat/callback * * Multimodal Support: * - Text: Simple string content * - Images: Base64-encoded PNG, JPEG, GIF, WebP (up to 25MB) * - Videos: Base64-encoded MP4, MOV, WebM, etc. (up to 25MB) * - Documents: Base64-encoded PDF, DOCX, CSV, etc. (up to 25MB) * * Supported Models (Multimodal): * - Claude 4.5 Series: Sonnet 4.5, Haiku 4.5, Opus 4.5 (images, up to 20 per request) * - Claude 3.5 Series: Sonnet v1/v2 (images, up to 20 per request) * - Amazon Nova: Lite, Pro, Micro (images, videos, documents) * * Usage Tips: * - Use base64 encoding for images/videos < 5-10MB * - Place media before text prompts for best results * - Label multiple media files (e.g., ‘Image 1:’, ‘Image 2:’) * - Maximum 25MB total payload size * - Streaming works with all content types (text, image, video, document)

Authorizations

Parameters

Path Parameters

organisation
required
string

The organisation ID

Request Body required

Chat request with optional multimodal content blocks

object
messages
required

Array of chat messages. Content can be a simple string or an array of content blocks for multimodal input.

Array<object>
>= 1 items
object
role
required
string
Allowed values: user assistant system
content
required
One of:

Simple text message

string
modelId
required

Model ID. Use Nova models for multimodal support.

string
amazon.nova-lite-v1:0
temperature
number
default: 0.7 <= 2
maxTokens

Max tokens. Claude 4.5 supports up to 64k.

integer
default: 4096 >= 1 <= 65536
topP
number
<= 1
systemPrompt

Optional custom system prompt. When tools are enabled, this is prepended with tool usage guidance.

string
stopSequences

Custom stop sequences

Array<string>
<= 4 items
responseFormat

Structured JSON output (Claude 3.5 Sonnet v1/v2, Nova Pro)

object
type
string
Allowed values: json
jsonSchema

JSON Schema defining expected structure

object
toolConfig

Function calling configuration (Claude 3+, Nova Pro)

object
tools
Array<object>
object
toolSpec
object
name
string
description
string
inputSchema
object
json

JSON Schema for function parameters

object
autoExecute

When true, backend automatically executes tools and feeds results back to AI. For async tools (e.g., image generation), returns executionId for polling. Security: Use allowedTools to whitelist which tools can auto-execute.

boolean
allowedTools

Whitelist of tool names that can be auto-executed. Required when autoExecute is true for security. Example: [‘get_weather’, ‘generate_image’]

Array<string>
sessionId

Optional session ID for conversation continuity. Omit to use stateless mode, include to continue an existing session.

string format: uuid
async

Enable async/durable execution mode. When true, returns 202 with pollUrl instead of streaming. Use for long-running inference, client-executed tools, or operations >30 seconds.

boolean
allowedTools

Top-level convenience alias for toolConfig.allowedTools. Whitelists which tools can be auto-executed.

Array<string>
[
  "get_weather",
  "generate_image"
]
guardrails

AWS Bedrock guardrails configuration for content filtering and safety.

object
guardrailIdentifier

Guardrail identifier from AWS Bedrock

string
guardrailVersion

Guardrail version

string
trace

Enable guardrail trace output

string
Allowed values: enabled disabled

Responses

200

Streaming response (text/event-stream, sync mode)

Server-Sent Events stream with chunks of generated text. Format: id, event, data lines separated by newlines.

string
Example
id: chunk-0
event: start
data: {"requestId":"abc123","model":"amazon.nova-lite-v1:0","streaming":true}


id: chunk-1
event: content
data: {"delta":"Hello","complete":false}


id: chunk-2
event: content
data: {"delta":" there!","complete":false}


id: chunk-3
event: done
data: {"complete":true,"usage":{"inputTokens":8,"outputTokens":15,"totalTokens":23}}

202

Async execution started (when async: true in request)

object
requestId
required

Unique request identifier for polling

string
XkdVWiEfSwMEPrw=
sessionId

Session ID for conversation continuity

string
session-1769056496430
status
required

Initial execution status

string
Allowed values: queued 
queued
message

Human-readable status message

string
Execution started. Poll the status endpoint for updates.
pollUrl
required

URL to poll for execution status

string
/ai/chat/executions/XkdVWiEfSwMEPrw%3D

500

Failed to perform streaming inference