Skip to content
QuantCDN

Chat inference via API Gateway (buffered responses) with multimodal support

POST
/api/v3/organizations/{organisation}/ai/chat
curl --request POST \
  --url https://dashboard.quantcdn.io/api/v3/organizations/example/ai/chat \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "messages": [ { "role": "user", "content": "What is the capital of Australia?" } ], "modelId": "amazon.nova-lite-v1:0", "temperature": 0.7, "maxTokens": 4096, "topP": 1, "stream": true, "systemPrompt": "example", "stopSequences": [ "example" ], "responseFormat": { "type": "json", "jsonSchema": {} }, "toolConfig": { "tools": [ { "toolSpec": { "name": "example", "description": "example", "inputSchema": { "json": {} } } } ], "autoExecute": false, "allowedTools": [ "example" ] }, "sessionId": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "async": false, "allowedTools": [ "get_weather", "generate_image" ], "guardrails": { "guardrailIdentifier": "example", "guardrailVersion": "example", "trace": "enabled" }, "longContext": false }'

Sends requests to the AI API Gateway endpoint which buffers responses. Supports text, images, videos, and documents via base64 encoding. * * Execution Modes: * - Sync Mode (default): Standard JSON response, waits for completion (200 response) * - Async Mode: Set async: true for long-running tasks with polling (202 response) * * Async/Durable Mode (async: true): * - Returns immediately with requestId and pollUrl (HTTP 202) * - Uses AWS Lambda Durable Functions for long-running inference * - Supports client-executed tools via waiting_callback state * - Poll /ai/chat/executions/{requestId} for status * - Submit client tool results via /ai/chat/callback * - Ideal for complex prompts, large contexts, or client-side tools * * Multimodal Support: * - Text: Simple string content * - Images: Base64-encoded PNG, JPEG, GIF, WebP (up to 25MB) * - Videos: Base64-encoded MP4, MOV, WebM, etc. (up to 25MB) * - Documents: Base64-encoded PDF, DOCX, CSV, etc. (up to 25MB) * * Supported Models (Multimodal): * - Claude 4.5 Series: Sonnet 4.5, Haiku 4.5, Opus 4.5 (images, up to 20 per request) * - Claude 3.5 Series: Sonnet v1/v2 (images, up to 20 per request) * - Amazon Nova: Lite, Pro, Micro (images, videos, documents) * * Usage Tips: * - Use base64 encoding for images/videos < 5-10MB * - Place media before text prompts for best results * - Label multiple media files (e.g., ‘Image 1:’, ‘Image 2:’) * - Maximum 25MB total payload size * * Response Patterns: * - Text-only: Returns simple text response when no tools requested * - Single tool: Returns toolUse object when AI requests one tool * - Multiple tools: Returns toolUse array when AI requests multiple tools * - Auto-execute sync: Automatically executes tool and returns final text response * - Auto-execute async: Returns toolUse with executionId and status for polling

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Path Parameters

Section titled “Path Parameters”
organisation
required
string

The organisation ID

Request Bodyrequired

Section titled “Request Bodyrequired”

Chat request with optional multimodal content blocks

Media typeapplication/json
object
messages
required

Array of chat messages. Content can be a simple string or an array of content blocks for multimodal input.

Array<object>
>= 1 items
object
role
required
string
Allowed values: user assistant system
content
required
One of:

Simple text message

string
modelId
required

Model ID. Use Nova models for multimodal support.

string
Example
amazon.nova-lite-v1:0
temperature
number
default: 0.7 <= 2
maxTokens

Max tokens. Claude 4.5 supports up to 64k.

integer
default: 4096 >= 1 <= 65536
topP
number
<= 1
stream

Ignored in buffered mode, always returns complete response

boolean
systemPrompt

Optional custom system prompt. When tools are enabled, this is prepended with tool usage guidance.

string
stopSequences

Custom stop sequences

Array<string>
<= 4 items
responseFormat

Structured JSON output (Claude 3.5 Sonnet v1/v2, Nova Pro)

object
type
string
Allowed values: json
jsonSchema

JSON Schema defining expected structure

object
toolConfig

Function calling configuration (Claude 3+, Nova Pro)

object
tools
Array<object>
object
toolSpec
object
name
string
description
string
inputSchema
object
json

JSON Schema for function parameters

object
autoExecute

When true, backend automatically executes tools and feeds results back to AI. For async tools (e.g., image generation), returns executionId for polling. Security: Use allowedTools to whitelist which tools can auto-execute.

boolean
allowedTools

Whitelist of tool names that can be auto-executed. Required when autoExecute is true for security. Example: [‘get_weather’, ‘generate_image’]

Array<string>
sessionId

Optional session ID for conversation continuity. Omit to use stateless mode, include to continue an existing session.

string format: uuid
async

Enable async/durable execution mode. When true, returns 202 with pollUrl instead of waiting for completion. Use for long-running inference, client-executed tools, or operations >30 seconds.

boolean
allowedTools

Top-level convenience alias for toolConfig.allowedTools. Whitelists which tools can be auto-executed.

Array<string>
Example
[
  "get_weather",
  "generate_image"
]
guardrails

AWS Bedrock guardrails configuration for content filtering and safety.

object
guardrailIdentifier

Guardrail identifier from AWS Bedrock

string
guardrailVersion

Guardrail version

string
trace

Enable guardrail trace output

string
Allowed values: enabled disabled
longContext

Enable 1M context window support regardless of token estimation. Use when sending large payloads (>200K tokens).

boolean

Responses

Section titled “Responses”

200

Section titled “200”

Chat inference completed (buffered response, sync mode)

Media typeapplication/json
object
response

Assistant’s response message. May contain text content and/or tool use requests.

object
role
string
Allowed values: assistant
content

Text response content

string
toolUse
One of:

Single tool request

object
toolUseId
string
name
string
input
object
executionId

Present for async tools with autoExecute

string
status

Execution status (pending/running/complete/failed) - present for async tools with autoExecute

string
result

Tool execution result (only present when status=‘complete’ for sync auto-executed tools). For async tools, poll /tools/executions/{executionId}

object
images

Base64 data URIs for images

Array<string>
s3Urls

Signed S3 URLs for downloads

Array<string>
model

Model used for generation

string
requestId

Unique request identifier

string
finishReason

Why the model stopped generating

string
Allowed values: stop length content_filter tool_use
usage

Token usage and cost for this inference call

object
inputTokens

Number of input tokens

integer
outputTokens

Number of output tokens

integer
totalTokens

Total tokens consumed

integer
costCents

Cost of this inference call in US cents

number format: float
Example
{
  "response": {
    "role": "assistant",
    "content": "The capital of Australia is Canberra."
  },
  "model": "amazon.nova-lite-v1:0",
  "requestId": "req-abc123",
  "finishReason": "stop",
  "usage": {
    "inputTokens": 12,
    "outputTokens": 8,
    "totalTokens": 20,
    "costCents": 0.18
  }
}

202

Section titled “202”

Async execution started (when async: true in request)

Media typeapplication/json
object
requestId
required

Unique request identifier for polling

string
sessionId

Session ID for conversation continuity

string
status
required

Initial execution status

string
Allowed values: queued
message

Human-readable status message

string
pollUrl
required

URL to poll for execution status

string
Example
{
  "requestId": "XkdVWiEfSwMEPrw=",
  "sessionId": "session-1769056496430",
  "status": "queued",
  "message": "Execution started. Poll the status endpoint for updates.",
  "pollUrl": "/ai/chat/executions/XkdVWiEfSwMEPrw%3D"
}

500

Section titled “500”

Failed to perform chat inference