Streaming Responses

Get real-time responses as the model generates tokens.

Enabling Streaming

Add stream: true to your request to enable streaming responses:

request.json

{
  "model": "qwen/qwen3-235b-a22b-instruct-2507-fp8",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "stream": true
}

Need full request parameters? See the chat completions endpoint in the API reference.

Streaming Response Schema

Streaming responses arrive as SSE chunks you parse incrementally.

Review the streaming response schema in the API reference for the full field list and [DONE] signal.

Code Examples

Examples for streaming in different environments:

python

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gonkagate.com/v1",
    api_key="gp-your-api-key"
)

stream = client.chat.completions.create(
    model="qwen/qwen3-235b-a22b-instruct-2507-fp8",
    messages=[{"role": "user", "content": "Write a poem"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

    # Final chunk contains usage info
    if chunk.usage:
        print(f"\n\nCost: ${chunk.usage.total_cost_usd:.6f}")

Cost Tracking in Streams

The final chunk includes full usage and cost breakdown:

Check the final chunk — Usage with total_cost_usd is only available in the last chunk before [DONE].

Usage fields are defined in the streaming response schema in the API reference.

cost-tracking.ts

// Track costs from streaming response
let totalCost = 0;

for await (const chunk of stream) {
  // ... handle content ...

  if (chunk.usage) {
    totalCost = chunk.usage.total_cost_usd;
    console.log(`Request cost: $${totalCost.toFixed(6)}`);
    console.log(`Breakdown: base $${chunk.usage.base_cost_usd.toFixed(6)} + fee $${chunk.usage.platform_fee_usd.toFixed(6)}`);
  }
}

Error Handling in Streams

Handle connection issues gracefully when streaming:

For status-specific guidance, see the error handling examples in the API reference.

Connection Drops

Network issues can interrupt streams. Implement reconnection logic for production use.

error-handling.ts

async function streamWithErrorHandling(messages: Message[]) {
  try {
    const stream = await client.chat.completions.create({
      model: "qwen/qwen3-235b-a22b-instruct-2507-fp8",
      messages,
      stream: true,
    });

    for await (const chunk of stream) {
      // Process chunk...
    }
  } catch (error) {
    if (error.code === "ECONNRESET" || error.code === "ETIMEDOUT") {
      // Connection was dropped
      console.error("Connection lost. Consider retrying.");
    } else if (error.status === 402) {
      // Insufficient balance
      showDepositModal();
    } else {
      throw error;
    }
  }
}