AI Streaming Pattern | Bootspring Docs

Implement real-time AI response streaming for chat interfaces and content generation with smooth user experience.

Overview#

Streaming AI responses provides immediate feedback to users, creating a more engaging experience similar to how humans communicate. Instead of waiting for a complete response, users see text appear incrementally as the AI generates it.

When to use:

Chat interfaces and conversational UIs
Long-form content generation
Real-time response display
Any AI interaction where perceived latency matters

Key features:

Incremental response display
Server-Sent Events (SSE) support
Native Anthropic streaming
Vercel AI SDK integration
Client-side stream consumption

Code Example#

Anthropic Native Streaming#

// app/api/chat/route.ts
import { anthropic } from '@/lib/anthropic'

export async function POST(req: Request) {
  const { messages } = await req.json()

  const stream = await anthropic.messages.stream({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 1024,
    messages
  })

  return new Response(stream.toReadableStream(), {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      Connection: 'keep-alive'
    }
  })
}

Vercel AI SDK Streaming#

// app/api/chat/route.ts
import { anthropic } from '@ai-sdk/anthropic'
import { streamText } from 'ai'

export async function POST(req: Request) {
  const { messages } = await req.json()

  const result = streamText({
    model: anthropic('claude-sonnet-4-20250514'),
    messages
  })

  return result.toDataStreamResponse()
}

Client-Side Stream Consumer#

// components/chat.tsx
'use client'
import { useState } from 'react'

interface Message {
  role: 'user' | 'assistant'
  content: string
}

export function Chat() {
  const [messages, setMessages] = useState<Message[]>([])
  const [input, setInput] = useState('')
  const [isLoading, setIsLoading] = useState(false)

  async function handleSubmit(e: React.FormEvent) {
    e.preventDefault()
    if (!input.trim()) return

    const userMessage = { role: 'user' as const, content: input }
    setMessages(prev => [...prev, userMessage])
    setInput('')
    setIsLoading(true)

    const response = await fetch('/api/chat', {
      method: 'POST',
      body: JSON.stringify({
        messages: [...messages, userMessage]
      })
    })

    const reader = response.body?.getReader()
    const decoder = new TextDecoder()
    let assistantMessage = ''

    // Add placeholder for assistant message
    setMessages(prev => [...prev, { role: 'assistant', content: '' }])

    while (reader) {
      const { done, value } = await reader.read()
      if (done) break

      assistantMessage += decoder.decode(value)
      setMessages(prev => [
        ...prev.slice(0, -1),
        { role: 'assistant', content: assistantMessage }
      ])
    }

    setIsLoading(false)
  }

  return (
    <div className="flex flex-col h-full">
      <div className="flex-1 overflow-y-auto p-4 space-y-4">
        {messages.map((m, i) => (
          <div
            key={i}
            className={`p-4 rounded-lg ${
              m.role === 'user' ? 'bg-blue-100 ml-12' : 'bg-gray-100 mr-12'
            }`}
          >
            {m.content}
          </div>
        ))}
      </div>
      <form onSubmit={handleSubmit} className="p-4 border-t">
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          placeholder="Type a message..."
          className="w-full p-2 border rounded"
          disabled={isLoading}
        />
      </form>
    </div>
  )
}

AI SDK useChat Hook#

The Vercel AI SDK provides a convenient useChat hook that handles streaming automatically:

// components/chat.tsx
'use client'
import { useChat } from 'ai/react'

export function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } =
    useChat()

  return (
    <div className="flex flex-col h-full">
      <div className="flex-1 overflow-y-auto p-4 space-y-4">
        {messages.map((m) => (
          <div
            key={m.id}
            className={`p-4 rounded-lg ${
              m.role === 'user' ? 'bg-blue-100 ml-12' : 'bg-gray-100 mr-12'
            }`}
          >
            {m.content}
          </div>
        ))}
      </div>
      <form onSubmit={handleSubmit} className="p-4 border-t">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Type a message..."
          className="w-full p-2 border rounded"
          disabled={isLoading}
        />
      </form>
    </div>
  )
}

Usage Instructions#

Set up the API route: Create a streaming endpoint that returns a ReadableStream
Configure headers: Set appropriate SSE headers for streaming
Implement client consumer: Use the Fetch API's body.getReader() or the AI SDK hooks
Handle loading states: Show typing indicators while streaming
Process incremental updates: Update UI as chunks arrive

Best Practices#

Use the AI SDK when possible - The useChat hook handles edge cases like reconnection and error handling
Show loading indicators - Display a typing indicator while waiting for the first chunk
Handle errors gracefully - Implement error boundaries and retry logic for failed streams
Consider connection limits - Be aware of browser connection limits when opening multiple streams
Implement abort signals - Allow users to cancel long-running requests
Buffer partial tokens - Some streaming implementations may split tokens; buffer appropriately
Test on slow connections - Verify behavior on throttled networks

Claude API Integration - Basic Claude API setup
Function Calling - AI tool use with streaming
RAG - Streaming with retrieval augmented generation