RAG (Retrieval-Augmented Generation) Pattern | Bootspring Docs

Build knowledge-powered AI systems that retrieve relevant context before generating responses.

Overview#

RAG combines retrieval systems with generative AI to answer questions using your own data. It retrieves relevant documents from a knowledge base and provides them as context for the AI to generate accurate, grounded responses.

When to use:

Knowledge bases and documentation search
Document Q&A systems
Support chatbots with company-specific knowledge
Search result enhancement
Any AI application requiring factual grounding

Key features:

Document ingestion and chunking
Vector similarity search
Context-aware generation
Source attribution
Streaming responses
Hybrid search (vector + keyword)

Code Example#

Document Ingestion#

// lib/rag/ingest.ts
import { OpenAI } from 'openai'
import { prisma } from '@/lib/db'

const openai = new OpenAI()

interface Document {
  id: string
  content: string
  metadata?: Record<string, any>
}

export async function ingestDocument(doc: Document) {
  // Split into chunks
  const chunks = splitIntoChunks(doc.content, 500)

  // Generate embeddings for each chunk
  const embeddingsResponse = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: chunks
  })

  // Store chunks with embeddings
  await prisma.$transaction(
    chunks.map((chunk, i) =>
      prisma.documentChunk.create({
        data: {
          documentId: doc.id,
          content: chunk,
          embedding: embeddingsResponse.data[i].embedding,
          metadata: doc.metadata
        }
      })
    )
  )
}

function splitIntoChunks(text: string, maxChunkSize: number): string[] {
  const sentences = text.split(/(?<=[.!?])\s+/)
  const chunks: string[] = []
  let currentChunk = ''

  for (const sentence of sentences) {
    if ((currentChunk + sentence).length > maxChunkSize && currentChunk) {
      chunks.push(currentChunk.trim())
      currentChunk = ''
    }
    currentChunk += sentence + ' '
  }

  if (currentChunk.trim()) {
    chunks.push(currentChunk.trim())
  }

  return chunks
}

Vector Search#

// lib/rag/search.ts
import { OpenAI } from 'openai'
import { prisma } from '@/lib/db'

const openai = new OpenAI()

export async function searchSimilar(query: string, limit = 5) {
  // Generate query embedding
  const embeddingResponse = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: query
  })

  const queryEmbedding = embeddingResponse.data[0].embedding

  // Search using pgvector
  const results = await prisma.$queryRaw<Array<{
    id: string
    content: string
    similarity: number
  }>>`
    SELECT
      id,
      content,
      1 - (embedding <=> ${queryEmbedding}::vector) as similarity
    FROM "DocumentChunk"
    ORDER BY embedding <=> ${queryEmbedding}::vector
    LIMIT ${limit}
  `

  return results
}

RAG Query#

// lib/rag/query.ts
import Anthropic from '@anthropic-ai/sdk'
import { searchSimilar } from './search'

const client = new Anthropic()

export async function ragQuery(question: string) {
  // Retrieve relevant chunks
  const relevantChunks = await searchSimilar(question, 5)

  // Build context
  const context = relevantChunks
    .map(chunk => chunk.content)
    .join('\n\n---\n\n')

  // Generate answer
  const response = await client.messages.create({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 1024,
    system: `You are a helpful assistant that answers questions based on the provided context.
If the context doesn't contain relevant information, say so.
Always cite which parts of the context you're using.`,
    messages: [
      {
        role: 'user',
        content: `Context:
${context}

Question: ${question}

Answer based on the context above:`
      }
    ]
  })

  const textBlock = response.content.find(b => b.type === 'text')

  return {
    answer: textBlock?.text ?? '',
    sources: relevantChunks.map(c => ({
      content: c.content.slice(0, 200),
      similarity: c.similarity
    }))
  }
}

Streaming RAG#

// lib/rag/streaming.ts
import Anthropic from '@anthropic-ai/sdk'
import { searchSimilar } from './search'

const client = new Anthropic()

export async function* streamingRagQuery(question: string) {
  // Yield search status
  yield { type: 'status', message: 'Searching knowledge base...' }

  const relevantChunks = await searchSimilar(question, 5)

  yield {
    type: 'sources',
    sources: relevantChunks.map(c => ({
      content: c.content.slice(0, 100),
      similarity: c.similarity
    }))
  }

  const context = relevantChunks.map(c => c.content).join('\n\n---\n\n')

  yield { type: 'status', message: 'Generating answer...' }

  const stream = client.messages.stream({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 1024,
    system: `Answer questions based on the provided context.`,
    messages: [
      {
        role: 'user',
        content: `Context:\n${context}\n\nQuestion: ${question}`
      }
    ]
  })

  for await (const event of stream) {
    if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
      yield { type: 'text', content: event.delta.text }
    }
  }

  yield { type: 'done' }
}

Hybrid Search#

Combine vector and keyword search for better results:

// lib/rag/hybrid-search.ts
import { searchSimilar } from './search'
import { prisma } from '@/lib/db'

export async function hybridSearch(query: string, limit = 5) {
  // Vector search
  const vectorResults = await searchSimilar(query, limit * 2)

  // Keyword search
  const keywordResults = await prisma.documentChunk.findMany({
    where: {
      content: {
        contains: query,
        mode: 'insensitive'
      }
    },
    take: limit * 2
  })

  // Combine and deduplicate
  const allResults = new Map()

  vectorResults.forEach((r, i) => {
    allResults.set(r.id, {
      ...r,
      vectorRank: i,
      keywordRank: Infinity
    })
  })

  keywordResults.forEach((r, i) => {
    if (allResults.has(r.id)) {
      allResults.get(r.id).keywordRank = i
    } else {
      allResults.set(r.id, {
        ...r,
        vectorRank: Infinity,
        keywordRank: i,
        similarity: 0
      })
    }
  })

  // RRF (Reciprocal Rank Fusion) scoring
  const k = 60
  const scoredResults = Array.from(allResults.values()).map(r => ({
    ...r,
    rrfScore: 1 / (k + r.vectorRank) + 1 / (k + r.keywordRank)
  }))

  return scoredResults
    .sort((a, b) => b.rrfScore - a.rrfScore)
    .slice(0, limit)
}

Usage Instructions#

Ingest documents: Split documents into chunks and generate embeddings
Store in vector database: Save chunks with embeddings in pgvector
Handle queries: Generate query embedding and find similar chunks
Build context: Concatenate relevant chunks for the AI
Generate response: Pass context to the LLM with clear instructions
Return sources: Include retrieved chunks for transparency

Best Practices#

Chunk wisely - Use sentence boundaries, aim for 300-800 tokens per chunk
Add overlap - Include 10-20% overlap between chunks
Include metadata - Store source, page numbers, dates for attribution
Filter by relevance - Set a minimum similarity threshold (e.g., 0.7)
Use hybrid search - Combine vector and keyword search for better recall
Limit context - Keep context within model limits (aim for 3-5 chunks)
Stream responses - Show sources first, then stream the answer
Handle edge cases - Respond gracefully when no relevant content is found

Embeddings - Vector embedding fundamentals
Streaming - Real-time AI response streaming
Full-Text Search - PostgreSQL text search