Knowledge Base Chat

Chat powered by knowledge bases

Build conversational interfaces that let users ask natural-language questions over your documents.

Get started in 3 steps

Create a knowledge base

Set up a knowledge base with your documents. Use Markdown format for best results when building Knowledge Base Chat.

Test with the built-in chat window

Open your knowledge base and use the chat interface to test questions and verify responses.

Integrate via REST API

Use the REST API to build custom chat interfaces in your applications, websites, or services.

What is Knowledge Base Chat?

Knowledge Base Chat is a conversational interface that uses knowledge bases to answer questions about your documents. They combine RAG (Retrieval-Augmented Generation) with LLM chat capabilities to provide accurate, context-aware responses.

Natural language queries: Users ask questions in plain English
Automatic search: The AI searches your knowledge base to find relevant information
Context-aware responses: Answers are grounded in your actual documents
Streaming support: Real-time responses for better user experience

Using the Built-in Chat Window

Every knowledge base includes a built-in chat interface accessible from the knowledge base details page.

Accessing the Chat Window

Navigate to Knowledge Bases in the sidebar
Click on a knowledge base to open its details
Select the Chat tab

Chat Features

The chat window provides:

Model selection: Choose from available LLM models (Claude, GPT-4, Gemini, etc.)
Conversation history: Maintains context across multiple messages
Streaming responses: See answers appear in real-time as they’re generated
Search filters: Optionally filter by tags or upload date ranges
Tool call visibility: See when the AI searches the knowledge base and how many results it finds

Using Filters

You can optionally set filters that suggest constraints to the AI:

Tag filters: Limit searches to documents with specific tags
Date ranges: Filter by document upload date (from/to)

Filters are suggestions — the AI can choose whether to use them when searching.

REST API Integration

Use the REST API to build custom chat interfaces that integrate with your applications.

Endpoint

POST /v0/orgs/{organization_id}/knowledge-bases/{kb_id}/chat

Request Format

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "What is the return policy?"
    }
  ],
  "stream": true,
  "temperature": 0.7,
  "max_tokens": 1000,
  "metadata_filter": {
    "tag_ids": ["tag_id_1", "tag_id_2"]
  },
  "upload_date_from": "2024-01-01",
  "upload_date_to": "2024-12-31"
}

Request Parameters

model (required) — LLM model to use (e.g., gpt-4o-mini, claude-3-5-sonnet-20241022)
messages (required) — Array of conversation messages with role (“user” or “assistant”) and content
stream (required) — Must be true for streaming responses
temperature (optional) — Controls randomness (0.0-2.0, default: 0.7)
max_tokens (optional) — Maximum tokens in the response
metadata_filter (optional) — Filter by document metadata (e.g., tag_ids)
upload_date_from (optional) — Filter documents uploaded after this date (ISO format)
upload_date_to (optional) — Filter documents uploaded before this date (ISO format)

Streaming Response

The API returns a streaming response with chunks that include:

Text chunks: {"chunk": "text content", "done": false}
Tool call events: {"type": "tool_call", "tool_name": "kb_search", "arguments": {...}, "iteration": 1}
Tool result events: {"type": "tool_result", "tool_name": "kb_search", "results_count": 5, "iteration": 1}
Completion: {"done": true}
Errors: {"error": "error message", "done": true}

Example: TypeScript SDK

import { DocRouterOrgApi } from '@docrouter/sdk';

const api = new DocRouterOrgApi(organizationId);

const request = {
  model: 'gpt-4o-mini',
  messages: [
    { role: 'user', content: 'What is the return policy?' }
  ],
  stream: true,
  temperature: 0.7
};

await api.runKBChatStream(
  kbId,
  request,
  (chunk) => {
    if ('chunk' in chunk) {
      process.stdout.write(chunk.chunk);
    } else if (chunk.type === 'tool_call') {
      console.log(`\n[Searching: ${chunk.tool_name}]`);
    } else if (chunk.done) {
      console.log('\n[Complete]');
    }
  }
);

Example: cURL

curl -X POST \
  "https://app.docrouter.ai/fastapi/v0/orgs/{organization_id}/knowledge-bases/{kb_id}/chat" \
  -H "Authorization: Bearer your_org_api_token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "What is the return policy?"}
    ],
    "stream": true
  }'

How It Works

When a user asks a question:

Question received: Knowledge Base Chat receives the user’s question
Knowledge base search: The AI automatically searches the knowledge base for relevant chunks
Context assembly: Relevant document chunks are assembled into context
Response generation: The LLM generates an answer grounded in the retrieved context
Streaming delivery: The response is streamed back to the user in real-time

The AI can make multiple search iterations if needed to gather comprehensive information before answering.

Best Practices

✓ Use Markdown Documents — Format your knowledge base documents in Markdown for better structure and readability in chat responses.

✓ Test Thoroughly — Use the built-in chat window to test various questions and refine your knowledge base content.

✓ Choose the Right Model — Use faster models like gpt-4o-mini for simple queries, and more capable models for complex reasoning.

✓ Use Filters Strategically — Apply tag or date filters when you want to limit the search scope, but let the AI decide when to use them.

✓ Handle Streaming — Implement proper streaming handling in your client to show real-time responses and improve user experience.

Learn More

Knowledge Bases — Learn how to create and manage knowledge bases
REST API — Complete API reference for chat endpoints
Python SDK — Python client library (chat support coming soon)
TypeScript SDK — TypeScript/JavaScript client library

Ready to try Knowledge Base Chat?

Open Dashboard