Documentation Sources

What Are Documentation Sources?

Documentation sources are websites that There There crawls and indexes so the AI assistant can search them when answering questions. By connecting your product documentation, knowledge base, or help center, the assistant can provide accurate, sourced answers to customer questions.

Documentation sources are configured per channel. This means different channels can have different documentation, which is useful if your workspace handles support for multiple products.

Adding a Documentation Source

1. Navigate to the channel settings for the channel you want to configure.
2. Open the Documentation section and click to add a new source.
3. Provide a name for the source (e.g., "Product Documentation").
4. Enter the base URL of your documentation site (e.g., https://docs.example.com).
5. Optionally specify include paths to limit crawling to specific sections, or exclude paths to skip irrelevant pages.
6. Save the source to begin crawling.

Each workspace can have up to 5 documentation sources across all channels.

Crawl Process

After you add a source, There There automatically starts crawling the website. The crawl status progresses through several stages: Pending, Crawling, Completed, or Failed. You can monitor the progress in real time, including the number of pages crawled and chunks created.

The crawler converts each page into markdown, splits it into smaller chunks, and generates embeddings for semantic search. This allows the AI assistant to find relevant passages even when the customer's question does not use the exact same wording as the documentation.

Keeping Docs Up to Date

Documentation sources can be re-crawled to pick up changes. If your product documentation is updated frequently, re-crawl periodically to ensure the AI assistant always has the latest information. If a crawl fails, check the error message for details. Common issues include unreachable URLs or pages that block crawlers.