Knowledge Base

Create Knowledge Base

Click [Knowledge Base] in the navigation bar of DigiAI AgentHub. You'll find a list of your existing knowledge bases. To create a new knowledge base, click [Create Knowledge Base] and provide a name for it. Select the Type of Knowledge Base： - Document Type : The system will segment the uploaded documents into manageable chunks, making it suitable for various document types. - Q&A Type : The system will automatically divide the uploaded content into question and answer pairs. Keep in mind that this option consumes more tokens due to the additional processing. Then, click [Add File] to add content. You can drag and drop or choose files manually for upload. Note that batch uploading is supported. Document Upload Guidelines: - Ensure your text content is ready, with formats including Long Text (TXT, Markdown, DOCX, HTML, or PDF files) and Structured Data (CSV, Excel, etc.). - Each document's maximum upload size is 15MB. After uploading, click [Next] to begin the processes of text chunking and data cleaning. This step prepares and structures the content for efficient use.

Knowledge Maintenance

Viewing Text Chunks Each document you upload to the knowledge base is organized into segments known as text chunks. You can browse the detailed content of these chunks by accessing the chunks list. Reviewing Chunk Quality The effectiveness of your knowledge base's Q&A capabilities is heavily influenced by the quality of these document chunks. It's advisable to manually review the chunks for quality assurance before linking the knowledge base to any agent. While automated chunking methods using character length, identifiers, or NLP semantic techniques can reduce labor for handling large text volumes, the intricacy of document formats and their semantic contexts necessitate manual checks to address machine chunk limitations in semantic processing. Important factors to observe when assessing chunk quality include: 1. Chunks that are too short, potentially losing semantic meaning. 2. Chunks that are too lengthy, introducing semantic noise that may impact accuracy. 3. Noticeable semantic truncation, which happens when exceeding segment length limits, causing forced truncation and omitting content during recall. Adding Text Chunks To include custom chunks in a document, navigate to the chunk list and click "Add Chunk." This allows you to insert one or multiple chunks tailored to the document. Editing Text Chunks Within the chunk list, you can modify the added chunks by updating text content and keywords to ensure clarity and relevance. Disabling Documents You have the option to deactivate documents or specific chunks from being indexed in the dataset. To do so, locate the disable button in the dataset document list to deactivate the document. You can also choose to disable an entire document or a particular chunk within the document details. Once disabled, documents will not be indexed until you click enable to restore them.

Knowledge Settings

To manage the settings of your chosen knowledge base, first select it, then navigate to the [Settings] tab. Here, you can adjust retrieval settings to optimize the performance and accessibility of your knowledge base. Retrieval Settings You have three options to choose from, each offering different advantages: - Hybrid Search (Default Option) : This method combines semantics and keyword querying to provide the most comprehensive results. It balances precision and recall by leveraging the strengths of both approaches, ensuring optimal query outcomes. - Keyword Search : This option focuses on keyword-based querying, making it ideal for locating specific content through exact terms like names of people, places, or other identifiers. Use this when you need precise, targeted searches. - Semantic Search : This approach employs semantic similarity for querying, which is ideal for retrieving similar answers based on descriptions or the meaning of questions. Choose this method when the context or meaning of queries is more important than specific keywords. TopK Define the number of text chunks retrieved based on similarity to the user's question. A default value of 5 is used, with higher values broadening the scope of retrieved text segments.