Chunking and Parsing
======================

.. image:: /images/foundry-kb-chunking-config.png
   :alt: Chunking and parsing configuration
   :align: center
   :width: 100%

|

When documents are ingested into an In-House knowledge base, they are split into chunks before being embedded and stored in the vector index. Chunking configuration directly affects retrieval quality: how much context each result carries, how precisely the search can target specific passages, and how much overlap exists between adjacent results.

Chunking Parameters
-------------------

Chunking is configured at knowledge base creation time and applies to all documents ingested into that knowledge base.

**Max Tokens per Chunk**

The maximum number of tokens allowed in a single chunk. Range is 100 tokens to the upper limit set by the embedding model.

* Smaller chunks (300–700 tokens) yield more focused retrieval. Queries return the specific passage most relevant to the question, with less surrounding noise.
* Larger chunks (1000–2000 tokens) preserve more context per result. Useful when answers require understanding a broader passage, such as multi-step procedures or policy sections.

The form auto-fills max tokens to the selected embedding model's limit when you choose a model. You can reduce it manually.

**Overlap Percentage**

The percentage of a chunk's content that is repeated at the start of the next chunk. Range is 5–50%.

Overlap prevents information at a chunk boundary from being split across two chunks and missed during retrieval. For example, if a sentence spans the end of chunk A and the start of chunk B, with overlap both chunks will contain that sentence, ensuring it is retrieved regardless of which chunk is ranked higher.

* Lower overlap (5–10%) reduces storage and indexing cost.
* Higher overlap (20–40%) reduces the risk of boundary information being missed, at the cost of some redundancy.

**Embedding Model**

The embedding model is set at creation time and cannot be changed. Each model has a maximum token limit that caps the Max Tokens per Chunk value. Switching models requires creating a new knowledge base, since existing embeddings are incompatible with a different model's vector space.

Parsing Strategy
----------------

The parsing strategy (set at creation) determines how documents are processed before chunking:

**With LLM**
   An LLM parses the document content before chunking. This produces higher-quality text extraction from complex layouts — PDFs with multi-column text, documents containing tables, scanned images with embedded text, and presentations with mixed content. LLM parsing adds ingestion time and cost.

**Without LLM**
   Plain text extraction is used. Faster and less expensive. Works well for clean, text-based files such as Markdown, plain-text PDFs, and Word documents with standard formatting.

The parsing strategy cannot be changed after a knowledge base is created.

Chunking Strategies (Bedrock)
------------------------------

For Bedrock knowledge bases, the chunking strategy is configured as part of the data source setup and offers more options:

* **Fixed Size** — chunks are split at a fixed token count with a configurable overlap.
* **Hierarchical** — documents are split into parent and child chunks. Parent chunks provide broader context; child chunks are used for retrieval. Useful for complex documents where both context and precision matter.
* **Semantic** — chunks are split at semantic boundaries (paragraph and sentence breaks) rather than at fixed token counts. Results in more natural chunks.

Best Practices
--------------

* Run a retrieval test after ingestion (see :doc:`retrieval_testing`) to verify that your chunking settings produce the expected results before connecting the knowledge base to an agent.
* If retrieval is returning off-topic passages, reduce Max Tokens per Chunk and re-ingest.
* If answers are incomplete because relevant context is spread across multiple results, increase Max Tokens per Chunk or increase Overlap Percentage.
* For technical documentation with dense cross-references, hierarchical chunking (Bedrock) often outperforms fixed-size chunking.