Chunking Strategies

Overview

Chunking is the process of splitting documents into smaller, meaningful pieces for storage and retrieval. The right chunking strategy significantly impacts your agent's ability to find and use relevant information.

Why Chunking Matters

┌──────────────────────────────────────────────────────────┐
│                    Large Document                         │
│  (Too big to fit in context window or embed effectively) │
└──────────────────────────────────────────────────────────┘
                           │
                           ▼ Chunking
┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐
│  Chunk 1 │  │  Chunk 2 │  │  Chunk 3 │  │  Chunk 4 │
│  (512)   │  │  (512)   │  │  (512)   │  │  (512)   │
└──────────┘  └──────────┘  └──────────┘  └──────────┘
     │             │             │             │
     ▼             ▼             ▼             ▼
  [embed]       [embed]       [embed]       [embed]
     │             │             │             │
     └─────────────┴─────────────┴─────────────┘
                           │
                           ▼
                   Vector Database
                   (Searchable chunks)

Chunking Strategies

1. Fixed Size Chunking

Splits text into chunks of a fixed character or token count.

Best For: Homogeneous content, simple documents

jsonCopy
{
  "chunking": {
    "strategy": "fixed_size",
    "chunk_size": 512,
    "chunk_overlap": 50,
    "size_unit": "tokens"
  }
}

Pros:

• Simple and predictable
• Consistent chunk sizes
• Fast processing

Cons:

• May split mid-sentence
• No semantic awareness
• Can break context

2. Sentence-based Chunking

Splits at sentence boundaries, grouping sentences to target size.

Best For: Articles, documentation, narrative content

jsonCopy
{
  "chunking": {
    "strategy": "sentence",
    "target_chunk_size": 512,
    "min_chunk_size": 100,
    "max_chunk_size": 1024,
    "overlap_sentences": 1
  }
}

How it works:

1. Split document into sentences
2. Group sentences until target size reached
3. Ensure sentence boundaries respected
4. Overlap by including last N sentences in next chunk

Pros:

• Preserves sentence meaning
• Better semantic coherence
• More natural breaks

Cons:

• Variable chunk sizes
• May create very small/large chunks

3. Paragraph-based Chunking

Splits at paragraph boundaries.

Best For: Well-structured documents, essays, blog posts

jsonCopy
{
  "chunking": {
    "strategy": "paragraph",
    "combine_short_paragraphs": true,
    "min_paragraph_length": 100,
    "max_chunk_paragraphs": 3
  }
}

Pros:

• Maintains topical coherence
• Natural document structure
• Good for well-formatted docs

Cons:

• Highly variable sizes
• Dependent on document formatting

4. Semantic Chunking

Uses AI to identify semantic boundaries and create meaningful chunks.

Best For: Complex documents, mixed content, maximum quality

jsonCopy
{
  "chunking": {
    "strategy": "semantic",
    "target_chunk_size": 512,
    "similarity_threshold": 0.7,
    "use_embeddings": true
  }
}

How it works:

1. Split into sentences
2. Generate embeddings for each sentence
3. Group sentences with high similarity
4. Split when similarity drops below threshold

Pros:

• Best semantic coherence
• Intelligent boundary detection
• Adapts to content

Cons:

• Slower processing
• Higher cost (embedding generation)
• More complex

5. Recursive Character Chunking

Tries multiple separators hierarchically until chunks fit target size.

Best For: Code, markdown, hierarchical documents

jsonCopy
{
  "chunking": {
    "strategy": "recursive",
    "chunk_size": 512,
    "chunk_overlap": 50,
    "separators": ["\n\n", "\n", ". ", " ", ""]
  }
}

How it works:

1. Try splitting by "\n\n" (paragraphs)
2. If chunks too big, try "\n" (lines)
3. If still too big, try ". " (sentences)
4. Continue until all chunks fit

Pros:

• Respects document hierarchy
• Good for structured content
• Flexible fallback

6. Header-based Chunking

Splits based on document headers/sections.

Best For: Documentation, manuals, structured reports

jsonCopy
{
  "chunking": {
    "strategy": "header",
    "header_levels": ["h1", "h2", "h3"],
    "include_header_in_chunk": true,
    "max_chunk_size": 2048
  }
}

How it works:

1. Identify headers (markdown #, HTML h1-h6)
2. Split content under each header
3. Include header text with content
4. Sub-split if sections too large

Pros:

• Maintains document structure
• Headers provide context
• Logical content grouping

Cons:

• Requires well-structured documents
• Very large sections problematic

Chunk Size Recommendations

Chunk Overlap

Overlap ensures context isn't lost at boundaries:

Chunk 1: [AAAAAAAAAA]
Chunk 2:        [BBBBBBBBBB]
                ↑
            Overlap region
            (shared content)

Benefits:

• Prevents information loss at boundaries
• Improves retrieval for boundary queries
• Maintains context continuity

Guidelines:

• 10-20% overlap is typical
• Higher overlap for dense, technical content
• Lower overlap for conversational content

Configuring Chunking in Arcanflows

Via UI

1. Go to Agents → Select agent → Knowledge Base
2. Click Settings icon
3. Choose chunking strategy
4. Adjust parameters
5. Click Reprocess to apply to existing documents

Via API

bashCopy
curl -X PATCH "https://api.arcanflows.com/api/v1/agents/{agent_id}/knowledge/settings" \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "chunking": {
      "strategy": "recursive",
      "chunk_size": 512,
      "chunk_overlap": 50,
      "separators": ["\n\n", "\n", ". ", " "]
    }
  }'

Per-Document Override

Apply different settings to specific documents:

bashCopy
curl -X POST "https://api.arcanflows.com/api/v1/agents/{agent_id}/knowledge/documents" \
  -H "X-API-Key: your_api_key" \
  -F "[email protected]" \
  -F "chunking={"strategy": "header", "chunk_size": 1024}"

Metadata Enrichment

Enhance chunks with metadata for better retrieval:

jsonCopy
{
  "chunk": {
    "text": "To reset your password, go to Settings...",
    "metadata": {
      "source": "user-guide.pdf",
      "page": 12,
      "section": "Account Settings",
      "document_type": "help",
      "last_updated": "2025-01-15"
    }
  }
}

Useful metadata:

• Source document name
• Page/section numbers
• Document category
• Date information
• Custom tags

Advanced Techniques

Parent-Child Chunking

Store both large parent chunks and small child chunks:

jsonCopy
{
  "chunking": {
    "strategy": "parent_child",
    "parent_chunk_size": 2048,
    "child_chunk_size": 256,
    "retrieval": "child",
    "context": "parent"
  }
}

How it works:

1. Create large parent chunks (for context)
2. Split parents into small children (for search)
3. Search against children
4. Return parent for context

Hypothetical Questions

Generate questions for each chunk to improve retrieval:

jsonCopy
{
  "enrichment": {
    "generate_questions": true,
    "questions_per_chunk": 3,
    "embed_questions": true
  }
}

Summary Chunks

Create summaries alongside original chunks:

jsonCopy
{
  "enrichment": {
    "generate_summaries": true,
    "summary_max_length": 100,
    "embed_summaries": true
  }
}

Evaluating Chunk Quality

Metrics to Monitor

Testing Queries

Create test queries for each document:

pythonCopy
test_queries = [
    "How do I reset my password?",
    "What are the pricing plans?",
    "How to configure webhooks?"
]

for query in test_queries:
    results = agent.search_knowledge(query, top_k=3)
    evaluate_relevance(query, results)

Troubleshooting

Chunks too small

• Increase chunk_size
• Use paragraph or header strategy
• Combine related sentences

Chunks missing context

• Increase chunk_overlap
• Include headers in chunks
• Use parent-child chunking

Poor retrieval results

• Try semantic chunking
• Add metadata enrichment
• Generate hypothetical questions

Slow processing

• Use fixed size for large batches
• Process documents asynchronously
• Reduce semantic chunking use