Supported Formats

Overview

Arcanflows supports a wide variety of document formats for your knowledge base. This guide covers each format, its optimal use cases, and tips for best results.

Supported Formats

Format Details

PDF Documents

Best For: Reports, manuals, research papers, forms

Processing Features:

• Text layer extraction
• OCR for scanned documents
• Table detection
• Image text extraction
• Hyperlink preservation

Optimization Tips:

• Use text-based PDFs when possible (not scanned images)
• Ensure good scan quality for OCR
• Keep file size reasonable for faster processing

jsonCopy
{
  "format": "pdf",
  "processing_options": {
    "ocr_enabled": true,
    "extract_tables": true,
    "extract_images": false,
    "language": "en"
  }
}

Word Documents (.docx, .doc)

Best For: Internal documentation, policies, procedures, contracts

Processing Features:

• Full text extraction
• Heading structure preservation
• Table extraction
• List formatting
• Comments and track changes (optional)

Optimization Tips:

• Use .docx format for best results
• Structure with headings for better chunking
• Remove unnecessary formatting

jsonCopy
{
  "format": "docx",
  "processing_options": {
    "preserve_structure": true,
    "extract_tables": true,
    "include_comments": false,
    "include_track_changes": false
  }
}

Plain Text (.txt)

Best For: Simple content, logs, code files, transcripts

Processing Features:

• Direct text ingestion
• Automatic encoding detection
• Line-based chunking option

Optimization Tips:

• Use UTF-8 encoding
• Structure with clear sections
• Add blank lines between topics

jsonCopy
{
  "format": "txt",
  "processing_options": {
    "encoding": "utf-8",
    "preserve_line_breaks": false
  }
}

Markdown (.md)

Best For: Technical documentation, wikis, README files, structured content

Processing Features:

• Heading hierarchy preserved
• Code block detection
• Link extraction
• List structure
• Table parsing

Optimization Tips:

• Use proper heading levels (# ## ###)
• Separate sections clearly
• Use code fences for code blocks

jsonCopy
{
  "format": "markdown",
  "processing_options": {
    "preserve_code_blocks": true,
    "extract_links": true,
    "parse_frontmatter": true
  }
}

HTML (.html, .htm)

Best For: Web content, help pages, scraped websites

Processing Features:

• Text extraction from tags
• Structure from semantic HTML
• Link preservation
• Table extraction
• Script/style removal

Optimization Tips:

• Use semantic HTML (article, section, header)
• Remove navigation and footer elements
• Clean up before upload

jsonCopy
{
  "format": "html",
  "processing_options": {
    "extract_main_content": true,
    "remove_navigation": true,
    "preserve_links": true,
    "extract_tables": true
  }
}

CSV Files (.csv)

Best For: Product catalogs, FAQs, structured data, lookups

Processing Features:

• Row-by-row processing
• Header detection
• Data type inference
• Searchable records

Optimization Tips:

• Include descriptive headers
• Keep data clean and consistent
• Consider combining related columns

Special Processing:

Each row becomes a searchable chunk:

csvCopy
question,answer,category
How do I reset my password?,Go to Settings > Security > Reset Password,Account
What are the pricing plans?,We offer Free, Pro, and Enterprise plans,Billing

jsonCopy
{
  "format": "csv",
  "processing_options": {
    "has_header": true,
    "delimiter": ",",
    "chunk_by_row": true,
    "combine_columns": ["question", "answer"]
  }
}

JSON Files (.json)

Best For: API documentation, configuration data, structured knowledge

Processing Features:

• Nested structure handling
• Array processing
• Key-value extraction
• Schema detection

Optimization Tips:

• Flatten deeply nested structures
• Include descriptive keys
• Add metadata fields

jsonCopy
{
  "format": "json",
  "processing_options": {
    "flatten_depth": 3,
    "array_handling": "separate_chunks",
    "include_keys_in_text": true
  }
}

Excel Files (.xlsx, .xls)

Best For: Spreadsheets, data tables, reports with multiple sheets

Processing Features:

• Multi-sheet support
• Table detection
• Formula result extraction
• Named range support

Optimization Tips:

• Use first row as headers
• Name sheets descriptively
• Remove empty rows/columns

jsonCopy
{
  "format": "xlsx",
  "processing_options": {
    "sheets": "all",
    "has_headers": true,
    "extract_formulas": false,
    "merge_sheets": false
  }
}

PowerPoint (.pptx, .ppt)

Best For: Presentations, training materials, slide decks

Processing Features:

• Slide text extraction
• Speaker notes
• Title/content structure
• Embedded text in shapes

Optimization Tips:

• Include speaker notes for context
• Use descriptive slide titles
• Keep text in text boxes (not images)

jsonCopy
{
  "format": "pptx",
  "processing_options": {
    "include_speaker_notes": true,
    "include_slide_numbers": true,
    "extract_from_shapes": true
  }
}

Web Content

URL Import

Import content directly from web pages:

bashCopy
curl -X POST "https://api.arcanflows.com/api/v1/agents/{agent_id}/knowledge/import-url" \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.example.com/guide",
    "crawl_depth": 2,
    "include_subpages": true
  }'

Sitemap Import

Import entire documentation sites:

jsonCopy
{
  "sitemap_url": "https://docs.example.com/sitemap.xml",
  "include_patterns": ["/docs/*", "/guides/*"],
  "exclude_patterns": ["/blog/*", "/changelog/*"],
  "max_pages": 100
}

Format Comparison

Best Practices

1. Choose the Right Format

• Technical docs: Markdown or HTML
• Business documents: Word or PDF
• Structured data: CSV or JSON
• Mixed content: PDF with text layer

2. Prepare Documents

• Remove unnecessary headers/footers
• Ensure consistent formatting
• Add section headings
• Remove confidential watermarks

3. Optimize for Search

• Use descriptive titles
• Include keywords naturally
• Structure with headings
• Add context to tables

4. Batch Upload

Upload multiple files efficiently:

bashCopy
curl -X POST "https://api.arcanflows.com/api/v1/agents/{agent_id}/knowledge/batch" \
  -H "X-API-Key: your_api_key" \
  -F "files[][email protected]" \
  -F "files[][email protected]" \
  -F "files[][email protected]" \
  -F "options={"auto_chunk": true}"

Troubleshooting

PDF not extracting text

• Check if PDF is image-based (enable OCR)
• Verify PDF isn't password protected
• Try re-saving from source application

CSV columns not recognized

• Ensure header row is present
• Check delimiter (comma, semicolon, tab)
• Remove BOM characters from UTF-8 files

HTML content incomplete

• Check for JavaScript-rendered content
• Verify main content selector
• Try different extraction options

File too large

• Split into smaller documents
• Compress images in PDFs
• Remove unnecessary pages