Create Basic Chunking Job
Create Basic Chunking Job
Overview
Flow ID: create-chunking-job
Category: Blockify Processing (Chunking Mode)
Estimated Duration: 1-2 minutes (setup) + Fast Processing
User Role: All Users
Complexity: Simple
Purpose: Creates a job that “mechanically” splits documents into pieces (chunks) and generates embeddings, without using AI to generating Questions/Answers. This is significantly faster and cheaper than Blockify but results in a “dumber” search experience (keyword/semantic similarity only).
Trigger
What initiates this flow:
- User manually initiates
Specific trigger: User selects “Chunking Only” mode in the Job Creation screen.
Prerequisites
Before starting, users must have:
- Active Embedding Model
- Files to process
User Intent Analysis
Primary Intent
Quickly ingest large amounts of documentation where specific Q&A structure is not required, or when testing embedding models.
Secondary Intents
- Saving tokens/credits (cheaper than Blockify)
- Processing speed (finished in minutes vs hours)
Step-by-Step Flow
Main Path (Happy Path)
Step 1: Select Mode
- User Action: Click “Chunking Only” tab (or “Standard Ingestion”).
- System Response: “Blockify Model” dropdown disappears (not needed).
Step 2: Configure Dataset
- User Action: Select Target Dataset (New/Existing).
- Reference: Select Target Dataset
Step 3: Upload Files
- User Action: Upload documents.
- Reference: Upload Files
Step 4: Start Job
- User Action: Click Start Job.
- System Response: Job begins immediately.
- Note: This job skips the “AI Blockify” stage and goes straight to “Chunking & Embedding”.
Error States & Recovery
Error 1: Wrong Expectation
Cause: User expects Q&A pairs but ran a Chunking job
User Experience: Dataset search is less effective (“I can’t find answers”)
Recovery: Must re-run as a Blockify job if Q&A structure is needed.
Pain Points & Friction
- Confusion with Blockify: Users might not understand the difference.
- Mitigation: Distinct iconography and Helper text (“Best for speed”, “No AI Q&A”).
Design Considerations
- Speed Indicators: Highlight that this mode is “Blazing Fast”.
- Visual Distinction: Use a different color badge (e.g., Green for Chunking, Purple for Blockify) in the job list.
Related Flows
- Create Blockify Job - The AI-enhanced alternative
- Configure Basic Chunks - Relevant settings
Technical References
src/components/blockify-corpus/new-job-screen.jssrc/engines/chunking-engine.js