Configure Context Window Size

Overview

Flow ID: configure-context-window
Category: Model Management
Estimated Duration: < 10 seconds
User Role: Power User
Complexity: Advanced

Purpose: Adjust the “Memory Limit” (Context Window) of the active model. Higher values allow processing larger documents and longer chats but consume VASTLY more RAM/VRAM and slow down generation (quadratic scaling).

Trigger

What initiates this flow:

• User manually initiates

Specific trigger: Settings > Chat Options > Context Window Slider.

Prerequisites

Before starting, users must have:

• Active Model
• Knowledge of their hardware RAM limits

Step-by-Step Flow

Main Path (Happy Path)

Step 1: Access Settings

• User Action: Go to Chat Options.

Step 2: Adjust Slider

• User Action: Drag slider.
  • 2048: Fast, low memory.
  • 4096: Standard balance.
  • 8192: High memory.
  • 16k/32k+: Extreme memory (requires high-end Mac/GPU).

Step 3: Apply

• User Action: Click Save.
• System Response: Model reloads (interrupts any active chat). “Reloading Model…”

Error States & Recovery

Error 1: Out of Memory (OOM)

Cause: Setting 16k context on an 8GB RAM machine.
User Experience: App crashes or OS freezes when loading model.
Recovery: Restart app, reset settings to safe default (2048/4096).

Pain Points & Friction

1. “Why is it so slow?”: Users max out the slider thinking “more is better” without understanding the performance penalty.
   • Mitigation: Warning text “High values significantly impact speed”.

Related Flows

• Select LLM Model

Technical References

• src/engines/llm-config.js (n_ctx parameter)