One of the most important technical decisions in any AI system is how to manage the context window — the amount of conversation history the AI can “see” at any given time.
During a recent dev call, Dave W hit the context window limit while testing BrainDrive across multiple projects. This kicked off a discussion about how we should handle it. Here’s an ~9 minute clip from that conversation:
The problem: Every AI model has a context window limit — state-of-the-art cloud models typically support 200K+ tokens, but local models might only handle 10K or less. And even within those limits, performance degrades as the window fills up. On top of that, a lot of what fills the context window isn’t even visible to you — reasoning tokens, tool calls, and system prompts all take up space behind the scenes.
The options we considered:
- Summarization / context compression — Condense older messages into a summary. Risk: you lose nuance, and if you corrected the AI earlier, the summary might not reflect that.
- Sliding window — Keep the most recent N messages and drop the rest. Simple, predictable, and preserves the current focus of the conversation.
- Full context — Keep everything. Works until it doesn’t.
What we chose: Sliding window. BrainDrive keeps roughly the last 10 messages in the active context. Here’s why this works better than it sounds:
-
Your important context lives outside the chat. Your specs, build plans, agent files, and profile are loaded into every conversation. These are the durable, refined versions of your goals and plans — not the messy back-and-forth where you changed your mind three times.
-
Nothing is lost. Full conversation history is always saved in your library. You can always go back and review what was discussed.
-
Per-project isolation. When you switch from your Finance project to your Career project, the context should reset to that project’s world — not carry over stale context from a different domain.
-
It matches how humans work. You remember the last few minutes of a conversation clearly. For everything else, you refer to your notes.
We’re continuing to refine this as we use BrainDrive daily. If you run into context-related issues or have ideas on how it should work, let us know in this thread.