Starter Pack Updates

BrainDrive Development
1

@DJJones starting a thread here to post Start Pack updates.

I’m making progress but not quite there yet with something that I am ready to replace what we have. Will keep working on it later this evening and in the AM.

Dave W.

1 Like
2

@DJJones Still not there but I think I may have found why I am struggling. I was thinking that all the instructions I had to optimize were in the starter pack, but it looks like there are instructions that sit outside the starter pack.

Since they are in the runtime I am kicking over to you.

Going forward I think our rule should be that any instructions on how the AI is to behave go into the starter pack and none outside it.

Unless there is a good reason not to do it this way pls remove all instructions that currently sit outside of the starter pack as per the below .md.

dave-j-runtime-pack-boundary-brief.md (5.7 KB)

Any questions or issues let me know.

Thanks
Dave

3

@DJJones PR Opened: Starter pack: de-cheat templates + add quality floors (playback gate, profile-propose, fitness baseline) by davewaring · Pull Request #181 · BrainDriveAI/BrainDrive · GitHub)

4

Because of how instructions are currently in the file I feared if we went scored earth on it all it would reduce the functionality of BrainDrive over all however at the same time there was some clear instructions that needed to be removed instantly…

First step I created an execution plan for handling the “now” problem and then once we get a chance to observe how these changes are handled, we have steps forward to clean up or deal with the remaining instructions.

dave-j-runtime-pack-boundary-execution-plan.md (17.4 KB)

In addition we have an updated audit on these instructions so we can not only track the work completed today, but also allow to check project as we move forward with the future changes.

dave-j-runtime-pack-boundary-runtime-audit.md (11.3 KB)

Your PR and above work has been all pushed and merged into the Dev branch. On a side note, GLM 5.2 is the model in BrainDrive Models (I have one remaining interface task which deals with cost but I will get to that) and GLM 5.2 should be the default model on the client side for BrainDrive Provider and Openrouter. Tomorrow I plan on doing a few test runs on the Release Automation so I can validate and/or work out any issues that hit during live runs.

5

@DJJones thanks I had the AI’s take a look and this is what they said. The finance starter pack issue is something I missed so I’ll take that one. Let me know what you think on the others when you have a chance.

Thanks
Dave W.

Thanks Dave J — this plan looks right to me. I agree with the conservative approach: we should not scorched-earth every runtime instruction just because it’s prose. Runtime still needs to provide state, capabilities, file context, and safety/scoping constraints. The boundary I want us to hold is:

▎ Starter-pack markdown owns owner-visible assistant behavior. Runtime supplies state, capabilities, and code-enforced safety.

I pulled the latest origin/dev and confirmed your Phase 1 / runtime cleanup is merged:

  • PR #181 starter-pack changes are on dev
  • Budget app retirement is on dev
  • GLM 5.2 default is on dev
  • Runtime forced-write cleanup is on dev
  • App-scope / child-app prompt injection removal is on dev

That’s real progress, and the prompt-level early-write and Budget/app-scope issues are much better now.

That said, I dug into the gateway code and I think there’s a more serious issue than the four prompt functions — and it means the early-write problem isn’t actually fixed yet. Below in priority order.

  1. The starter snapshot auto-write path is a second, model-independent early-write source (blocker)

There’s a runtime code path that writes/merges starter artifacts directly, regardless of what the model does:

  • gateway/server.ts:754-768 calls persistStarterProjectArtifactSnapshots(…) on every /message POST that carries a projectId — i.e. every chat turn in a starter project.
  • persistStarterProjectArtifactSnapshot(…) (~3807-3818) then merges an “Owner Conversation Snapshot” + derived anchors into documents//spec.md and documents//plan.md whenever those files exist.
  • buildStarterDerivedAnchors(…) (~3897-3950) hardcodes persona/scenario literals — e.g. spec.push(“Finance signal: Katie wants to get better with money…”) and “knee injury history means Katie needs safe next steps” — pushed in on a regex match of the owner’s text.
  • starterSnapshotProjectIds(…) (~3833-3847) fans this out: for your-agent / braindrive-plus-one it regex-detects career/finance/fitness/relationships from owner text and writes snapshots into those project folders too, so one chat can mutate up to five spec/plan files.

The important part: Phase 1 removed the prompt nudge, but this path writes durable owner files mid-interview no matter which model we run. So GLM 5.2 won’t change this behavior. I don’t think we should consider the early-write issue closed until this is handled. It’s also where the “Katie” persona literals live, so it’s the runtime source of the over-fit we couldn’t see from the pack-only harness.

Proposed action:

  • Treat this as blocker-level. Either remove/disable the auto-write path, or reclassify it explicitly with KEEP / DELETE / MOVE / CODIFY like the prompt functions.
  • Remove the persona/scenario-derived anchors from runtime entirely.
  • Add tests proving runtime does not auto-write starter spec.md / plan.md outside the approved assistant/tool/write flow.
  1. Profile heading mismatch — let’s resolve before dev→main

Current dev has a mismatch:

  • starter-pack profile begins with # Your Profile
  • memory/init.ts still seeds # Owner Profile
  • memory/init.test.ts still expects # Owner Profile

My call: # Your Profile is canonical — it matches the second-person owner voice we use everywhere (Your Goals / Your Plan). So I’d update init.ts + init.test.ts to # Your Profile unless you see a reason not to. Either way let’s decide it explicitly before this rides to main rather than leaving the suite red.

  1. Live runtime-sim regression coverage

The pack-only harness missed the original issue because the hidden behavior came from the live gateway-assembled context. Before we delete the remaining per-domain runtime guards, I think we need a runtime-sim regression that checks the assembled live prompt/context.

It should catch:

  • forced write timing outside starter-pack gates
  • Finance statement/budget/app-scope behavior
  • child-app/app-scope prompt text
  • page-specific interview behavior remaining in runtime once we decide it should move/delete

And importantly, it should also catch the non-prompt runtime mutation from item #1 — fail if the snapshot auto-write path writes starter spec.md/plan.md outside the approved flow.

  1. Leave the remaining per-domain runtime guards for Phase 3

I agree with not deleting buildProjectSpecificTurnGuard immediately. It still contains page-specific behavior, so it doesn’t fully match the boundary, but it’s less urgent than the auto-write path.

Once runtime-sim coverage exists, we can decide guard-by-guard:

  • delete if the pack already covers it,
  • move if the pack needs it,
  • codify if it’s a safety or correctness guarantee,
  • or keep temporarily with an explicit follow-up.

On my side (pack lane)

A couple of Finance pack lines still nudge toward statement/upload/budget-workflow behavior:

  • finance/AGENT.md: “upload, statement, spending, report, or other execution work”
  • finance/run-interview.md: “make the first step a small statement-gathering or rough-estimate task”

These are mine to fix — I’ll reframe them so Finance writes provisional Your Goals / Your Plan from rough owner-provided estimates, marks exact numbers unknown, and makes exact balance/statement gathering a later plan step only when useful. Flagging so we don’t both touch them.

Net: Phase 1 was the right conservative move and a real improvement. The two things I’d prioritize next are (1) the starter snapshot auto-write path — I think that’s the actual remaining early-write source — and (2) the profile heading/test mismatch before dev→main. After that, runtime-sim coverage should be the gate before any broader Phase 3 cleanup.

6

@DJJones finance changes from my side in the following PR aligned with another change designed to better state the write goal for spec and build plans: Starter pack: explicit write framing in page AGENT.md + Finance defer fix by davewaring · Pull Request #184 · BrainDriveAI/BrainDrive · GitHub

Any questions or issues let me know.

Thanks
Dave W.

7

Your PR is merged.

Based on your post, I created an execution plan for the remaining work and implemented it.

dave-w-runtime-followup-execution-plan.md (16.4 KB)

Which is now merged in the dev.

I updated the model pricing to $1.24 and #3.90 (30% over Openrouter list price for GLM 5.2)

Next on my list is working with the release automation

8

Great I just did a manual run through finance and it worked well. The convo abilities of GLM are great with the starter pack. The only downside I am seeing so far is speed. It’s significantly slower than Haiku but I would rather have slow and accurate at this stage.

Thanks,
Dave W.

1 Like
9

I just ran, tested, tweaked and fixed the Release Automation. It did have some speed bumps as I fully expected and we were able to work through the issues. It took about 15 minutes from start to finish, I went ahead and posted the new version on Release as 26.6.28.

Unless something explodes we are good for tomorrow. I will send out the change log probably in the morning as normal but in the meantime…

CHANGELOG_2026-06-28.md (3.4 KB)

10

@DJJones I’m running into an issue with the remaining runtime instructions that we deferred to phase 3 earlier in this thread so I need to go ahead and remove them so I can get the your agent page working as intended.

Pls review the below and unless you see issues implement when you have a chance.

Thanks
Dave W.

Runtime/pack boundary — Phase 3 (the remaining cleanup)

Principle’s settled (runtime = state + safety; pack owns behavior). Phases 1–2 are done — forced-write triggers, app-scope injection, the snapshot auto-write path, and the Fitness guards are out. Here’s the remaining work. I cross-referenced the pack against every remaining runtime guard, so the dispositions below are confirmed, not “go check.” Full line-by-line audit is in runtime-pack-boundary.md.

  1. Make buildProjectChatContext() neutral / file-driven — do this first; it unblocks the rest.
    Today it hardcodes the page scaffold: read spec.md + plan.md, the Finance-specific run-interview.md/run-planning.md, “the project has spec.md and plan.md,” etc. Replace with:
  • “You’re in project {projectId}.”
  • the current file list (keep as-is)
  • “Read AGENT.md first if present, then AGENT-user.md; use the project’s other instruction files per its AGENT.md.”
  • keep the file-list / delete / list accuracy guidance
  • drop every hardcoded assumption that a project has spec.md / plan.md / run-interview.md / run-planning.md.

Why first: this is what forces every page to look identical. Until it’s neutral, Your Agent (or any page) can’t have a different file shape.

  1. Delete buildProjectSpecificTurnGuard() entirely — confirmed fully covered by the pack.
    Each per-domain guard injects two behaviors: mirror the owner’s exact phrases and record-unknown / don’t-repeat-the-setup-question. Both are already in that page’s run-interview.md (often word-for-word and better-worded):
  • Career → career/run-interview.md:23 · Finance → finance/run-interview.md:23 (incl. “say each amount back”) · Relationships → :23 + :29 · New Project → :23 · Your Agent → your-agent/run-interview.md:32.

Clincher: this function is also the last hiding spot for persona-literal cheats — Career keys on “product marketing” / “marketing coordinator”, New Project on “vegetable garden” / “tomatoes” / “herbs” / “peppers”. Same overfit we stripped from the pack, still live in the gateway. Deleting it removes the duplication and the cheats.

  1. Delete buildProjectConversationGuard()'s behavioral injection — also confirmed covered.
  • “don’t re-ask the same label/question; mark unknown; pick a different action” → covered by every run-interview.md (“adjacent info → record unknown → move to a different question”).
  • “ask exactly one question, at most one question mark” → covered by base/AGENT.md:46 + every run-interview.
  • The function also computes the list of already-asked labels/questions — that’s conversation state, not behavior. Either drop it (the model already sees its own prior turns) or keep it telemetry-only, never injected into the prompt. Your call; everything behavioral comes out.
  1. Gate every removal with the runtime-sim regression.
    Extend the project-chat-context.test.ts you built for the snapshot fix. For each removal assert: (a) the runtime no longer contains the behavioral string (the per-domain “guard” strings, the spec.md/plan.md/run-interview.md hardcoding, the one-question rule), and (b) the assembled context is still complete without it.

**Done = ** runtime injects only: active project id, current file list, tool/capability constraints, and safety. All interview / mirroring / no-repeat / routing behavior lives in the pack — one inspectable surface.

11

Sorry about the delay, I needed to wrap up some other code things I had started before I read the above. I have the above already going into action, I am not sure if we want to push to a release until after I you review the work.. So I am going to recommend that I tell you when it done and pushed into the dev branch. You can check it out, we can make changes… and when satisfied I will create a new release.

If you are expecting something else, just let me know.

12

no worries I just posted it so you are not late.

yep that sounds good I think we review everything together in our morning call that is on dev and then agree there what to push to release on the call so we are all clear.

Thanks,
Dave W.

13

It is in the Dev branch now, below is the PR description just for reference:

Summary

This implements DW’s Runtime Phase 3 cleanup for the gateway prompt boundary.

  • Makes project chat context neutral and file-driven instead of assuming every page has spec.md, plan.md, run-interview.md, and run-planning.md.
  • Removes runtime conversation guard prompt injection for repeated questions, unknown marking, one-question behavior, and artifact write/update next actions.
  • Deletes page-specific runtime interview guards for Career, Finance, Relationships, New Project, and Your Agent.
  • Keeps runtime-owned context: active project id, current project file list, file-list freshness guidance, index.md discovery guidance, scoped delete/list accuracy guidance, and path safety.

Why

Starter-pack markdown now owns owner-visible interview behavior. Runtime should only provide state, capabilities, and safety. The deleted guards duplicated pack behavior and included scenario-literal anchors that overfit specific test personas.

Validation

  • cd builds/typescript && npm run test -- gateway/project-chat-context.test.ts passed: 6 tests.
  • Forbidden runtime guard/source searches passed with no matches in builds/typescript/gateway/server.ts.
  • cd builds/typescript && npm run build passed.
  • cd builds/typescript && npm run test still fails in the existing memory/starter-pack-draft3-layout.test.ts lint only: 26 files passed, 1 failed; 144 tests passed, 1 failed. The remaining failure is the known starter-pack layout lint unrelated to this gateway-only change.

Scope Notes

No provider config, secrets, hosted URLs, starter-pack templates, Budget app, child-app, or snapshot auto-write code was changed.

1 Like
14

@DJJones Thanks. I just opened a pull request to change the “Your Agent” page so it no longer has a spec and a build plan associated with it.

The “Your Agent” page now has only an agent file that describes its job and to route things to the appropriate page.

Tested with both Haiku and GLM 5.2 and it’s working well.

Any questions or issues let me know.

Thanks
Dave W.