Starter Pack Updates

BrainDrive Development
1

@DJJones starting a thread here to post Start Pack updates.

I’m making progress but not quite there yet with something that I am ready to replace what we have. Will keep working on it later this evening and in the AM.

Dave W.

1 Like
2

@DJJones Still not there but I think I may have found why I am struggling. I was thinking that all the instructions I had to optimize were in the starter pack, but it looks like there are instructions that sit outside the starter pack.

Since they are in the runtime I am kicking over to you.

Going forward I think our rule should be that any instructions on how the AI is to behave go into the starter pack and none outside it.

Unless there is a good reason not to do it this way pls remove all instructions that currently sit outside of the starter pack as per the below .md.

dave-j-runtime-pack-boundary-brief.md (5.7 KB)

Any questions or issues let me know.

Thanks
Dave

3

@DJJones PR Opened: Starter pack: de-cheat templates + add quality floors (playback gate, profile-propose, fitness baseline) by davewaring · Pull Request #181 · BrainDriveAI/BrainDrive · GitHub)

4

Because of how instructions are currently in the file I feared if we went scored earth on it all it would reduce the functionality of BrainDrive over all however at the same time there was some clear instructions that needed to be removed instantly…

First step I created an execution plan for handling the “now” problem and then once we get a chance to observe how these changes are handled, we have steps forward to clean up or deal with the remaining instructions.

dave-j-runtime-pack-boundary-execution-plan.md (17.4 KB)

In addition we have an updated audit on these instructions so we can not only track the work completed today, but also allow to check project as we move forward with the future changes.

dave-j-runtime-pack-boundary-runtime-audit.md (11.3 KB)

Your PR and above work has been all pushed and merged into the Dev branch. On a side note, GLM 5.2 is the model in BrainDrive Models (I have one remaining interface task which deals with cost but I will get to that) and GLM 5.2 should be the default model on the client side for BrainDrive Provider and Openrouter. Tomorrow I plan on doing a few test runs on the Release Automation so I can validate and/or work out any issues that hit during live runs.

5

@DJJones thanks I had the AI’s take a look and this is what they said. The finance starter pack issue is something I missed so I’ll take that one. Let me know what you think on the others when you have a chance.

Thanks
Dave W.

Thanks Dave J — this plan looks right to me. I agree with the conservative approach: we should not scorched-earth every runtime instruction just because it’s prose. Runtime still needs to provide state, capabilities, file context, and safety/scoping constraints. The boundary I want us to hold is:

▎ Starter-pack markdown owns owner-visible assistant behavior. Runtime supplies state, capabilities, and code-enforced safety.

I pulled the latest origin/dev and confirmed your Phase 1 / runtime cleanup is merged:

  • PR #181 starter-pack changes are on dev
  • Budget app retirement is on dev
  • GLM 5.2 default is on dev
  • Runtime forced-write cleanup is on dev
  • App-scope / child-app prompt injection removal is on dev

That’s real progress, and the prompt-level early-write and Budget/app-scope issues are much better now.

That said, I dug into the gateway code and I think there’s a more serious issue than the four prompt functions — and it means the early-write problem isn’t actually fixed yet. Below in priority order.

  1. The starter snapshot auto-write path is a second, model-independent early-write source (blocker)

There’s a runtime code path that writes/merges starter artifacts directly, regardless of what the model does:

  • gateway/server.ts:754-768 calls persistStarterProjectArtifactSnapshots(…) on every /message POST that carries a projectId — i.e. every chat turn in a starter project.
  • persistStarterProjectArtifactSnapshot(…) (~3807-3818) then merges an “Owner Conversation Snapshot” + derived anchors into documents//spec.md and documents//plan.md whenever those files exist.
  • buildStarterDerivedAnchors(…) (~3897-3950) hardcodes persona/scenario literals — e.g. spec.push(“Finance signal: Katie wants to get better with money…”) and “knee injury history means Katie needs safe next steps” — pushed in on a regex match of the owner’s text.
  • starterSnapshotProjectIds(…) (~3833-3847) fans this out: for your-agent / braindrive-plus-one it regex-detects career/finance/fitness/relationships from owner text and writes snapshots into those project folders too, so one chat can mutate up to five spec/plan files.

The important part: Phase 1 removed the prompt nudge, but this path writes durable owner files mid-interview no matter which model we run. So GLM 5.2 won’t change this behavior. I don’t think we should consider the early-write issue closed until this is handled. It’s also where the “Katie” persona literals live, so it’s the runtime source of the over-fit we couldn’t see from the pack-only harness.

Proposed action:

  • Treat this as blocker-level. Either remove/disable the auto-write path, or reclassify it explicitly with KEEP / DELETE / MOVE / CODIFY like the prompt functions.
  • Remove the persona/scenario-derived anchors from runtime entirely.
  • Add tests proving runtime does not auto-write starter spec.md / plan.md outside the approved assistant/tool/write flow.
  1. Profile heading mismatch — let’s resolve before dev→main

Current dev has a mismatch:

  • starter-pack profile begins with # Your Profile
  • memory/init.ts still seeds # Owner Profile
  • memory/init.test.ts still expects # Owner Profile

My call: # Your Profile is canonical — it matches the second-person owner voice we use everywhere (Your Goals / Your Plan). So I’d update init.ts + init.test.ts to # Your Profile unless you see a reason not to. Either way let’s decide it explicitly before this rides to main rather than leaving the suite red.

  1. Live runtime-sim regression coverage

The pack-only harness missed the original issue because the hidden behavior came from the live gateway-assembled context. Before we delete the remaining per-domain runtime guards, I think we need a runtime-sim regression that checks the assembled live prompt/context.

It should catch:

  • forced write timing outside starter-pack gates
  • Finance statement/budget/app-scope behavior
  • child-app/app-scope prompt text
  • page-specific interview behavior remaining in runtime once we decide it should move/delete

And importantly, it should also catch the non-prompt runtime mutation from item #1 — fail if the snapshot auto-write path writes starter spec.md/plan.md outside the approved flow.

  1. Leave the remaining per-domain runtime guards for Phase 3

I agree with not deleting buildProjectSpecificTurnGuard immediately. It still contains page-specific behavior, so it doesn’t fully match the boundary, but it’s less urgent than the auto-write path.

Once runtime-sim coverage exists, we can decide guard-by-guard:

  • delete if the pack already covers it,
  • move if the pack needs it,
  • codify if it’s a safety or correctness guarantee,
  • or keep temporarily with an explicit follow-up.

On my side (pack lane)

A couple of Finance pack lines still nudge toward statement/upload/budget-workflow behavior:

  • finance/AGENT.md: “upload, statement, spending, report, or other execution work”
  • finance/run-interview.md: “make the first step a small statement-gathering or rough-estimate task”

These are mine to fix — I’ll reframe them so Finance writes provisional Your Goals / Your Plan from rough owner-provided estimates, marks exact numbers unknown, and makes exact balance/statement gathering a later plan step only when useful. Flagging so we don’t both touch them.

Net: Phase 1 was the right conservative move and a real improvement. The two things I’d prioritize next are (1) the starter snapshot auto-write path — I think that’s the actual remaining early-write source — and (2) the profile heading/test mismatch before dev→main. After that, runtime-sim coverage should be the gate before any broader Phase 3 cleanup.

6

@DJJones finance changes from my side in the following PR aligned with another change designed to better state the write goal for spec and build plans: Starter pack: explicit write framing in page AGENT.md + Finance defer fix by davewaring · Pull Request #184 · BrainDriveAI/BrainDrive · GitHub

Any questions or issues let me know.

Thanks
Dave W.

7

Your PR is merged.

Based on your post, I created an execution plan for the remaining work and implemented it.

dave-w-runtime-followup-execution-plan.md (16.4 KB)

Which is now merged in the dev.

I updated the model pricing to $1.24 and #3.90 (30% over Openrouter list price for GLM 5.2)

Next on my list is working with the release automation

8

Great I just did a manual run through finance and it worked well. The convo abilities of GLM are great with the starter pack. The only downside I am seeing so far is speed. It’s significantly slower than Haiku but I would rather have slow and accurate at this stage.

Thanks,
Dave W.

1 Like