Skip to content

feat: project-local .gstack/ data storage with auto-migration#505

Open
alfadb wants to merge 11 commits intogarrytan:mainfrom
alfadb:feat/project-local-gstack-data
Open

feat: project-local .gstack/ data storage with auto-migration#505
alfadb wants to merge 11 commits intogarrytan:mainfrom
alfadb:feat/project-local-gstack-data

Conversation

@alfadb
Copy link
Copy Markdown

@alfadb alfadb commented Mar 26, 2026

Summary

Moves project-scoped data from ~/.gstack/projects/$SLUG/ to {git-root}/.gstack/, enabling:

  • Team collaboration: design docs, review logs, test plans are now git-tracked and visible to all team members
  • Data portability: project data follows the repo when cloned to a new machine
  • Clean separation: machine-local state (browse daemon, logs, caches) lives in .gstack/local/ (gitignored), shared data lives in .gstack/ root (tracked)
  • Zero-friction migration: auto-migrates on first gstack use per project — handles both legacy global data AND existing flat-structure local files

Directory structure

{project}/.gstack/
├── {BRANCH}-reviews.jsonl      ← git-tracked
├── designs/                     ← git-tracked (design docs, architecture, audits)
├── plans/                       ← git-tracked (test plans, autoplan, eng review, CEO plans)
├── evals/                       ← git-tracked (E2E test baselines)
├── local/                       ← gitignored (machine-local)
│   ├── browse.json
│   ├── browse-*.log
│   └── repo-mode.json
├── .gitignore                   ← contains "local/" and ".migrated"
└── .migrated                    ← gitignored, marks auto-migration done

Auto-migration (two phases, fully automatic)

On first gstack skill invocation per project, the preamble runs gstack-migrate-local:

Phase 1 — Global → Local: ~/.gstack/projects/$SLUG/*.gstack/{designs,plans,...}/
Phase 2 — Flat → Organized: .gstack/*-design-*.md.gstack/designs/ (reorganize existing files in place)

  • Deduplicates automatically (same-name files → keep the one in the subdirectory)
  • Handles browse.json/logs → .gstack/local/, architecture docs, implementation plans, roadmaps
  • Creates .migrated marker to skip on subsequent runs
  • Manual: ~/.claude/skills/gstack/bin/gstack-migrate-local [--dry-run]

Changes

New files:

  • bin/gstack-migrate-local: standalone migration script (idempotent, --dry-run supported)

Core infrastructure:

  • bin/gstack-slug: new PROJECT_DATA_DIR output variable
  • scripts/resolvers/preamble.ts + scripts/gen-skill-docs.ts: auto-runs migration in preamble

Data path updates:

  • bin/gstack-review-log, bin/gstack-review-read, bin/gstack-repo-mode: use project-local paths
  • browse/src/config.ts: browse state moves to .gstack/local/; .gitignore management creates .gstack/.gitignore instead of blanket exclusion
  • test/helpers/eval-store.ts: evals write to .gstack/evals/
  • All resolvers (utility.ts, design.ts, testing.ts, review.ts) and 12 .tmpl files updated
  • All SKILL.md files regenerated

Backward compatibility

  • Reads: all read paths check project-local first, fall back to ~/.gstack/projects/$SLUG/
  • Writes: only write to project-local paths
  • Auto-migration: runs once per project, idempotent, non-blocking on failure
  • Browse daemon: old daemon auto-recovers on next invocation at new path

Breaking changes

  • Running browse daemons will not be found after upgrade (auto-recovers on restart)
  • Projects with .gstack/ in .gitignore will have that line removed and replaced with .gstack/.gitignore containing local/

Rebase status

Rebased on upstream v0.14.3.0 (2026-03-31). All upstream changes (Review Army, ship idempotency, scope drift) integrated cleanly. SKILL.md regenerated post-rebase.

Merge conflict analysis (updated 2026-03-31)

Reviewed open PRs for file-level overlap against current diff (55 files, 10 commits).

Resolved since last analysis

Current potential overlaps

PR Conflict files Risk
#684 (fix ship reruns) ship/SKILL.md.tmpl Low — they fix idempotency (already in v0.14.3.0), we change data paths
#685 (preview deploy) ship/SKILL.md.tmpl Low — adds new step, different area from our path changes
#632 (ship log) bin/ scripts Low — new file, no overlap with our bin changes
#634 (GitLab land-and-deploy) land-and-deploy/SKILL.md.tmpl Low — adds GitLab support, different lines from our path changes
#664 (browse security fixes) browse/src/config.ts Medium — both modify config resolution, but different functions

No other open PR addresses project-local data storage. This PR remains unique in scope.

Test plan

  • bun run gen:skill-docs generates all SKILL.md files successfully
  • bun test passes (only pre-existing "Contributor mode" failures, unrelated)
  • gstack-slug outputs 3 variables: SLUG, BRANCH, PROJECT_DATA_DIR
  • All .tmpl legacy path references are in fallback positions only (verified via grep)
  • gstack-migrate-local --dry-run correctly identifies files to migrate
  • Migration tested on real project — both phases work correctly
  • Rebased cleanly on v0.14.3.0 (Review Army + ship idempotency + scope drift)
  • SKILL.md regenerated post-rebase, all resolver outputs correct

🤖 Generated with Claude Code

@alfadb
Copy link
Copy Markdown
Author

alfadb commented Mar 26, 2026

Merge conflict analysis (updated)

Reviewed all 50 open PRs for file-level overlap. 8 PRs intersect with this one:

High risk (manual merge needed)

PR Conflict files Nature
#472 (JSON validation for review-log) bin/gstack-review-log They add JSON validation, we change the write path. Both changes should coexist: validate → write to $PROJECT_DATA_DIR
#440 (launchPersistentContext + cookie persistence) browse/src/config.ts They add storageFile path + cookie persistence, we restructure stateDir to .gstack/local/. Needs careful merge — both modify resolveConfig() and ensureStateDir()
#445 (add bin/ to .gitignore) bin/gstack-slug, bin/gstack-review-log, bin/gstack-review-read, bin/gstack-repo-mode They git rm --cached all bin/ scripts. Structural conflict — if they go first, our bin changes need to be re-added after

Medium risk (different lines, likely auto-merge)

PR Conflict files Nature
#471 (Codex -C flag) scripts/resolvers/design.ts, review.ts Different lines: codex exec flags vs data paths
#470 (temp file cleanup) scripts/resolvers/design.ts, review.ts Different lines: rm -f cleanup vs path changes
#500/#501 (install root inference) scripts/gen-skill-docs.ts Different areas: setup/install logic vs preamble/path changes

Low risk (cosmetic overlap)

PR Conflict files Nature
#452 (review section in CLAUDE.md) docs/skills.md Different lines
#416 (community telemetry) scripts/gen-skill-docs.ts Large PR but touches telemetry, not data paths. Needs regen (bun run gen:skill-docs) after rebase

No overlap

No other PR addresses project-local data storage or the ~/.gstack/projects/.gstack/ migration. This PR is unique in scope.

Happy to rebase against any of these after they land.

@alfadb alfadb changed the title feat: project-local .gstack/ data storage for collaboration feat: project-local .gstack/ data storage with auto-migration Mar 26, 2026
@alfadb alfadb force-pushed the feat/project-local-gstack-data branch 8 times, most recently from 9a133d2 to fb44e45 Compare March 31, 2026 05:22
@alfadb
Copy link
Copy Markdown
Author

alfadb commented Mar 31, 2026

Rebased on upstream v0.14.3.0 (2026-03-31). All 3 upstream commits (Review Army, ship idempotency, scope drift) integrated cleanly — no conflicts. SKILL.md regenerated post-rebase.

Updated PR description with current merge conflict analysis. Previous conflicts (#472, #440, #445) have all been merged upstream and are no longer relevant.

alfadb and others added 11 commits April 2, 2026 13:08
gstack-slug now outputs a third variable PROJECT_DATA_DIR pointing to
{git-root}/.gstack, enabling project-local data storage instead of the
global ~/.gstack/projects/$SLUG path. Falls back to the legacy global
path for non-git environments.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- gstack-review-log: writes to $PROJECT_DATA_DIR instead of
  ~/.gstack/projects/$SLUG
- gstack-review-read: reads from project-local first, falls back to
  legacy global path for backward compatibility
- gstack-repo-mode: caches repo-mode.json to .gstack/local/ (gitignored)
  instead of ~/.gstack/projects/$SLUG

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…gement

Browse daemon state (browse.json, logs) now lives in .gstack/local/
instead of .gstack/ root. The .gitignore strategy changes from blanket
.gstack/ exclusion to a .gstack/.gitignore containing only "local/",
enabling git-tracked shared data (designs, plans, reviews) in .gstack/.

Migration: automatically removes .gstack/ line from project .gitignore
and creates .gstack/.gitignore with local/ entry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
getProjectEvalDir() now resolves via git root first (.gstack/evals/),
falling back to the legacy slug-based ~/.gstack/projects/$SLUG/evals/
path for backward compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- utility.ts: generateSlugSetup uses $PROJECT_DATA_DIR
- design.ts: design audit docs write to .gstack/designs/
- testing.ts: test plans write to .gstack/plans/
- review.ts: design doc lookup checks .gstack/designs/ first,
  falls back to legacy ~/.gstack/projects/$SLUG/
- gen-skill-docs.ts: sync local resolver copies with the above

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nd docs

- 12 .tmpl files: all ~/.gstack/projects/$SLUG/ references replaced with
  $PROJECT_DATA_DIR paths, legacy paths kept as fallback for reads
- 28 SKILL.md files regenerated from updated templates
- test/skill-validation.test.ts: adapt assertions for 3-line gstack-slug
  output and $PROJECT_DATA_DIR greptile patterns
- review/greptile-triage.md: use gstack-slug for PROJECT_DATA_DIR
- docs/skills.md: update path references

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bin/gstack-migrate-local: standalone migration script (--dry-run supported)
that handles two phases:
- Phase 1: ~/.gstack/projects/$SLUG/ → .gstack/{designs,plans,evals,...}/
- Phase 2: .gstack/ root flat files → subdirectories (reorganize in place)

Deduplicates automatically, handles browse.json/logs → .gstack/local/,
architecture docs, implementation plans, and roadmaps.

Integrated into preamble: auto-runs on first gstack use per project,
creates .migrated marker to skip on subsequent runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- land-and-deploy/SKILL.md.tmpl: read from $PROJECT_DATA_DIR with
  ~/.gstack/projects/$SLUG fallback, write to $PROJECT_DATA_DIR
- bin/gstack-migrate-local: add land-deploy-confirmed to Phase 1
  legacy migration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The rebase conflict resolution committed stale SKILL.md files that
were missing the auto-migrate code block from preamble.ts. Regenerated
all 24 SKILL.md files via gen:skill-docs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Upstream v0.14.3.0 changed ship, review, and autoplan templates.
Rebase merged the resolver code correctly but SKILL.md build
artifacts were stale. Regenerated via setup to reflect combined
upstream + project-local resolver output.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Upstream v0.15.0.0 added /checkpoint and /health skills. Regenerating
injects the project-local auto-migrate preamble and updates slug setup
paths to use $PROJECT_DATA_DIR.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alfadb alfadb force-pushed the feat/project-local-gstack-data branch from 3e96186 to 2fabdb8 Compare April 2, 2026 05:18
Copilot AI review requested due to automatic review settings April 2, 2026 05:18
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates gstack’s project-scoped artifacts from ~/.gstack/projects/$SLUG/ into a repo-local .gstack/ directory (with .gstack/local/ for machine-local state), and wires in automatic one-time migration via the skill preamble.

Changes:

  • Add PROJECT_DATA_DIR to gstack-slug and update scripts/docs to prefer $PROJECT_DATA_DIR with legacy fallbacks.
  • Introduce bin/gstack-migrate-local and auto-run it (once) from skill preambles.
  • Move browse daemon state into .gstack/local/ and manage .gstack/.gitignore + cleanup of blanket .gstack/ ignores.

Reviewed changes

Copilot reviewed 51 out of 57 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/skill-validation.test.ts Updates validation expectations for $PROJECT_DATA_DIR and new gstack-slug output.
test/helpers/eval-store.ts Writes evals to repo-local .gstack/evals/ with legacy read fallback.
ship/SKILL.md.tmpl Updates guidance to use $PROJECT_DATA_DIR; removes review-log persistence step from template.
ship/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR paths.
setup-deploy/SKILL.md Adds auto-migration preamble.
setup-browser-cookies/SKILL.md Adds auto-migration preamble.
scripts/resolvers/utility.ts Updates SLUG_SETUP generator to create $PROJECT_DATA_DIR.
scripts/resolvers/testing.ts Updates test plan artifact paths to .gstack/plans/.
scripts/resolvers/review.ts Updates design doc discovery to prefer $PROJECT_DATA_DIR/designs/ with legacy fallback.
scripts/resolvers/preamble.ts Adds auto-migration hook into the shared preamble resolver.
scripts/resolvers/design.ts Updates design audit paths into .gstack/designs/.
review/greptile-triage.md Switches per-project greptile history to $PROJECT_DATA_DIR.
review/SKILL.md Adds auto-migration preamble.
retro/SKILL.md.tmpl Switches ship review-log reads to $PROJECT_DATA_DIR.
retro/SKILL.md Regenerated skill: switches ship review-log reads to $PROJECT_DATA_DIR.
qa/SKILL.md.tmpl Updates test plan lookup paths to .gstack/plans/.
qa/SKILL.md Regenerated skill: adds auto-migration preamble and uses .gstack/plans/.
qa-only/SKILL.md.tmpl Updates test plan lookup paths to .gstack/plans/.
qa-only/SKILL.md Regenerated skill: adds auto-migration preamble and uses .gstack/plans/.
plan-eng-review/SKILL.md.tmpl Updates design doc lookup to prefer $PROJECT_DATA_DIR/designs/.
plan-eng-review/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR/designs/.
plan-design-review/SKILL.md Adds auto-migration preamble.
plan-ceo-review/SKILL.md.tmpl Updates CEO plan + handoff paths to $PROJECT_DATA_DIR.
plan-ceo-review/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR.
office-hours/SKILL.md.tmpl Updates design doc discovery/writes to .gstack/.
office-hours/SKILL.md Regenerated skill: adds auto-migration preamble and uses .gstack/.
learn/SKILL.md Adds auto-migration preamble.
land-and-deploy/SKILL.md.tmpl Uses $PROJECT_DATA_DIR for deploy confirmation and logging, with legacy fallback.
land-and-deploy/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR.
investigate/SKILL.md Adds auto-migration preamble.
health/SKILL.md Adds auto-migration preamble; updates mkdir targets to $PROJECT_DATA_DIR.
document-release/SKILL.md Adds auto-migration preamble.
docs/skills.md Updates documentation references from ~/.gstack/projects/ to .gstack/.
design-shotgun/SKILL.md Adds auto-migration preamble.
design-review/SKILL.md.tmpl Moves design review report dir into $PROJECT_DATA_DIR/designs/.
design-review/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR/designs/.
design-html/SKILL.md Adds auto-migration preamble.
design-consultation/SKILL.md.tmpl Prefers $PROJECT_DATA_DIR for office-hours artifacts with legacy fallback.
design-consultation/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR.
cso/SKILL.md Adds auto-migration preamble.
connect-chrome/SKILL.md Adds auto-migration preamble.
codex/SKILL.md Adds auto-migration preamble.
checkpoint/SKILL.md Adds auto-migration preamble; updates mkdir targets to $PROJECT_DATA_DIR.
canary/SKILL.md.tmpl Switches review-dashboard logging to $PROJECT_DATA_DIR.
canary/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR.
browse/src/config.ts Moves browse state into .gstack/local/ and manages .gstack/.gitignore + cleanup.
browse/SKILL.md Adds auto-migration preamble.
bin/gstack-slug Adds PROJECT_DATA_DIR output (repo-local .gstack/ when available).
bin/gstack-review-read Reads review logs from $PROJECT_DATA_DIR with legacy fallback.
bin/gstack-review-log Writes review logs to $PROJECT_DATA_DIR.
bin/gstack-repo-mode Moves repo-mode cache into .gstack/local/ when in a git repo.
bin/gstack-migrate-local New idempotent migration + reorganization script with --dry-run.
benchmark/SKILL.md Adds auto-migration preamble.
autoplan/SKILL.md.tmpl Writes restore/test-plan artifacts under $PROJECT_DATA_DIR/plans/.
autoplan/SKILL.md Regenerated skill: adds auto-migration preamble and uses $PROJECT_DATA_DIR/plans/.
SKILL.md Adds auto-migration preamble to the root skill.
Comments suppressed due to low confidence (3)

retro/SKILL.md.tmpl:1

  • This command changes behavior due to shell operator precedence: only the legacy cat output is piped to grep, and the $PROJECT_DATA_DIR cat is not filtered at all. Wrap the fallback cat commands in a subshell (or group) before piping, e.g. (cat ... || cat ...) | grep ... | grep ... || echo ..., so both sources are filtered consistently.
    retro/SKILL.md:1
  • Same precedence issue as the template: the first cat is not piped through the grep filters, so it can emit unfiltered JSONL and also bypass the intended NO_PLAN_DATA logic. Group the fallback cat commands before the pipeline so both new and legacy sources are filtered identically.
    plan-eng-review/SKILL.md.tmpl:1
  • $PROJECT_DATA_DIR is used here but is never initialized in this block (it previously derived paths from $SLUG). This will make the preferred lookup path empty/incorrect. Fix by initializing $PROJECT_DATA_DIR in the same block (e.g., eval \"$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)\") or by replacing the manual SLUG/BRANCH derivation with the standard {{SLUG_EVAL}} pattern so $PROJECT_DATA_DIR is always available.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

// stateDir write failed too — nothing more we can do
}
if (!content.match(/^\.migrated$/m)) {
content = content + '.migrated\n';
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If .gstack/.gitignore exists and does not end with a newline, appending '.migrated\\n' can merge with the previous entry (e.g., local/.migrated), making both patterns ineffective. Add a newline separator when needed (similar to the local/ block) before appending .migrated.

Suggested change
content = content + '.migrated\n';
const separator = content.length > 0 && !content.endsWith('\n') ? '\n' : '';
content = content + separator + '.migrated\n';

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants