Migrate review to skill-based architecture by factory-nizar · Pull Request #64 · Factory-AI/droid-action

factory-nizar · 2026-03-23T21:42:55Z

Summary

Migrates the review methodology into a skill-based architecture. The canonical review skill lives in the factory-plugins core plugin (plugins/core/skills/review/SKILL.md) and is loaded at runtime by CI.

Skill: plugins/core/skills/review/SKILL.md

How It Works

load-skill.ts tries two sources in order:

Local plugin cache (~/.factory/plugins/cache/factory-plugins/core/<hash>/skills/review/SKILL.md) -- used by the Droid CLI locally
GitHub fallback (fetches from raw.githubusercontent.com/Factory-AI/factory-plugins/main/...) -- used in CI where the plugin cache does not exist

The shared methodology (bug patterns, analysis discipline, reporting gate, priority levels, finding format) is extracted via  /  markers and injected into both the candidates and validator prompt templates.

Changes

New: Skill Loader (`src/utils/load-skill.ts`)

loadSkill(name) -- loads from local cache, falls back to GitHub
formatSkillSection(content) -- extracts shared methodology and wraps in <code_review_methodology> tags
extractSharedMethodology(content) -- parses marker-delimited section

Skill Integration

PreparedContext extended with reviewSkillContent field
createPrompt passes reviewSkillContent through to prompt generators
review.ts, review-validator.ts, generate-review-prompt.ts load the skill and pass it to createPrompt
review-candidates-prompt.ts and review-validator-prompt.ts inject skill via formatSkillSection()

Testing

378 tests pass
Typecheck clean
Format clean

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix(review): escape suggestion blocks in prompt Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix(review): enforce suggestion-only replacements Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> update

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

- Review methodology lives in factory-plugins core plugin (code-review skill) - CI loads shared methodology from plugin cache at runtime - Skill uses BEGIN/END_SHARED_METHODOLOGY markers to separate: - Shared: bug patterns, analysis discipline, reporting gate, priorities, format - Shallow (TUI): local context detection + local findings output - Deep (CI): pre-computed artifacts, MCP tools, posting, validator flow - CI always posts comments (deep flow); TUI skill returns locally (shallow flow) - formatSkillSection extracts only shared methodology for CI prompt injection Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

factory-droid · 2026-03-24T00:38:43Z

Droid finished @factory-nizar's task —— View job

Droid reviewed this PR and found 3 issues (3 P1).

Summary: The skill-loader integration has a genuine path-resolution bug (HOME fallback to ~ doesn't expand in Node/Bun) and both prompt templates can silently lose their review methodology when skill loading fails, since the previously hardcoded guidance was removed without a fallback. Consider using os.homedir() and either failing fast on missing skills or keeping a minimal inlined methodology.

src/utils/load-skill.ts

src/create-prompt/templates/review-validator-prompt.ts

src/create-prompt/templates/review-candidates-prompt.ts

factory-droid

The skill-loader integration has a genuine path-resolution bug (HOME fallback to ~ doesn't expand in Node/Bun) and both prompt templates can silently lose their review methodology when skill loading fails, since the previously hardcoded guidance was removed without a fallback. Consider using os.homedir() and either failing fast on missing skills or keeping a minimal inlined methodology.

- load-skill.ts: try local plugin cache first, then fetch from GitHub - Re-wire skill loading after rebase (review-prompt.ts removed by PR #59) - Inject skill into candidates and validator prompt templates Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

jonathan-factory

Looks good, might be good to confirm that the plugin gets properly loaded before the action runs

Possible that it only gets installed in interactive mode

The factory-plugins repo uses 'master' as its default branch, not 'main'. The GitHub fallback was always 404ing in CI. Added unit tests for the loader functions and an E2E test that verifies the skill downloads from GitHub with no local cache (simulating a CI runner). Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

factory-nizar · 2026-03-25T17:24:29Z

i'll test on the mono-repo now that the plugin is merged!

factory-nizar · 2026-03-25T17:53:47Z

@jonathan-factory
loaded review skill! https://github.com/Factory-AI/factory-mono/pull/11403:

Also added some additional tests on the branch!

factory-nizar mentioned this pull request Mar 23, 2026

Add review skill to core plugin Factory-AI/factory-plugins#11

Merged

factory-nizar force-pushed the feat/review-skill-migration branch 4 times, most recently from 576653a to a3eefc9 Compare March 23, 2026 22:31

factory-nizar and others added 10 commits March 23, 2026 16:01

fix(review): allow insert-only suggestions

985c572

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

fix(review): clarify insert-only suggestions

a7eb364

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

fix(review): keep anchors consistent across phases

49131a6

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

refactor(review): slim suggestion rules

4dfbf50

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

update 250 to 100

7e121d3

Add validator_model input to allow separate model for validator pass

601d683

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

Add strict deduplication rules to validator prompt

33bd45e

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

Balanced: soft candidates + strict P2 validator filtering

ff0dc3a

factory-nizar force-pushed the feat/review-skill-migration branch from a3eefc9 to 0d493c4 Compare March 23, 2026 23:07

factory-nizar changed the title ~~Migrate review to skill-based architecture with shallow default~~ Migrate review to skill-based architecture Mar 23, 2026

factory-nizar changed the base branch from feat/sb-v3-balanced to dev March 23, 2026 23:34

factory-nizar force-pushed the feat/review-skill-migration branch 4 times, most recently from 07f09c6 to 7e4fc66 Compare March 24, 2026 00:32

factory-nizar marked this pull request as ready for review March 24, 2026 00:38

factory-nizar requested a review from jonathan-factory March 24, 2026 00:38

factory-droid bot reviewed Mar 24, 2026

View reviewed changes

src/utils/load-skill.ts Outdated Show resolved Hide resolved

factory-droid bot reviewed Mar 24, 2026

View reviewed changes

src/create-prompt/templates/review-validator-prompt.ts Show resolved Hide resolved

factory-droid bot reviewed Mar 24, 2026

View reviewed changes

src/create-prompt/templates/review-candidates-prompt.ts Show resolved Hide resolved

factory-droid bot reviewed Mar 24, 2026

View reviewed changes

factory-nizar force-pushed the feat/review-skill-migration branch from 7e4fc66 to 7c007f8 Compare March 24, 2026 00:53

jonathan-factory reviewed Mar 24, 2026

View reviewed changes

factory-nizar requested a review from jonathan-factory March 25, 2026 17:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate review to skill-based architecture#64

Migrate review to skill-based architecture#64
factory-nizar wants to merge 12 commits intodevfrom
feat/review-skill-migration

factory-nizar commented Mar 23, 2026 •

edited

Loading

Uh oh!

factory-droid bot commented Mar 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

factory-droid bot left a comment

Uh oh!

jonathan-factory left a comment

Uh oh!

factory-nizar commented Mar 25, 2026

Uh oh!

factory-nizar commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

factory-nizar commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How It Works

Changes

New: Skill Loader (src/utils/load-skill.ts)

Skill Integration

Testing

Uh oh!

factory-droid bot commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

factory-droid bot left a comment

Choose a reason for hiding this comment

Uh oh!

jonathan-factory left a comment

Choose a reason for hiding this comment

Uh oh!

factory-nizar commented Mar 25, 2026

Uh oh!

factory-nizar commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

factory-nizar commented Mar 23, 2026 •

edited

Loading

New: Skill Loader (`src/utils/load-skill.ts`)

factory-droid bot commented Mar 24, 2026 •

edited

Loading