Skip to content

Add One-Shot Skills Reliability and Guardrails#397

Merged
ppinchuk merged 7 commits intomainfrom
feature/skills-pr
Mar 26, 2026
Merged

Add One-Shot Skills Reliability and Guardrails#397
ppinchuk merged 7 commits intomainfrom
feature/skills-pr

Conversation

@bpulluta
Copy link
Collaborator

@bpulluta bpulluta commented Mar 17, 2026

Summary

This PR hardens the one-shot extraction skills to make setup and iteration more repeatable, simpler to follow, and safer against runtime misconfiguration.

What Changed

  • Updated extraction-run/SKILL.md
    • Added preflight checks before running.
    • Added explicit pass/fail gates:
      • treat runs with zero extracted jurisdictions as failed extraction quality.
      • treat config exceptions in logs as failures even if process exits successfully.
  • Updated plugin-config-setup/SKILL.md
    • Added non-negotiable runtime constraints.
    • Added canonical required heuristic_keywords structure with all required lists.
    • Added minimal run-config contract that prompts users to provide their own model name and client settings.
  • Updated document-retrieval/SKILL.md
    • Added required heuristic list expectations and explicit failure behavior when missing/empty.
    • Normalized keyword naming guidance to canonical forms.
  • Updated schema-creation/SKILL.md
    • Added quality gate requiring smoke runs to produce extracted rows, not just process completion.

Why

users can avoid common setup mistakes and get a working, repeatable baseline faster.

Testing Notes

  • Documentation/skill-only updates.
  • Guidance now reflects validated runtime constraints and triage behavior.

Copilot AI review requested due to automatic review settings March 17, 2026 19:15

This comment was marked as outdated.

This comment was marked as resolved.

@bpulluta

This comment was marked as duplicate.

This comment was marked as duplicate.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

This comment was marked as resolved.

@bpulluta
Copy link
Collaborator Author

@copilot open a new pull request to apply changes based on the comments in this thread

Copy link
Contributor

Copilot AI commented Mar 17, 2026

@bpulluta I've opened a new pull request, #399, to work on those changes. Once the pull request is ready, I'll request review from you.

…rmatting (#399)

* Initial plan

* Fix all review comments in skills documentation

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>
ppinchuk

This comment was marked as resolved.

Copy link
Collaborator Author

@bpulluta bpulluta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my review

@bpulluta bpulluta requested a review from ppinchuk March 26, 2026 22:42
Copy link
Collaborator

@ppinchuk ppinchuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@NatLabRockies NatLabRockies deleted a comment from bpulluta Mar 26, 2026
@ppinchuk ppinchuk merged commit fbe3d8e into main Mar 26, 2026
4 checks passed
@ppinchuk ppinchuk deleted the feature/skills-pr branch March 26, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants