Skip to content

Add PVR triage taskflow#58

Draft
anticomputer wants to merge 28 commits intomainfrom
anticomputer/pvr-triage
Draft

Add PVR triage taskflow#58
anticomputer wants to merge 28 commits intomainfrom
anticomputer/pvr-triage

Conversation

@anticomputer
Copy link
Copy Markdown
Contributor

@anticomputer anticomputer commented Mar 2, 2026

Taskflow for triaging Private Vulnerability Reports arriving as GHSAs in triage state.

Fetches the advisory, verifies the vulnerability claim against source code at the affected version, and writes a structured markdown triage report with response draft.

Files

New MCP servers:

  • mcp_servers/pvr_ghsa.py — advisory fetch, version-to-SHA resolution, file fetch at ref, write-back (accept/reject/comment), similarity search, report read/save, pending-response listing, sent-marker creation
  • mcp_servers/reporter_reputation.py — per-reporter triage history in SQLite; record/history/score tools

New toolboxes:

  • toolboxes/pvr_ghsa.yaml — confirm-gated for write-back tools
  • toolboxes/reporter_reputation.yaml

New/updated taskflows:

  • taskflows/pvr_triage/pvr_triage.yaml — 8 tasks: quality gate with reputation-gated fast-close decision tree, patch status at HEAD, CVSS assessment, response draft, reputation update
  • taskflows/pvr_triage/pvr_respond.yaml — write-back taskflow (action=accept|reject|comment); calls mark_response_sent on success
  • taskflows/pvr_triage/pvr_triage_batch.yaml — inbox scoring with Age (days) column and tie-break sort by created_at
  • taskflows/pvr_triage/pvr_respond_batch.yaml — bulk respond: scans REPORT_DIR for unsent drafts and posts them in one session
  • taskflows/pvr_triage/README.md — full usage docs including all four taskflows
  • taskflows/pvr_triage/SCORING.md — authoritative scoring reference with three-path fast-close decision table and reputation x fast-close matrix

Tests:

  • tests/test_pvr_mcp.py — 32 unit tests (MCP tools + YAML structure)

Scripts:

  • scripts/run_pvr_triage.sh — local test/demo script

Usage

# Score entire triage inbox (with Age column)
./scripts/run_pvr_triage.sh batch owner/repo

# Triage one advisory
./scripts/run_pvr_triage.sh triage owner/repo GHSA-xxxx-xxxx-xxxx

# Post response for one advisory (confirm-gated)
./scripts/run_pvr_triage.sh respond owner/repo GHSA-xxxx-xxxx-xxxx accept
./scripts/run_pvr_triage.sh respond owner/repo GHSA-xxxx-xxxx-xxxx comment
./scripts/run_pvr_triage.sh respond owner/repo GHSA-xxxx-xxxx-xxxx reject

# Post all pending response drafts at once
./scripts/run_pvr_triage.sh respond_batch owner/repo comment

# Full demo pipeline
./scripts/run_pvr_triage.sh demo owner/repo

Requires GH_TOKEN (repo + security_events scope), AI_API_TOKEN, AI_API_ENDPOINT.

Advisory state transitions

  • accept — triage → draft (vulnerability confirmed, publishing intent)
  • comment — no state change (request more info or acknowledge)
  • reject — triage → closed (invalid or low-quality report)

Maintainer quality-of-life features

Age column in batch reportpvr_triage_batch now shows Age (days) for each advisory (today minus created_at). Ties in priority score are broken by age ascending (oldest first).

Reporter trust gates fast-close — the quality gate in pvr_triage uses a three-path decision tree:

  • High-trust reporter — always full verification, regardless of quality signals.
  • Skepticism reporter — fast-close on three absent quality signals alone (no prior similar report required).
  • Normal / no history — original four-condition logic (three absent signals + prior similar report).

Bulk respondpvr_respond_batch scans REPORT_DIR for drafts without a *_response_sent.md marker and posts them all in one session. Both pvr_respond and pvr_respond_batch write the sent marker on success so re-runs are idempotent.

Copilot AI review requested due to automatic review settings March 2, 2026 19:19
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “PVR triage” taskflow to triage Private Vulnerability Reports (draft repository GHSAs): fetch the advisory, verify claims against code at the affected version (not HEAD), and emit a structured markdown triage report.

Changes:

  • Introduces a new pvr_ghsa MCP server (gh CLI-based) to fetch draft advisories, resolve version tags to SHAs, fetch file contents at a ref, and save a triage report.
  • Adds a new 5-step pvr_triage taskflow plus a dedicated analyst personality and model configuration.
  • Wires the new MCP server via a toolbox YAML.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/seclab_taskflows/toolboxes/pvr_ghsa.yaml Adds toolbox wiring for the new PVR GHSA MCP server (stdio).
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage.yaml Defines the end-to-end PVR triage pipeline (fetch → verify → report → save).
src/seclab_taskflows/personalities/pvr_analyst.yaml Adds a dedicated “PVR analyst” personality for evidence-based triage.
src/seclab_taskflows/mcp_servers/pvr_ghsa.py Implements the gh CLI-backed MCP tools for advisory fetch/version resolution/file fetch/report save.
src/seclab_taskflows/configs/model_config_pvr_triage.yaml Adds a taskflow-specific model mapping and temperatures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/seclab_taskflows/mcp_servers/pvr_ghsa.py
Comment thread src/seclab_taskflows/toolboxes/pvr_ghsa.yaml
Comment thread src/seclab_taskflows/toolboxes/pvr_ghsa.yaml Outdated
Comment thread src/seclab_taskflows/taskflows/pvr_triage/pvr_triage.yaml
Comment thread src/seclab_taskflows/personalities/pvr_analyst.yaml
Comment thread src/seclab_taskflows/configs/model_config_pvr_triage.yaml
Copilot AI review requested due to automatic review settings March 3, 2026 16:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (3)

src/seclab_taskflows/mcp_servers/pvr_ghsa.py:454

  • Docstring says matching is done on “header lines” and for both vuln_type and affected_component, but the implementation currently scans the entire file and matches if either substring is present. Please update the docstring to reflect the actual behavior (or adjust the code to match the documented behavior) so taskflow expectations stay aligned.
    """
    Search existing triage reports for similar vulnerability types and affected components.

    Scans REPORT_DIR for *_triage.md files and performs case-insensitive substring
    matching on the header lines for vuln_type and affected_component.
    Returns a JSON list of matching reports with ghsa_id, verdict, quality, and path.
    """
    if not REPORT_DIR.exists():
        return json.dumps([])

    matches = []
    vuln_lower = vuln_type.lower()
    component_lower = affected_component.lower()

    for report_path in sorted(REPORT_DIR.glob("*_triage.md")):
        # Skip batch queue reports and response drafts — only match individual GHSA triage reports
        stem = report_path.stem  # e.g. "GHSA-xxxx-xxxx-xxxx_triage"
        if stem.startswith("batch_queue_") or stem.endswith("_response_triage"):
            continue
        try:
            content = report_path.read_text(encoding="utf-8")
        except OSError:
            continue

        content_lower = content.lower()
        if vuln_lower not in content_lower and component_lower not in content_lower:
            continue

tests/test_pvr_mcp.py:316

  • This import is unused: from seclab_taskflows.mcp_servers.reporter_reputation import get_reporter_history. The test calls self.backend.get_reporter_history(...) directly, so the extra import can be removed (or update the test to exercise the MCP tool wrapper if that was the intent).
        from seclab_taskflows.mcp_servers.reporter_reputation import get_reporter_history
        # Use the MCP tool wrapper to test the string return

src/seclab_taskflows/taskflows/pvr_triage/README.md:33

  • The environment variable table lists REPORTER_DB_DIR as required for pvr_respond, but pvr_respond.yaml doesn’t use the reporter_reputation toolbox or write to the DB. Please update the table to avoid implying an unnecessary requirement.
| `REPORT_DIR` | all | Directory where triage reports are written. Defaults to `./reports` |
| `LOG_DIR` | all | Directory for MCP server logs. Auto-detected via `platformdirs` if not set |
| `REPORTER_DB_DIR` | `pvr_triage`, `pvr_respond` | Directory for the reporter reputation SQLite database. Auto-detected if not set |


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/seclab_taskflows/mcp_servers/pvr_ghsa.py
Comment thread tests/test_pvr_mcp.py Outdated
Comment thread tests/test_pvr_mcp.py Outdated
Comment thread src/seclab_taskflows/mcp_servers/pvr_ghsa.py Outdated
Comment thread src/seclab_taskflows/mcp_servers/reporter_reputation.py Outdated
Comment thread src/seclab_taskflows/mcp_servers/reporter_reputation.py Outdated
Comment thread src/seclab_taskflows/taskflows/pvr_triage/README.md
Comment thread src/seclab_taskflows/mcp_servers/pvr_ghsa.py
Copilot AI review requested due to automatic review settings March 3, 2026 17:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 11 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/seclab_taskflows/mcp_servers/pvr_ghsa.py Outdated
Comment thread src/seclab_taskflows/mcp_servers/reporter_reputation.py Outdated
Comment thread src/seclab_taskflows/taskflows/pvr_triage/pvr_triage_batch.yaml
Comment thread tests/test_pvr_mcp.py
Comment thread src/seclab_taskflows/configs/model_config_pvr_triage.yaml Outdated
Comment thread src/seclab_taskflows/taskflows/pvr_triage/SCORING.md
Comment thread scripts/run_pvr_triage.sh
Comment thread src/seclab_taskflows/taskflows/pvr_triage/pvr_triage.yaml Outdated
Comment thread src/seclab_taskflows/toolboxes/pvr_ghsa.yaml Outdated
Comment thread src/seclab_taskflows/mcp_servers/pvr_ghsa.py Outdated
Copilot AI review requested due to automatic review settings March 3, 2026 18:09
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +18 to +20
GH_TOKEN: "{{ env('GH_TOKEN') }}"
LOG_DIR: "{{ env('LOG_DIR') }}"
REPORTER_DB_DIR: "{{ env('REPORTER_DB_DIR', '') }}"
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The toolbox passes GH_TOKEN into the reporter_reputation MCP server even though the server code never uses it. This unnecessarily exposes a privileged token to a process that only needs local SQLite access; consider removing GH_TOKEN from this toolbox env block (keep only LOG_DIR / REPORTER_DB_DIR).

Suggested change
GH_TOKEN: "{{ env('GH_TOKEN') }}"
LOG_DIR: "{{ env('LOG_DIR') }}"
REPORTER_DB_DIR: "{{ env('REPORTER_DB_DIR', '') }}"
LOG_DIR: "{{ env('LOG_DIR') }}"
REPORTER_DB_DIR: "{{ env('REPORTER_DB_DIR', '') }}"

Copilot uses AI. Check for mistakes.
Copilot AI review requested due to automatic review settings March 3, 2026 19:45
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Reads REPORT_DIR/{ghsa_id}_triage.md and returns its content.
Returns an error string if the file does not exist.
"""
safe_name = "".join(c for c in ghsa_id if c.isalnum() or c in "-_")
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

read_triage_report sanitizes ghsa_id, but if it sanitizes to an empty string it will attempt to read REPORT_DIR/_triage.md. For consistency with save_triage_report / mark_response_sent (which error on empty-after-sanitization) and to avoid surprising reads, add an explicit empty check and return a clear error when safe_name is empty.

Suggested change
safe_name = "".join(c for c in ghsa_id if c.isalnum() or c in "-_")
safe_name = "".join(c for c in ghsa_id if c.isalnum() or c in "-_")
if not safe_name:
return "Error: ghsa_id produced an empty filename after sanitization"

Copilot uses AI. Check for mistakes.
Copilot AI review requested due to automatic review settings March 3, 2026 23:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.


import json
import logging
import os
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os is imported but not used in this module. Please remove the unused import (or use it if needed) to keep the MCP server code lint-clean.

Suggested change
import os

Copilot uses AI. Check for mistakes.
Copilot AI review requested due to automatic review settings March 3, 2026 23:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +68 to +72
Retrieve "pvr_queue" from memcache.

Extract owner and repo from "{{ globals.repo }}" (format: owner/repo).

For each advisory in pvr_queue:
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

list_pvr_advisories returns a JSON string, but this task later treats pvr_queue as an iterable list. To avoid the agent iterating over a raw string, please add an explicit step to parse the JSON returned by list_pvr_advisories (and/or the memcache value) into a list before looping.

Copilot uses AI. Check for mistakes.
Comment on lines +373 to +378
existing_desc = adv_data.get("description", "") or ""
updated_desc = existing_desc + f"\n\n## Maintainer Response\n\n{body}"
_, patch_err = _gh_api(adv_path, method="PATCH", body={"description": updated_desc})
if patch_err:
return f"Error updating advisory description: {patch_err}"
return "Comment appended to advisory description (comments API unavailable)."
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback behavior appends a new "## Maintainer Response" section to the advisory description every time the comments API is unavailable. This can easily lead to duplicated responses on retries or batch runs. Consider making the fallback idempotent by replacing an existing "Maintainer Response" section (if present) or adding a unique marker and updating in place instead of always appending.

Copilot uses AI. Check for mistakes.
Comment thread tests/test_pvr_mcp.py


# ---------------------------------------------------------------------------
# Helpers: patch mcp_data_dir so imports don't fail in CI (no platformdirs dir)
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment says the helper patches mcp_data_dir, but _patch_report_dir() actually patches REPORT_DIR in pvr_ghsa. Please update the comment to match what’s being patched (or rename the helper) to avoid confusion when debugging CI import failures.

Suggested change
# Helpers: patch mcp_data_dir so imports don't fail in CI (no platformdirs dir)
# Helpers: patch pvr_ghsa REPORT_DIR so imports don't fail in CI (no platformdirs dir)

Copilot uses AI. Check for mistakes.
Comment on lines +110 to +114
Call get_reporter_score with that login and store the result as reporter_score.

Call find_similar_triage_reports with:
- vuln_type: pvr_parsed.vuln_type
- affected_component: pvr_parsed.affected_component
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_reporter_score and find_similar_triage_reports both return JSON strings from MCP tools (see their implementations). This prompt uses reporter_score.recommendation and treats similar_reports as a list, which will be unreliable unless the agent explicitly parses the returned JSON first. Please add an explicit step to JSON-parse both tool outputs before accessing fields / storing them in quality_gate.

Copilot uses AI. Check for mistakes.
…cking

- pvr_ghsa.py: reject/withdraw/comment write-back tools, similarity search,
  read_triage_report; _gh_api now accepts a request body for PATCH/POST
- reporter_reputation.py: new MCP server tracking per-reporter triage history
  in SQLite with record/history/score tools
- pvr_triage.yaml: extended to 8 tasks (quality gate, patch status at HEAD,
  CVSS assessment, response draft, reputation update)
- pvr_respond.yaml: write-back taskflow (action=reject|comment|withdraw)
- pvr_triage_batch.yaml: inbox scoring taskflow with ranked markdown output
- reporter_reputation.yaml, pvr_ghsa.yaml: new toolbox + confirm gates
- tests/test_pvr_mcp.py: 23 unit tests, all passing
- README.md: usage docs for all three taskflows
- Add from __future__ import annotations for Python 3.9 compat (dict|None, Path|str)
- Fix REPORT_DIR empty-string handling: treat empty env var as unset
- Add pagination to list_pvr_advisories; return JSON list consistently (empty list instead of string on no results)
- Guard find_similar_triage_reports against empty/whitespace inputs; update docstring to reflect full-file scan
- ReporterReputationBackend: use explicit "sqlite://" sentinel for in-memory; mkdir for filesystem paths instead of silent fallback
- get_reporter_history MCP tool: return JSON list consistently (empty list instead of "No history" string)
- pvr_ghsa.yaml: add default value for REPORT_DIR env var
- pvr_triage_batch.yaml: remove dead already_triaged_penalty from scoring formula (entries are filtered out before scoring; aligns with SCORING.md)
- Tests: remove unused sys/get_reporter_history imports; switch tempfile to TemporaryDirectory with tearDown cleanup; fix setUp to use sqlite:// sentinel; rename _patch_mcp_data_dir_pvr_ghsa -> _patch_report_dir
- Docs: align AI_API_ENDPOINT default to https://api.githubcopilot.com across README, pvr_triage.yaml, model_config; remove pvr_respond from REPORTER_DB_DIR required-by list
pvr_ghsa.py:
- Remove backwards fallback quality regex in find_similar_triage_reports
  (r"\b(High|Medium|Low)\b.*quality" matched wrong direction; primary regex sufficient)
- save_triage_report: return error on empty safe_name after sanitization
- fetch_file_at_ref: cap length at 500 lines; return error if start_line exceeds file length
- list_pvr_advisories: add max_pages=50 guard on pagination loop

reporter_reputation.py:
- Add UniqueConstraint("login","ghsa_id") to ReporterRecord table
- Add VALID_VERDICTS / VALID_QUALITIES constants; validate inputs in record_triage_result
- MCP record_triage_result tool: surface ValueError as error string to agent

pvr_triage.yaml:
- Task 5: also store triage_outcome {verdict, quality} in memcache after report generation
- Tasks 7+8: use triage_outcome.verdict/quality directly instead of re-parsing report text

pvr_respond.yaml, pvr_triage_batch.yaml:
- Fix AI_API_ENDPOINT comment to show actual default URL

tests:
- Add test_save_triage_report_empty_after_sanitization
- Add test_record_invalid_verdict_raises, test_record_invalid_quality_raises
PT009/PT027: test_pvr_mcp.py uses unittest.TestCase style assertions
throughout. Converting 71 assertions is not warranted; suppress via
per-file-ignores for tests/*.

PLC0415: imports inside setUp/test methods are deliberate — needed for
the patch.object pattern that avoids early import side-effects.

SIM105: pvr_ghsa.py:315 try/except assigns to a variable on success;
contextlib.suppress cannot capture that assignment. False positive;
suppress globally.

Also carry forward PLW0603 (global statement for module-level state),
S101 (assert in tests), and SLF001 (private member access in tests)
suppressions from the shell-toolbox branch.
GitHub API PATCH only accepts published/closed/draft. rejected and withdrawn
are not valid PATCH states. Remove withdraw_pvr_advisory (withdrawn is
read-only, reporter-initiated) and fix reject_pvr_advisory to use closed.
Accepting a PVR (confirmed vulnerability) moves the advisory from triage
to draft state. New tool accept_pvr_advisory PATCHes state=draft and posts
a comment. Added as action=accept throughout respond/respond_batch taskflows,
runner script, and docs.
The /repos/.../security-advisories/{ghsa_id}/comments endpoint does not
exist in the public API (404). Remove _post_advisory_comment and
add_pvr_advisory_comment entirely. Strip the comment parameter from
accept_pvr_advisory and reject_pvr_advisory — both now only apply the
state transition (triage→draft and triage→closed respectively).

Remove the comment action from pvr_respond and pvr_respond_batch; valid
actions are now accept and reject only.

Add MANUAL_RESPONSE.md with steps for posting the generated response
draft manually via the advisory URL after the state transition is applied.
Deduplication:
- Add _fingerprint_advisory and _compare_fingerprints to pvr_ghsa.py
  for structural comparison of advisories by CWE, package, version
  range, file paths, and normalized summary
- Add compare_advisories MCP tool: fetches inbox, fingerprints all
  advisories, clusters duplicates via union-find, returns match levels
  (strong/moderate/weak/none) with reasons
- Wire into pvr_triage_batch: batch scorer now runs compare_advisories,
  adds Duplicates column to queue output, flags clusters with
  'Likely Duplicate -- Triage Best' action
- Wire into pvr_triage quality gate (Task 3): checks for duplicates in
  inbox, surfaces in report but never auto-closes

Container validation (optional, Task 4b):
- New optional task gated by PVR_CONTAINER_VALIDATION=true
- Clones repo at affected version into seclab-shell-sast container
- Runs semgrep SAST scan on reported files
- Traces call graph for reachability analysis (pyan3/cscope/rg)
- Attempts best-effort PoC reproduction (safe commands only)
- Diffs affected version vs HEAD for patch analysis
- Results appear in new Validation Results report section
- Reachability findings factor into severity assessment

Tests:
- 12 new tests for fingerprinting, comparison, and dedup detection
- Fix YAML structure tests for upstream AvailableTools API change
  (dict -> Pydantic model attributes)

Docs:
- SCORING.md: add Sections 5 (Duplicate Detection) and 6 (Container
  Validation) with match level tables and prerequisites
- README.md: update task list (8->9), add Duplicate Detection and
  Container Validation sections, add new env vars
Structural compare_advisories is a cheap first pass, but returns 'none'
when advisories lack structured metadata (no CWE, no package, no file
paths). Low-quality and AI-generated reports often have exactly this
problem.

Changes:
- Batch scorer: after structural comparison, agent now reads all
  advisory descriptions and identifies semantic duplicates (same code
  path, attack scenario, or root cause) regardless of metadata overlap.
  Semantic groups flagged with match_level='semantic'.
- Single-advisory quality gate: agent reads other triage-state advisory
  summaries and applies semantic judgment alongside structural results.
  Both structural and semantic duplicates surfaced in report.
- _compare_fingerprints: add note field clarifying that 'none' means
  insufficient metadata, not necessarily distinct vulnerabilities.
- Duplicate detection is always additive, never exclusionary.
Add fetch_security_policy tool to pvr_ghsa.py. Checks SECURITY.md at
the standard GitHub locations (/, /.github/, /docs/) and the org-level
.github repo as fallback.

Wire into pvr_triage:
- Task 2: fetch security policy alongside the advisory
- Task 3 (quality gate): evaluate report against policy for scope
  compliance, required elements, version coverage, explicit exclusions,
  and preferred reporting channel
- Task 5 (report): new Security Policy Compliance section (only when
  a policy exists) summarizing whether the report is compliant,
  partially compliant, or non-compliant

This helps maintainers quickly assess whether a PVR report meets their
stated requirements before spending time on code verification.
Moderate matches (same package, different CWE/version/files) were being
union-find merged into clusters, causing unrelated advisories targeting
the same repo to appear as duplicates. Now only strong matches (package
AND cwe/version/files overlap) cluster. Moderate and weak matches are
still reported as informational signals for the agent's semantic
analysis layer.

Found via live testing against anticomputer/vulnerable-test-app: an AI
slop report (CWE-94, version <= 99.0.0) was incorrectly clustered with
two legitimate SQL injection reports (CWE-89, version <= 0.0.1) because
all three shared the same Go package.
Both pvr_triage and pvr_triage_batch now accept a 'state' global
(default: triage). Pass state=draft when testing with owner-created
advisories, which land in draft state rather than triage.

Add scripts/demo_pvr_triage.sh for live testing against
anticomputer/vulnerable-test-app. Subcommands:
  tools  - test MCP tools against live API (no AI calls)
  batch  - run batch scoring taskflow
  triage - run full single-advisory triage
  all    - everything in sequence

Uses gh auth token for GitHub API and passage for AI endpoint token.
Copilot AI review requested due to automatic review settings April 15, 2026 19:23
@anticomputer anticomputer force-pushed the anticomputer/pvr-triage branch from c1b5dba to 631ecc6 Compare April 15, 2026 19:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “PVR triage” workflow to help maintainers process GitHub Security Advisories created via Private Vulnerability Reporting (PVR): fetch + parse advisories, dedup/quality-gate, verify against code at affected versions, generate a triage report + response draft, and track reporter reputation.

Changes:

  • Introduces new MCP servers for GHSA/PVR operations and reporter reputation tracking (SQLite-backed).
  • Adds four taskflows covering single-advisory triage, inbox batch scoring, and single/batch state-transition response flows.
  • Adds supporting toolboxes, documentation, scripts, and unit tests; updates Ruff configuration for the new tests.
Show a summary per file
File Description
tests/test_pvr_mcp.py Unit tests for the new MCP tools and YAML/toolbox wiring.
src/seclab_taskflows/toolboxes/reporter_reputation.yaml Toolbox definition for the reporter reputation MCP server.
src/seclab_taskflows/toolboxes/pvr_ghsa.yaml Toolbox definition for PVR GHSA MCP server + confirm-gating for write actions.
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage_batch.yaml Batch taskflow to score/sort triage inbox and write a ranked queue report.
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage.yaml Main single-advisory triage taskflow including quality gate, verification, and report drafting.
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond_batch.yaml Bulk state-transition taskflow for advisories with pending response drafts.
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond.yaml Single-advisory state-transition taskflow gated by confirm.
src/seclab_taskflows/taskflows/pvr_triage/SCORING.md Authoritative scoring + fast-close + dedup reference for maintainers.
src/seclab_taskflows/taskflows/pvr_triage/README.md Usage and operational docs for the four taskflows.
src/seclab_taskflows/taskflows/pvr_triage/MANUAL_RESPONSE.md Instructions for manually posting reporter replies via the GH UI.
src/seclab_taskflows/personalities/pvr_analyst.yaml Adds a dedicated “pvr_analyst” personality for evidence-based triage output.
src/seclab_taskflows/mcp_servers/reporter_reputation.py New SQLite-backed reporter reputation backend + MCP tools.
src/seclab_taskflows/mcp_servers/pvr_ghsa.py New MCP server for advisory fetch/list/dedup, version-to-ref, file fetch, report IO, and write-back state transitions.
src/seclab_taskflows/configs/model_config_pvr_triage.yaml Model config for PVR flows (triage vs extraction roles).
scripts/run_pvr_triage.sh Local runner script for the taskflows (batch/triage/respond/respond_batch/demo).
scripts/demo_pvr_triage.sh Live demo script against a test repo to exercise the pipeline.
pyproject.toml Ruff ignore/per-file-ignores tweaks to accommodate new tests.
docs/pvr_triage_overview.md High-level overview/diagram of the PVR triage workflow for a quick sync/reference.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 18/18 changed files
  • Comments generated: 5

Comment thread scripts/run_pvr_triage.sh
Comment on lines +10 to +11
# ./scripts/run_pvr_triage.sh respond <owner/repo> <GHSA-xxxx-xxxx-xxxx> <accept|comment|reject>
# ./scripts/run_pvr_triage.sh respond_batch <owner/repo> <accept|comment|reject>
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script usage text advertises an action=comment mode for respond/respond_batch, but the implementation only accepts accept|reject (and the taskflows/toolbox also only implement accept/reject). Either remove comment from the usage/docs here, or implement the missing comment action end-to-end so the script matches actual capabilities.

Suggested change
# ./scripts/run_pvr_triage.sh respond <owner/repo> <GHSA-xxxx-xxxx-xxxx> <accept|comment|reject>
# ./scripts/run_pvr_triage.sh respond_batch <owner/repo> <accept|comment|reject>
# ./scripts/run_pvr_triage.sh respond <owner/repo> <GHSA-xxxx-xxxx-xxxx> <accept|reject>
# ./scripts/run_pvr_triage.sh respond_batch <owner/repo> <accept|reject>

Copilot uses AI. Check for mistakes.
Comment thread tests/test_pvr_mcp.py
Comment on lines +570 to +573
# Should still find the cluster
self.assertTrue(len(result["clusters"]) >= 1 or len(result.get("weak_matches", [])) >= 0)


Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion is effectively a no-op: len(result.get("weak_matches", [])) >= 0 is always true, so the test will pass even if compare_advisories returns no matches for target_ghsa. Tighten the assertion to require at least one relevant match (e.g., a cluster containing the target GHSA, or weak_matches length > 0 with a/b involving the target).

Suggested change
# Should still find the cluster
self.assertTrue(len(result["clusters"]) >= 1 or len(result.get("weak_matches", [])) >= 0)
target_ghsa = "GHSA-aaaa-1111-aaaa"
clusters = result.get("clusters", [])
weak_matches = result.get("weak_matches", [])
cluster_contains_target = any(
target_ghsa in cluster
for cluster in clusters
)
weak_match_contains_target = any(
match.get("a") == target_ghsa or match.get("b") == target_ghsa
for match in weak_matches
)
# Should still find at least one relevant match for the requested GHSA.
self.assertTrue(cluster_contains_target or weak_match_contains_target)

Copilot uses AI. Check for mistakes.
Comment on lines +764 to +765
# Extract ghsa_id: remove "_response_triage" suffix
ghsa_id = stem.replace("_response_triage", "")
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

list_pending_responses derives ghsa_id via stem.replace("_response_triage", ""), which will replace occurrences anywhere in the stem (not just a suffix). Using a strict suffix removal (e.g., removesuffix) avoids accidental mangling if the GHSA id ever contains that substring.

Suggested change
# Extract ghsa_id: remove "_response_triage" suffix
ghsa_id = stem.replace("_response_triage", "")
# Extract ghsa_id: remove only the trailing "_response_triage" suffix
ghsa_id = stem.removesuffix("_response_triage")

Copilot uses AI. Check for mistakes.
"match_level": result["match_level"],
"reasons": result["reasons"],
})
if result["match_level"] == "strong":
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compare_advisories only unions advisories into clusters when match_level == "strong", but the matcher can return moderate (e.g., same package only) and the docs/taskflows treat strong+moderate as duplicate clusters. This means moderate duplicates will incorrectly appear as singles (only showing up in weak_matches/pair list), which will undermine dedup surfacing in batch + quality-gate. Consider unioning for both strong and moderate (and keeping weak informational-only).

Suggested change
if result["match_level"] == "strong":
if result["match_level"] in {"strong", "moderate"}:

Copilot uses AI. Check for mistakes.
for m in _FILE_PATH_RE.finditer(text):
p = m.group(1)
ext = p.rsplit(".", 1)[-1].lower() if "." in p else ""
if ext in _SRC_EXTS and "/" in p:
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_extract_file_paths' regex allows Windows-style \ separators, but the filter only keeps matches containing /. This drops valid paths like src\\foo\\bar.py that might appear in reports copied from Windows environments. Consider accepting either / or \\ as a separator when deciding whether a match is a path rather than a bare filename.

Suggested change
if ext in _SRC_EXTS and "/" in p:
if ext in _SRC_EXTS and ("/" in p or "\\" in p):

Copilot uses AI. Check for mistakes.
Add explicit execution directives to personality and all task prompts:
- 'Execute all steps above, then stop. Do not ask what to do next.'
- Personality: 'Do not offer to do additional work. Do not say if
  you'd like or shall I. Complete every step, then stop.'

Prevents the model from pausing mid-task to ask for user direction
instead of executing the defined workflow autonomously.
FastMCP 3.x changed @mcp.tool() to return the bare function instead
of a wrapper with a .fn attribute. Update all test calls to invoke
the tool functions directly.
Copilot AI review requested due to automatic review settings April 15, 2026 19:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new end-to-end “PVR triage” workflow to the seclab taskflow ecosystem, including MCP servers for GHSA/PVR operations and reporter reputation tracking, plus taskflows/scripts/docs to run single-advisory triage, batch inbox scoring, and write-back state transitions.

Changes:

  • Introduces MCP servers for PVR GHSA operations (pvr_ghsa) and reporter reputation scoring/storage (reporter_reputation).
  • Adds taskflows for single-advisory triage, batch scoring, and (bulk) respond flows, with supporting toolboxes and personalities.
  • Adds documentation, a model config, scripts for local runs/demos, and a comprehensive unit test suite.
Show a summary per file
File Description
tests/test_pvr_mcp.py Unit tests covering the new MCP tools and YAML structure.
src/seclab_taskflows/toolboxes/reporter_reputation.yaml Toolbox definition for the reporter reputation MCP server.
src/seclab_taskflows/toolboxes/pvr_ghsa.yaml Toolbox definition for the PVR GHSA MCP server, with confirm-gated write-back tools.
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage_batch.yaml Batch inbox scoring taskflow (priority scoring + dedup + Age column output).
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage.yaml Single-advisory triage taskflow (quality gate, verification, reporting, response drafting, reputation update).
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond_batch.yaml Bulk state-transition taskflow for all pending response drafts.
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond.yaml Single-advisory state-transition taskflow using saved outputs.
src/seclab_taskflows/taskflows/pvr_triage/SCORING.md Authoritative scoring/fast-close/dedup reference for the new flows.
src/seclab_taskflows/taskflows/pvr_triage/README.md Usage documentation for all PVR triage taskflows and expected outputs.
src/seclab_taskflows/taskflows/pvr_triage/MANUAL_RESPONSE.md Documents manual posting of responses due to lack of advisory comment API.
src/seclab_taskflows/personalities/pvr_analyst.yaml Adds a dedicated “PVR analyst” personality with evidence-based guidance.
src/seclab_taskflows/mcp_servers/reporter_reputation.py SQLite-backed reporter triage history + reputation scoring MCP server.
src/seclab_taskflows/mcp_servers/pvr_ghsa.py GHSA/PVR MCP server: fetch/list, version-to-ref, file fetch, report IO, dedup utilities, write-back transitions.
src/seclab_taskflows/configs/model_config_pvr_triage.yaml Defines the model roles used by the PVR triage taskflows.
scripts/run_pvr_triage.sh Local runner script for batch/triage/respond workflows.
scripts/demo_pvr_triage.sh Live demo script exercising MCP tools and taskflows against a test repo.
pyproject.toml Updates ruff ignores/per-file-ignores to accommodate new tests patterns.
docs/pvr_triage_overview.md High-level overview doc for the new PVR triage system.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 18/18 changed files
  • Comments generated: 4

Comment thread tests/test_pvr_mcp.py
Comment on lines +53 to +59
with patch.object(self.pvr, "_gh_api", side_effect=fake_gh_api):
result = self.pvr.accept_pvr_advisory(
owner="owner",
repo="repo",
ghsa_id="GHSA-1234-5678-abcd",
)

Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests call FastMCP tool functions directly (e.g., accept_pvr_advisory(...)), but elsewhere in the repo MCP tools are invoked via the .fn attribute (e.g., shell_exec.fn(...)). If @mcp.tool() wraps functions into tool objects, direct calls may bypass expected behavior or break as FastMCP evolves. Update this file to consistently call MCP tools via .fn (and patch/mock against that invocation style).

Copilot uses AI. Check for mistakes.
Comment thread scripts/run_pvr_triage.sh
Comment on lines +8 to +12
# ./scripts/run_pvr_triage.sh batch <owner/repo>
# ./scripts/run_pvr_triage.sh triage <owner/repo> <GHSA-xxxx-xxxx-xxxx>
# ./scripts/run_pvr_triage.sh respond <owner/repo> <GHSA-xxxx-xxxx-xxxx> <accept|comment|reject>
# ./scripts/run_pvr_triage.sh respond_batch <owner/repo> <accept|comment|reject>
# ./scripts/run_pvr_triage.sh demo <owner/repo>
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usage text and argument examples include an action=comment option, but this script enforces accept|reject only (and the taskflows/toolbox don’t implement a comment action). This mismatch will confuse users and break documented commands. Either add support for comment end-to-end (script + taskflow + MCP tool behavior), or remove comment from the usage/docs/examples here and elsewhere.

Copilot uses AI. Check for mistakes.

| Role | Used for | Default model |
|---|---|---|
| `triage` | Code verification and report generation | `claude-opus-4.6-1m` |
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README claims the default triage model is claude-opus-4.6-1m, but model_config_pvr_triage.yaml sets triage: claude-opus-4.6. Please align the docs with the actual configured model name (or update the config) so users know what will run by default.

Suggested change
| `triage` | Code verification and report generation | `claude-opus-4.6-1m` |
| `triage` | Code verification and report generation | `claude-opus-4.6` |

Copilot uses AI. Check for mistakes.
Comment on lines +201 to +214
Strong and moderate matches are clustered via union-find. The batch queue output shows each cluster with its member GHSAs and match reasons.

### Effect on triage

- Batch scorer: strong-match clusters get "Likely Duplicate -- Triage Best" suggested action
- Single-advisory triage: quality gate surfaces duplicate info but does NOT auto-close. Maintainers decide.
- Triage report: Duplicate/Prior Reports section prominently flags cluster membership

### Conservative design

Dedup detection is intentionally conservative:
- Only structural field overlap, no semantic similarity
- Never auto-closes advisories based on dedup alone
- Weak matches are surfaced as informational, not clustered
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SCORING.md states that strong and moderate matches are clustered via union-find, but the current compare_advisories implementation only clusters strong matches. Update this section to match the implemented behavior, or change the code to also cluster moderate matches (and ensure the output reflects them).

Suggested change
Strong and moderate matches are clustered via union-find. The batch queue output shows each cluster with its member GHSAs and match reasons.
### Effect on triage
- Batch scorer: strong-match clusters get "Likely Duplicate -- Triage Best" suggested action
- Single-advisory triage: quality gate surfaces duplicate info but does NOT auto-close. Maintainers decide.
- Triage report: Duplicate/Prior Reports section prominently flags cluster membership
### Conservative design
Dedup detection is intentionally conservative:
- Only structural field overlap, no semantic similarity
- Never auto-closes advisories based on dedup alone
- Weak matches are surfaced as informational, not clustered
Only strong matches are clustered via union-find. Moderate matches are surfaced as duplicate signals and match reasons, but they are not merged into union-find clusters. The batch queue output shows each strong-match cluster with its member GHSAs and match reasons.
### Effect on triage
- Batch scorer: strong-match clusters get "Likely Duplicate -- Triage Best" suggested action
- Single-advisory triage: quality gate surfaces duplicate info, including moderate-match signals, but does NOT auto-close. Maintainers decide.
- Triage report: Duplicate/Prior Reports section prominently flags strong-cluster membership and may include non-clustered duplicate signals
### Conservative design
Dedup detection is intentionally conservative:
- Only structural field overlap, no semantic similarity
- Never auto-closes advisories based on dedup alone
- Moderate and weak matches are surfaced as informational signals unless they qualify as strong matches; only strong matches are clustered

Copilot uses AI. Check for mistakes.
- Document the 'state' global (default: triage, use draft for testing)
  with examples in both taskflow command blocks
- Add semantic duplicate analysis to Duplicate Detection section
- Fix model name: claude-opus-4.6 (not 1m variant)
- Add globals reference table
- Add demo script section
- Clarify that structural 'none' means insufficient metadata, not
  distinct vulnerabilities
Copilot AI review requested due to automatic review settings April 15, 2026 20:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new PVR (Private Vulnerability Report) triage workflow to the seclab taskflow ecosystem, including MCP tooling to fetch/compare advisories, persist reporter history, and generate/save triage + response artifacts for maintainers.

Changes:

  • Introduces pvr_ghsa MCP server + toolbox for GHSA triage-state fetching, deduping, report persistence, and write-back state transitions (accept/reject).
  • Adds reporter_reputation MCP server + toolbox backed by SQLite to record triage outcomes and compute trust recommendations used by the quality gate.
  • Adds four new PVR taskflows (single triage, batch scoring, single respond, bulk respond) plus documentation, scripts, and unit tests.
Show a summary per file
File Description
tests/test_pvr_mcp.py Unit tests for new MCP tools and YAML/toolbox structure.
src/seclab_taskflows/toolboxes/reporter_reputation.yaml Toolbox definition for the reporter reputation MCP server.
src/seclab_taskflows/toolboxes/pvr_ghsa.yaml Toolbox definition for PVR GHSA MCP server + confirm gating for write-back tools.
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage_batch.yaml Batch inbox scoring taskflow that ranks advisories and saves a queue report.
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage.yaml Main end-to-end single-advisory triage taskflow (quality gate, verification, report, response, reputation update).
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond_batch.yaml Bulk state-transition taskflow for pending response drafts.
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond.yaml Single-advisory state-transition taskflow using saved drafts/reports.
src/seclab_taskflows/taskflows/pvr_triage/SCORING.md Authoritative reference for scoring, fast-close logic, reputation thresholds, and dedup semantics.
src/seclab_taskflows/taskflows/pvr_triage/README.md Full user-facing documentation for running the four taskflows.
src/seclab_taskflows/taskflows/pvr_triage/MANUAL_RESPONSE.md Documents manual posting requirement (no advisory comments API).
src/seclab_taskflows/personalities/pvr_analyst.yaml New personality to enforce evidence-based vulnerability triage behavior.
src/seclab_taskflows/mcp_servers/reporter_reputation.py SQLite-backed MCP server to record triage outcomes and compute reporter reputation.
src/seclab_taskflows/mcp_servers/pvr_ghsa.py MCP server for listing/fetching advisories, dedup comparison, report IO, and accept/reject transitions.
src/seclab_taskflows/configs/model_config_pvr_triage.yaml Model config defining extraction vs triage model roles for PVR flows.
scripts/run_pvr_triage.sh Local helper script to run the PVR flows end-to-end.
scripts/demo_pvr_triage.sh Live demo script to exercise tools/taskflows against a test repository.
pyproject.toml Ruff ignore/overrides to accommodate new test patterns.
docs/pvr_triage_overview.md High-level overview doc for internal/quick reference on the new workflow.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 18/18 changed files
  • Comments generated: 1

Comment on lines +493 to +498
return json.dumps({
"clusters": clusters,
"weak_matches": weak_matches,
"singles": sorted(singles),
"total": len(fps),
}, indent=2)
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compare_advisories() documents that when target_ghsa is set, only comparisons involving that advisory are returned, but the current output still includes "singles" for all advisories (including those unrelated to target_ghsa). This makes the response ambiguous for the caller and can cause the taskflow to treat unrelated advisories as part of the comparison result. Consider filtering singles/clusters down to only entries that include target_ghsa (or adjust the docstring/output contract).

Copilot uses AI. Check for mistakes.
Copilot AI review requested due to automatic review settings April 15, 2026 20:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “PVR triage” workflow to the seclab_taskflows ecosystem, including MCP servers and taskflows to fetch/score/triage PVR-submitted GHSAs, generate structured reports + response drafts, and persist reporter reputation in SQLite.

Changes:

  • Introduces pvr_ghsa MCP server for advisory fetch/list, report IO, response markers, version→ref resolution, and structural dedup detection.
  • Introduces reporter_reputation MCP server + toolbox for recording triage outcomes and computing reputation scores.
  • Adds four new PVR taskflows (single triage, batch scoring, single respond, bulk respond), plus docs, scripts, and unit tests.
Show a summary per file
File Description
tests/test_pvr_mcp.py Unit tests for new MCP tools and YAML/toolbox loading.
src/seclab_taskflows/toolboxes/reporter_reputation.yaml Toolbox wiring for reporter reputation MCP server.
src/seclab_taskflows/toolboxes/pvr_ghsa.yaml Toolbox wiring + confirm-gating for write-back tools.
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage_batch.yaml Batch inbox scoring + dedup surfacing + report generation instructions.
src/seclab_taskflows/taskflows/pvr_triage/pvr_triage.yaml End-to-end single-advisory triage flow (quality gate, verification, report, response draft, reputation update).
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond_batch.yaml Bulk state-transition runner over pending response drafts.
src/seclab_taskflows/taskflows/pvr_triage/pvr_respond.yaml Single-advisory write-back runner (accept/reject) with sent-marker creation.
src/seclab_taskflows/taskflows/pvr_triage/SCORING.md Authoritative scoring/fast-close/dedup reference for the new flows.
src/seclab_taskflows/taskflows/pvr_triage/README.md Usage and operational documentation for the PVR taskflows.
src/seclab_taskflows/taskflows/pvr_triage/MANUAL_RESPONSE.md Instructions for manually posting response drafts (no advisory comments API).
src/seclab_taskflows/personalities/pvr_analyst.yaml New analyst personality focused on evidence-based PVR triage.
src/seclab_taskflows/mcp_servers/reporter_reputation.py SQLite-backed reporter history + score computation exposed as MCP tools.
src/seclab_taskflows/mcp_servers/pvr_ghsa.py GHSA/PVR MCP server: API calls via gh, report IO, response markers, dedup compare, version/tag resolution, file-at-ref fetch.
src/seclab_taskflows/configs/model_config_pvr_triage.yaml Model-role configuration for extraction vs triage tasks.
scripts/run_pvr_triage.sh Local runner for batch/triage/respond/bulk-respond flows.
scripts/demo_pvr_triage.sh Live demo driver against a test repo; exercises MCP tools + taskflows.
pyproject.toml Ruff ignore additions and test per-file ignores to support new unittest-style tests.
docs/pvr_triage_overview.md High-level overview/cheatsheet for the new taskflows.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

src/seclab_taskflows/mcp_servers/pvr_ghsa.py:498

  • When target_ghsa is provided, the docstring says only comparisons involving that advisory are returned. However, the function still returns singles for every other advisory (because clustering is computed over all fingerprints regardless of target filtering). This makes the output noisy and violates the documented behavior; consider filtering the output so clusters/weak_matches/singles only mention target_ghsa and its matches.
    return json.dumps({
        "clusters": clusters,
        "weak_matches": weak_matches,
        "singles": sorted(singles),
        "total": len(fps),
    }, indent=2)

src/seclab_taskflows/mcp_servers/pvr_ghsa.py:458

  • compare_advisories() only unions (clusters) "strong" matches, but still computes best_level as if clusters could include moderate matches. As implemented, moderate matches are neither clustered nor surfaced separately (only weak_matches are returned), which can hide meaningful duplicates and contradicts the stated behavior in docs/tests. Consider either unioning on both strong+moderate, or returning a separate moderate_matches list (and documenting that only strong matches are clustered).
            result = _compare_fingerprints(fps[i], fps[j])
            if result["match_level"] != "none":
                matches.append({
                    "a": fps[i]["ghsa_id"],
                    "b": fps[j]["ghsa_id"],
                    "match_level": result["match_level"],
                    "reasons": result["reasons"],
                })
                if result["match_level"] == "strong":
                    union(i, j)

  • Files reviewed: 18/18 changed files
  • Comments generated: 6

data, err = _gh_api(f"{base_path}&page={page}")
if err:
return f"Error listing advisories: {err}"
if not isinstance(data, list) or not data:
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In compare_advisories(), if the GitHub API returns a non-list JSON payload (unexpected schema), the code currently breaks out of pagination silently (if not isinstance(data, list) or not data: break). This can mask server-side/API changes and produce incomplete/empty results without an error. Return an explicit error when data is not a list, similar to list_pvr_advisories().

This issue also appears in the following locations of the same file:

  • line 448
  • line 493
Suggested change
if not isinstance(data, list) or not data:
if not isinstance(data, list):
return f"Error listing advisories: unexpected response schema (expected list, got {type(data).__name__})"
if not data:

Copilot uses AI. Check for mistakes.
Comment on lines +129 to +135
### Output

Saved to `REPORT_DIR/batch_queue_<repo>_<date>.md`:

```markdown
# PVR Batch Triage Queue: owner/repo

Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation says the batch queue report is saved as batch_queue_<repo>_<date>.md, but the implementation writes reports via save_triage_report, which always appends _triage.md. With the current taskflow, the actual filename will be batch_queue_<repo>_<date>_triage.md. Update the docs (or change the writer) so users can find the generated file reliably.

Copilot uses AI. Check for mistakes.
| `GHSA-xxxx_triage.md` | pvr_triage | Full analysis report |
| `GHSA-xxxx_response_triage.md` | pvr_triage | Draft reply to reporter |
| `GHSA-xxxx_response_sent.md` | pvr_respond / batch | State-transition applied marker (idempotent) |
| `batch_queue_<repo>_<date>.md` | pvr_triage_batch | Ranked inbox table |
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overview lists the batch queue output as batch_queue_<repo>_<date>.md, but pvr_triage_batch uses save_triage_report, which appends _triage.md. Unless the writer is changed, the actual output will be batch_queue_<repo>_<date>_triage.md. Align the overview with the real filename to avoid confusion.

Suggested change
| `batch_queue_<repo>_<date>.md` | pvr_triage_batch | Ranked inbox table |
| `batch_queue_<repo>_<date>_triage.md` | pvr_triage_batch | Ranked inbox table |

Copilot uses AI. Check for mistakes.
Comment on lines +199 to +203
### Clustering

Strong and moderate matches are clustered via union-find. The batch queue output shows each cluster with its member GHSAs and match reasons.

### Effect on triage
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SCORING.md describes duplicate clustering as: "Strong and moderate matches are clustered via union-find", but the compare_advisories() implementation currently unions only "strong" matches. Either update the doc to match the current behavior or change the clustering logic to include moderate matches as documented.

Copilot uses AI. Check for mistakes.
Comment thread scripts/run_pvr_triage.sh
Comment on lines +8 to +12
# ./scripts/run_pvr_triage.sh batch <owner/repo>
# ./scripts/run_pvr_triage.sh triage <owner/repo> <GHSA-xxxx-xxxx-xxxx>
# ./scripts/run_pvr_triage.sh respond <owner/repo> <GHSA-xxxx-xxxx-xxxx> <accept|comment|reject>
# ./scripts/run_pvr_triage.sh respond_batch <owner/repo> <accept|comment|reject>
# ./scripts/run_pvr_triage.sh demo <owner/repo>
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script usage header advertises comment as a valid action for respond / respond_batch, but the implementation only accepts accept|reject (and errors otherwise). Either implement the comment action end-to-end or update these usage lines to avoid misleading users.

Copilot uses AI. Check for mistakes.
Comment thread tests/test_pvr_mcp.py
self.assertIsNotNone(result)
confirm = result.confirm or []
self.assertIn("accept_pvr_advisory", confirm)
self.assertIn("reject_pvr_advisory", confirm)
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test currently asserts that add_pvr_advisory_comment is not in the toolbox confirm list, but it never asserts that a comment tool actually exists. As a result, the test will pass even if comment write-back support is missing entirely. If comment is intended to be supported, add an assertion that the tool is present (and then assert whether it is confirm-gated or not).

Suggested change
self.assertIn("reject_pvr_advisory", confirm)
self.assertIn("reject_pvr_advisory", confirm)
tool_names = set()
toolbox_tools = getattr(result, "tools", None)
if isinstance(toolbox_tools, dict):
tool_names.update(toolbox_tools.keys())
elif isinstance(toolbox_tools, list):
for tool in toolbox_tools:
if isinstance(tool, str):
tool_names.add(tool)
else:
name = getattr(tool, "name", None)
if name:
tool_names.add(name)
nested_tool = getattr(tool, "tool", None)
nested_name = getattr(nested_tool, "name", None)
if nested_name:
tool_names.add(nested_name)
self.assertIn(
"add_pvr_advisory_comment",
tool_names,
"pvr_ghsa toolbox must expose the add_pvr_advisory_comment tool",
)

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants