Skip to content

fix(agents): fix plugin compatibility and robustness for coding-standards code review agents#1289

Draft
erikschlegel wants to merge 5 commits intomicrosoft:mainfrom
erikschlegel:fix/code-review-file-refs
Draft

fix(agents): fix plugin compatibility and robustness for coding-standards code review agents#1289
erikschlegel wants to merge 5 commits intomicrosoft:mainfrom
erikschlegel:fix/code-review-file-refs

Conversation

@erikschlegel
Copy link
Copy Markdown
Contributor

@erikschlegel erikschlegel commented Apr 3, 2026

Related Issue(s)

Fixes #1286

Description

These changes were discovered while testing the Code Review Standards and Code Review Full agents from the Coding Standards Plugin installation in VS Code. The plugin install surface exposed three categories of silent failure invisible when running the agents directly from the plugin.

Three commits fix path resolution failures, reference correctness, and agent robustness.

Commit 1 — correct review-artifacts path in Step 4

The #file: reference in code-review-standards.agent.md Step 4 pointed to ../../instructions/code-review/review-artifacts.instructions.md, missing the coding-standards/ path segment. Corrected to ../../instructions/coding-standards/code-review/review-artifacts.instructions.md.

Commit 2 — fix plugin compatibility for coding-standards skill resolution

The plugin generator strips the .github/ prefix from all artifact paths. An agent at .github/agents/coding-standards/ (3 levels deep in the repo) becomes agents/coding-standards/ (2 levels deep in the plugin), breaking any reference that assumed .github/ depth.

docs/templates/ relative links escape the plugin root. #file:../../../docs/templates/ worked in the repo but escaped the plugin root entirely. Converted to workspace-root prose (at `docs/templates/file.md`), consistent with adr-creation, brd-builder, and system-architecture-reviewer.

Skill loading failed entirely in plugin context.
Steps 3b and 3c assumed .github/skills/ prefix. Skills live under skills/ in the plugin. Both steps now use a dual path fallback: try .github/skills/... first, fall back to skills/....

pr-reference skill missing from the coding-standards collection.
The diff computation protocol relies on scripts in pr-reference, absent from coding-standards.collection.yml. Added so diff computation works for coding-standards-only plugin users.

Commit 3 — improve robustness of code review agents

code-review-full.agent.md:

  • Documented the disable-model-invocation / Step 4 interaction: manual invocation retains full model reasoning; added verbatim-output fallback with ⚠️ warning when transformation rules cannot be applied
  • Replaced vague "instruct it to skip" with prescribed skip phrases in Steps 2 and 3 so subagent skip behavior is deterministic
  • Defined fallback sources for standards-exclusive report sections (metadata description, Recommended Actions, Risk Assessment) for when the standards subagent is skipped
  • Added ## Error Recovery section covering diff failure, subagent failure, malformed output, persistence failure, and clarifying-question deadlock

code-review-standards.agent.md:

  • Changed template references from passive "Follow" to active "Read and use" with missing-file recovery language for consumer workspaces where docs/ may be absent

Related Issue(s)

Fixes #

Type of Change

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)

AI Artifacts:

  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Sample Prompts (for AI Artifact Contributions)

User Request:

"Review my current branch changes against our Python coding standards."

Execution Flow:

  1. Agent computes diff using pr-reference scripts (now packaged in the collection)
  2. Step 3b reads python-foundational SKILL.md from .github/skills/... (repo) or skills/... (plugin)
  3. Step 3c discovers consumer skills under .github/skills/ or skills/ depending on context
  4. Reads report template from docs/templates/standards-review-output-format.md; recovers gracefully if absent
  5. Findings persisted via corrected review-artifacts.instructions.md path

Success Indicators:
Skills are loaded and cited in findings. Report template is read from disk. Behavior is identical from repo or plugin install. No "skill unavailable" warnings for python-foundational.

Testing

  • Validated end-to-end from the Coding Standards Plugin installation in VS Code, which surfaced all three failure categories
  • Verified dual-path fallback text present at Steps 3b and 3c via grep
  • Confirmed pr-reference symlink at plugins/coding-standards/skills/shared/pr-reference
  • Ran npm run plugin:generate, plugin:validate, lint:md, and lint:frontmatter — all passed with 0 errors

Checklist

Required Checks

  • Documentation is updated (if applicable)

…tep 4

The #file: reference was missing the coding-standards/ segment,
pointing to instructions/code-review/ instead of
instructions/coding-standards/code-review/.
…lution

- Convert docs/templates references from relative paths to workspace-root
  prose pattern (matching adr-creation, brd-builder, system-architecture-
  reviewer convention) — ensures resolution works in both repo and plugin
  contexts where .github/ prefix stripping shifts relative depth

- Fix Step 3b skill catalog path: try .github/skills/ first, fall back to
  skills/ for plugin/extension compatibility

- Fix Step 3c skill discovery scope: search .github/skills/ if exists,
  otherwise skills/ — ensures consumer skill discovery works in plugin

- Add pr-reference skill to coding-standards collection so diff-computation
  scripts are available to coding-standards-only plugin users

- Delete orphaned foundational-coding-guidelines.md from python-foundational
  references/ (duplicate of design-principles.md; not in SKILL.md table)
@erikschlegel erikschlegel self-assigned this Apr 3, 2026
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.71%. Comparing base (84ddd5d) to head (69ecd87).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1289      +/-   ##
==========================================
- Coverage   87.72%   87.71%   -0.02%     
==========================================
  Files          61       61              
  Lines        9320     9320              
==========================================
- Hits         8176     8175       -1     
- Misses       1144     1145       +1     
Flag Coverage Δ
pester 85.31% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

code-review-full:
- Document disable-model-invocation / Step 4 interaction; add verbatim
  fallback with warning when transformation rules cannot be applied
- Replace vague 'instruct it to skip' with prescribed skip phrases for
  both subagent invocations so skip behavior is deterministic
- Define fallback sources for standards-exclusive report sections
  (metadata description, Recommended Actions, Risk Assessment) when
  the standards subagent is skipped
- Add Error Recovery section covering diff failure, subagent failure,
  malformed output, persistence failure, and clarifying-question deadlock

code-review-standards:
- Change template references from passive 'Follow' to active 'Read and
  use' with missing-file recovery language for consumer workspaces where
  docs/ may be absent
@erikschlegel erikschlegel marked this pull request as ready for review April 3, 2026 09:57
@erikschlegel erikschlegel requested a review from a team as a code owner April 3, 2026 09:57
@erikschlegel erikschlegel changed the title Fix/code review file refs fix(agents): fix plugin compatibility and robustness for coding-standards code review agents Apr 3, 2026
Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these updates, I left a few comments if we can still finetune the skills discovery to be left over at the platform level and not file location (which is how we expect this should work). Wdyt?

Note in the issue comment you mention you deleted the orphaned file but I don't seen any deletionn?

…Steps 3b and 3c

Replace path-based skill resolution with platform-discoverable skill matching
to ensure portability across repo, extension, and plugin install contexts.
@erikschlegel
Copy link
Copy Markdown
Contributor Author

Note in the issue comment you mention you deleted the orphaned file but I don't seen any deletionn?

This was related to a PR fix that was merged yesterday

@erikschlegel
Copy link
Copy Markdown
Contributor Author

erikschlegel commented Apr 3, 2026

Good call @katriendg - I implemented your suggestions from the last review. LMK if those look good?

@erikschlegel erikschlegel marked this pull request as draft April 3, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: Code Review Standards agent: skill loading and template references fail silently in plugin/extension context

4 participants