fix: Code Review Standards agent: skill loading and template references fail silently in plugin/extension context

[erikschlegel]

Component

Custom Agents

Bug Description

Title

Code Review Standards agent: skill loading and template references fail silently in plugin/extension context

Body

When the Code Review Standards agent is installed via the VS Code plugin or extension (i.e., from the coding-standards collection), skill loading and several file references fail silently, producing reviews with no applied coding standards.

Context

The plugin generator strips the .github/ prefix from all artifact paths when building the plugin. An agent at .github/agents/coding-standards/ (3 levels deep in the repo) becomes agents/coding-standards/ (2 levels deep in the plugin). Any path reference that assumes the .github/ depth is wrong in plugin context.

Observed Failures

Relative docs/templates/ links escape the plugin root. The agent references ../../../docs/templates/standards-review-output-format.md and ../../../docs/templates/engineering-fundamentals.md. In plugin context, traversing three levels up escapes the plugin root entirely — the files are never found and the agent has no output template or design principles to apply.

Skill loading fails entirely. Step 3b resolves skill names via .github/skills/coding-standards/{name}/SKILL.md and Step 3c searches under .github/skills/. Neither path exists in the plugin — skills live under skills/ (no .github/ prefix). The agent proceeds with zero loaded skills, so no coding standard findings are ever produced.

pr-reference scripts unavailable. The diff computation protocol relies on scripts from the pr-reference skill, but that skill is not included in coding-standards.collection.yml. Users who install the coding-standards collection cannot run the diff computation step.

Orphaned reference file. foundational-coding-guidelines.md exists under python-foundational/references/ but is not listed in SKILL.md's References table and duplicates the content of design-principles.md.

Labels

bug, agents, skills, packaging, priority-2

Expected Behavior

The agent should load skills, resolve template references, and execute diff computation correctly regardless of whether it is run from the repo or from a plugin/extension install.

Steps to Reproduce

No response

Additional Context

No response

[katriendg]

Thanks for logging this issue, good thing we have only released in experimental which is where these kind of issues surface.

1. PR reference skill is a dependency but since it's in the agent grammar we cannot automatically detect the collection is missing it. Should be added to the collection.
2. The discovery of skills should be by their title for progressive disclosure, some testing is needed but addressing the skill by its name or contents of description should allows us to solve this issue. I did not realize we had left any hard coded references not using the skill name vs the skill file location. This slipped when we merged.
3. Orphaned file needs to be referenced, seems like we can remove one of the duplicates and keep the right one in the skill?
4. The docs template location is a more complex one, we need to account for the extension distribution as well and should try to get repo cloning, extension or plugin distribution to have the same directory levels. It's been a recurring problem for which we have fixed other folder references but clearly not for some of our agents. This latter one may require additional work on generating a different folder structure or instructing the agent to search elsewhere.

Would you want to take on points 1 to 3, and we look internally how to best address point 4 which is probably still affecting other artifacts as well?

[erikschlegel]

Agreed @katriendg . Posted PR #1289 which should address the issue and points raised above. Happy to address any additional feedback when you get a chance to review.

#	Issue	Status
1	pr-reference skill missing from the collection	✅ Fixed — added to coding-standards.collection.yml
2	Skill discovery using hardcoded file paths instead of name/description	✅ Fixed — Steps 3b and 3c now use a dual-path fallback (.github/skills/ → skills/) and discover by description
3	Orphaned duplicate file	✅ Fixed — foundational-coding-guidelines.md deleted, design-principles.md retained
4	docs/templates/ path broken in extension/plugin distribution	✅ Resolved for this case — converted from #file:../../../docs/templates/ relative links to workspace-root prose (`docs/templates/file.md`), matching the convention used by other agents. The broader structural concern for other artifacts remains open.