Skip to content

SDK supports model selection#6

Open
kinseydurhamgrace wants to merge 7 commits intomainfrom
kdg/support-model-selection
Open

SDK supports model selection#6
kinseydurhamgrace wants to merge 7 commits intomainfrom
kdg/support-model-selection

Conversation

@kinseydurhamgrace
Copy link
Copy Markdown

@kinseydurhamgrace kinseydurhamgrace commented Mar 23, 2026

We need to support model selection (I think optionally) in the new engine SDK. This PR adds support for this.

https://github.com/github/sweagentd/issues/10067

@kinseydurhamgrace kinseydurhamgrace marked this pull request as ready for review March 24, 2026 16:44
@kinseydurhamgrace kinseydurhamgrace requested a review from a team as a code owner March 24, 2026 16:44
Copilot AI review requested due to automatic review settings March 24, 2026 16:44
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class “model selection” support to the Copilot Engine SDK, allowing engines (and the local test CLI) to consume platform-selected model metadata and resolve an effective model to use.

Changes:

  • Extend JobDetails with model selection fields (features.model_selection, selected_model, default_model, available_models, etc.).
  • Add SDK helpers isModelSelectionEnabled and resolveSelectedModel and export them from the package entrypoint.
  • Update the integration guide and engine-cli mock platform/runner to surface model selection fields and environment variables.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/index.ts Exposes model selection helpers/types from the SDK public API.
src/client.ts Adds job fields plus helper APIs to detect/resolve selected models.
docs/integration-guide.md Documents new job fields/env vars and shows SDK usage.
cli/internal/server/server.go Mock platform server optionally returns model selection fields/feature flag.
cli/internal/runner/runner.go Injects model selection-related env vars into the engine process.
cli/cmd/engine-cli/main.go Adds CLI flags to control model selection fields and passes them into mock server/runner.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@josebalius
Copy link
Copy Markdown
Member

This looks good to me in general, but question, how is an engine supposed to describe the types of models it cares about or that it is compatible with?

@kinseydurhamgrace
Copy link
Copy Markdown
Author

@josebalius That is a good question. We still need the notion of "filtering" here so I will think on how we can do that. Thank you!

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/client.ts:561

  • When available_models is empty/absent, this returns candidates[0], which can be undefined even though model selection is enabled. Consider handling the “enabled but no candidates” case explicitly (e.g., require fallbackModel, warn/throw, or return a deterministic default) so engines don’t silently proceed without a resolved model.
    if (availableModels.length === 0) {
        return candidates[0];
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +515 to +517
* Returns `undefined` when model selection is not enabled for the job
* (i.e. `features.model_selection` is not `true`), allowing engines that
* do not support model selection to ignore it entirely.
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSDoc says this function returns undefined only when model selection is not enabled, but the implementation can also return undefined when model selection is enabled and there are no usable candidates (e.g. missing selected_model, default_model, and no fallback provided). Please either update the doc to reflect this behavior or change the function to always resolve a model (or throw) when features.model_selection === true.

This issue also appears on line 559 of the same file.

Copilot uses AI. Check for mistakes.
Comment on lines +285 to +290
if enableModelSelection {
env.SelectedModel = selectedModel
env.DefaultModel = defaultModel
env.AvailableModels = availableModels
env.ModelVendors = modelVendors
}
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With --enable-model-selection, the mock job response will set features.model_selection=true even if none of --selected-model, --default-model, or --available-model are provided, resulting in a job payload with the feature enabled but no model information. Consider validating the flag combination and returning an error (or auto-populating a default) to avoid confusing local engine testing.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants