Conversation
There was a problem hiding this comment.
Pull request overview
Adds first-class “model selection” support to the Copilot Engine SDK, allowing engines (and the local test CLI) to consume platform-selected model metadata and resolve an effective model to use.
Changes:
- Extend
JobDetailswith model selection fields (features.model_selection,selected_model,default_model,available_models, etc.). - Add SDK helpers
isModelSelectionEnabledandresolveSelectedModeland export them from the package entrypoint. - Update the integration guide and
engine-climock platform/runner to surface model selection fields and environment variables.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/index.ts |
Exposes model selection helpers/types from the SDK public API. |
src/client.ts |
Adds job fields plus helper APIs to detect/resolve selected models. |
docs/integration-guide.md |
Documents new job fields/env vars and shows SDK usage. |
cli/internal/server/server.go |
Mock platform server optionally returns model selection fields/feature flag. |
cli/internal/runner/runner.go |
Injects model selection-related env vars into the engine process. |
cli/cmd/engine-cli/main.go |
Adds CLI flags to control model selection fields and passes them into mock server/runner. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
This looks good to me in general, but question, how is an engine supposed to describe the types of models it cares about or that it is compatible with? |
|
@josebalius That is a good question. We still need the notion of "filtering" here so I will think on how we can do that. Thank you! |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
src/client.ts:561
- When
available_modelsis empty/absent, this returnscandidates[0], which can beundefinedeven though model selection is enabled. Consider handling the “enabled but no candidates” case explicitly (e.g., requirefallbackModel, warn/throw, or return a deterministic default) so engines don’t silently proceed without a resolved model.
if (availableModels.length === 0) {
return candidates[0];
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * Returns `undefined` when model selection is not enabled for the job | ||
| * (i.e. `features.model_selection` is not `true`), allowing engines that | ||
| * do not support model selection to ignore it entirely. |
There was a problem hiding this comment.
The JSDoc says this function returns undefined only when model selection is not enabled, but the implementation can also return undefined when model selection is enabled and there are no usable candidates (e.g. missing selected_model, default_model, and no fallback provided). Please either update the doc to reflect this behavior or change the function to always resolve a model (or throw) when features.model_selection === true.
This issue also appears on line 559 of the same file.
| if enableModelSelection { | ||
| env.SelectedModel = selectedModel | ||
| env.DefaultModel = defaultModel | ||
| env.AvailableModels = availableModels | ||
| env.ModelVendors = modelVendors | ||
| } |
There was a problem hiding this comment.
With --enable-model-selection, the mock job response will set features.model_selection=true even if none of --selected-model, --default-model, or --available-model are provided, resulting in a job payload with the feature enabled but no model information. Consider validating the flag combination and returning an error (or auto-populating a default) to avoid confusing local engine testing.
We need to support model selection (I think optionally) in the new engine SDK. This PR adds support for this.
https://github.com/github/sweagentd/issues/10067