Skip to content

Listener metrics missing gha_available_jobs and gha_acquired_jobs from RunnerScaleSetStatistic #4429

@PrateekKumar1709

Description

@PrateekKumar1709

Checks

Controller Version

0.14.0

Deployment Method

Helm

Checks

  • This isn't a question or user support case (For Q&A and community support, go to Discussions).
  • I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

1. Deploy ARC with listener metrics enabled
2. Scrape the listener's /metrics endpoint
3. Observe that `gha_assigned_jobs`, `gha_running_jobs`, `gha_registered_runners`, `gha_busy_runners`, and `gha_idle_runners` are present
4. Note that `gha_available_jobs` and `gha_acquired_jobs` are missing despite the data being available in `RunnerScaleSetStatistic`

Describe the bug

The RunnerScaleSetStatistic struct from github.com/actions/scaleset provides 7 fields, but RecordStatistics() in cmd/ghalistener/metrics/metrics.go only exposes 5 of them as Prometheus gauges. Two fields — TotalAvailableJobs and TotalAcquiredJobs — are silently dropped.

The metrics ADR (docs/adrs/2023-05-08-exposing-metrics.md) explicitly proposed available_jobs and acquired_jobs as listener metrics, but they were never implemented.

RunnerScaleSetStatistic field Listener metric Status
TotalAvailableJobs Not exposed
TotalAcquiredJobs Not exposed
TotalAssignedJobs gha_assigned_jobs Exposed
TotalRunningJobs gha_running_jobs Exposed
TotalRegisteredRunners gha_registered_runners Exposed
TotalBusyRunners gha_busy_runners Exposed
TotalIdleRunners gha_idle_runners Exposed

Without these metrics, operators cannot observe the job queue depth for their scale sets.

Describe the expected behavior

All 7 fields from RunnerScaleSetStatistic should be exposed as listener metrics. Specifically:

  • gha_available_jobs — number of jobs with matching runs-on labels waiting to be acquired by the scale set
  • gha_acquired_jobs — number of jobs acquired by the scale set but not yet assigned to a runner

These represent the job queue depth (available + acquired = jobs waiting for runners) and are essential for capacity planning and monitoring.

Additional Context

N/A - This is a code-level issue. The data is already flowing from the Actions service
through the listener via RunnerScaleSetStatistic. The fix is purely additive: define two
new gauge metrics and wire them in RecordStatistics(). No CRD, interface, or dependency
changes required.

Controller Logs

N/A - This is not a runtime bug. It is a missing metric implementation identified through
code review. The relevant code is in cmd/ghalistener/metrics/metrics.go RecordStatistics().

Runner Pod Logs

N/A - See above. No runtime reproduction needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggha-runner-scale-setRelated to the gha-runner-scale-set modeneeds triageRequires review from the maintainers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions