Skip to content

Add MapDataReplayState coordinator class#5137

Open
ltiao wants to merge 1 commit intofacebook:mainfrom
ltiao:export-D98741817
Open

Add MapDataReplayState coordinator class#5137
ltiao wants to merge 1 commit intofacebook:mainfrom
ltiao:export-D98741817

Conversation

@ltiao
Copy link
Copy Markdown
Contributor

@ltiao ltiao commented Apr 3, 2026

Summary:
The experiment replay system (MapDataReplayMetric, MapDataReplayRunner, replay_experiment) is hardcoded for single-objective optimization, blocking multi-objective early stopping. MapDataReplayMetric conflates data serving with progression state, so multiple metrics cannot share a coherent timeline. This diff series extracts shared state into a MapDataReplayState coordinator.

This diff (1/3) adds the MapDataReplayState class -- a shared state coordinator that manages normalized cursor-based progression across multiple metrics and trials. Uses a global min/max MAP_KEY to preserve cross-metric timing alignment. Serves original MAP_KEY values; downstream ESS normalizes independently via _maybe_normalize_map_key.

Existing MapDataReplayMetric and helpers are unchanged in this diff.

Differential Revision: D98741817

Summary:
The experiment replay system (`MapDataReplayMetric`, `MapDataReplayRunner`, `replay_experiment`) is hardcoded for single-objective optimization, blocking multi-objective early stopping. `MapDataReplayMetric` conflates data serving with progression state, so multiple metrics cannot share a coherent timeline. This diff series extracts shared state into a `MapDataReplayState` coordinator.

This diff (1/3) adds the `MapDataReplayState` class -- a shared state coordinator that manages normalized cursor-based progression across multiple metrics and trials. Uses a global min/max MAP_KEY to preserve cross-metric timing alignment. Serves original MAP_KEY values; downstream ESS normalizes independently via `_maybe_normalize_map_key`.

Existing `MapDataReplayMetric` and helpers are unchanged in this diff.

Differential Revision: D98741817
@meta-cla meta-cla bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Apr 3, 2026
@meta-codesync
Copy link
Copy Markdown

meta-codesync bot commented Apr 3, 2026

@ltiao has exported this pull request. If you are a Meta employee, you can view the originating Diff in D98741817.

ltiao added a commit to ltiao/Ax that referenced this pull request Apr 3, 2026
Summary:

The experiment replay system (`MapDataReplayMetric`, `MapDataReplayRunner`, `replay_experiment`) is hardcoded for single-objective optimization, blocking multi-objective early stopping. `MapDataReplayMetric` conflates data serving with progression state, so multiple metrics cannot share a coherent timeline. This diff series extracts shared state into a `MapDataReplayState` coordinator.

This diff (1/3) adds the `MapDataReplayState` class -- a shared state coordinator that manages normalized cursor-based progression across multiple metrics and trials. Uses a global min/max MAP_KEY to preserve cross-metric timing alignment. Serves original MAP_KEY values; downstream ESS normalizes independently via `_maybe_normalize_map_key`.

Existing `MapDataReplayMetric` and helpers are unchanged in this diff.

Differential Revision: D98741817
ltiao added a commit to ltiao/Ax that referenced this pull request Apr 3, 2026
Summary:

The experiment replay system (`MapDataReplayMetric`, `MapDataReplayRunner`, `replay_experiment`) is hardcoded for single-objective optimization, blocking multi-objective early stopping. `MapDataReplayMetric` conflates data serving with progression state, so multiple metrics cannot share a coherent timeline. This diff series extracts shared state into a `MapDataReplayState` coordinator.

This diff (1/3) adds the `MapDataReplayState` class -- a shared state coordinator that manages normalized cursor-based progression across multiple metrics and trials. Uses a global min/max MAP_KEY to preserve cross-metric timing alignment. Serves original MAP_KEY values; downstream ESS normalizes independently via `_maybe_normalize_map_key`.

Existing `MapDataReplayMetric` and helpers are unchanged in this diff.

Differential Revision: D98741817
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.68421% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.41%. Comparing base (227243c) to head (2f64057).

Files with missing lines Patch % Lines
ax/metrics/map_replay.py 95.65% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main    #5137    +/-   ##
========================================
  Coverage   96.40%   96.41%            
========================================
  Files         613      613            
  Lines       68191    68342   +151     
========================================
+ Hits        65740    65889   +149     
- Misses       2451     2453     +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ltiao added a commit to ltiao/Ax that referenced this pull request Apr 3, 2026
Summary:

The experiment replay system (`MapDataReplayMetric`, `MapDataReplayRunner`, `replay_experiment`) is hardcoded for single-objective optimization, blocking multi-objective early stopping. `MapDataReplayMetric` conflates data serving with progression state, so multiple metrics cannot share a coherent timeline. This diff series extracts shared state into a `MapDataReplayState` coordinator.

This diff (1/3) adds the `MapDataReplayState` class -- a shared state coordinator that manages normalized cursor-based progression across multiple metrics and trials. Uses a global min/max MAP_KEY to preserve cross-metric timing alignment. Serves original MAP_KEY values; downstream ESS normalizes independently via `_maybe_normalize_map_key`.

Existing `MapDataReplayMetric` and helpers are unchanged in this diff.

Differential Revision: D98741817
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants