Skip to content

Add Swarming service for Remote Task Gate#5206

Open
jardondiego wants to merge 18 commits intomasterfrom
swarming-service-remote-task
Open

Add Swarming service for Remote Task Gate#5206
jardondiego wants to merge 18 commits intomasterfrom
swarming-service-remote-task

Conversation

@jardondiego
Copy link
Collaborator

@jardondiego jardondiego commented Mar 13, 2026

Add remote task service for Swarming and include it in Remote Task gate logic.

Summary
This PR integrates Swarming as a first-class remote task execution service within the RemoteTaskGate. It introduces the SwarmingService, which encapsulates the logic for pushing uworker main tasks to Swarming, and updates the
RemoteTaskGate to utilize this service for unified task distribution.

Key Changes

  • New SwarmingService Implementation: Added a new service that implements the RemoteTaskInterface. It handles task creation by pushing them to Swarming using
    swarming.push_swarming_task, providing a standardized interface consistent with Kubernetes and GCP Batch services.
  • RemoteTaskGate Integration & Refactoring:
    • Service Map Expansion: Added SwarmingService to the internal service map of RemoteTaskGate.
    • Unified Batch Task Handling: Refactored create_utask_main_jobs to delegate task creation to SwarmingService when the SWARMING_REMOTE_EXECUTION feature flag is enabled. This simplifies the logic by allowing the SwarmingService to
      filter and process swarming-eligible tasks and return the remaining tasks for distribution to other backends.
    • Single Task Routing: Updated create_utask_main_job to explicitly route swarming-eligible tasks to the SwarmingService.
  • RemoteTaskAdapters Update: Added SWARMING to the RemoteTaskAdapters enum. This allows for dynamic control of its execution frequency via feature flags, matching the behavior of other remote execution platforms.

@jardondiego jardondiego force-pushed the swarming-service-remote-task branch 2 times, most recently from 6adf5ef to 241f5b6 Compare March 13, 2026 05:25
@jardondiego jardondiego force-pushed the swarming-service-remote-task branch from 0033ed9 to a54d1d5 Compare March 13, 2026 13:41
@jardondiego jardondiego force-pushed the swarming-service-remote-task branch 3 times, most recently from 017887b to 9702d8d Compare March 18, 2026 04:07
@jardondiego jardondiego force-pushed the swarming-service-remote-task branch from 9702d8d to 7200eda Compare March 18, 2026 04:41
@jardondiego jardondiego marked this pull request as ready for review March 19, 2026 00:30
@jardondiego jardondiego requested a review from a team as a code owner March 19, 2026 00:30
@jardondiego jardondiego requested a review from IvanBM18 March 19, 2026 00:30
@jardondiego jardondiego self-assigned this Mar 19, 2026
@jardondiego jardondiego force-pushed the swarming-service-remote-task branch 5 times, most recently from 9da970f to 615788e Compare March 19, 2026 01:52
Co-authored-by: Diego Jardon <37823380+jardondiego@users.noreply.github.com>
@jardondiego jardondiego force-pushed the swarming-service-remote-task branch from 2917cf7 to b79d077 Compare March 20, 2026 18:39
@jardondiego jardondiego force-pushed the swarming-service-remote-task branch from 45274e8 to 150e132 Compare March 20, 2026 19:10
Copy link
Collaborator

@IvanBM18 IvanBM18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@javanlacerda javanlacerda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, and I think we can merge it once some minor fixes are addressed, and then we can make the other adjustments in the following PRs.

We just need to be sure that the changes will not have any side effect in prod.

if len(remote_tasks) == 1:
unscheduled_tasks = []

if feature_flags.FeatureFlags.SWARMING_REMOTE_EXECUTION.enabled:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could keep this function cleaner if we could move this logic to an auxiliary function.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I see an auxiliary function with this single line. I guess I probably do not understand very well what would we encapsulate in that auxiliary function. Can you clarify?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point here is more to avoid having swarming specific logic into the generic create_utask_main_job.

@jardondiego jardondiego force-pushed the swarming-service-remote-task branch 3 times, most recently from c78524e to 78a1502 Compare March 21, 2026 06:17
@jardondiego jardondiego force-pushed the swarming-service-remote-task branch from 78a1502 to d4e44d5 Compare March 21, 2026 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants