Skip to content

feat: Round-robin session queue scheduling across users#127

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/enhancement-round-robin-job-scheduling
Draft

feat: Round-robin session queue scheduling across users#127
Copilot wants to merge 2 commits intomainfrom
copilot/enhancement-round-robin-job-scheduling

Conversation

Copy link

Copilot AI commented Mar 10, 2026

In multiuser mode, a single user could monopolize the queue by enqueueing large batches, forcing other users to wait indefinitely. This adds a round_robin queue mode that interleaves jobs across users so each gets a turn before any user gets a second slot.

Changes

  • New config field session_queue_mode ("FIFO" | "round_robin", default "round_robin"): controls dequeue ordering. Configurable via invokeai.yaml, env var (INVOKEAI_SESSION_QUEUE_MODE), or CLI.
  • Single-user mode always uses FIFOsession_queue_mode is ignored when multiuser=False.
  • Round-robin dequeue() SQL: uses two CTEs — user_last_served tracks MAX(started_at) per user; user_next_item selects each user's best pending item (priority DESC, item_id ASC). Rows are ordered by COALESCE(last_served_at, '1970-01-01') ASC so the least-recently-served user always goes next.
  • Tests: 10 new tests covering FIFO and round-robin behavior, including the exact interleaving example from the issue:
Queued Processed
A1, A2, B1, C1, C2, A3 A1, B1, C1, A2, C2, A3

QA Instructions

  1. Run with multiuser: true in invokeai.yaml (default session_queue_mode: round_robin).
  2. Enqueue several batches as two different users — confirm jobs alternate per user rather than draining one user's queue fully before moving to the next.
  3. Set session_queue_mode: FIFO and confirm strict insertion-order is restored.
  4. Run with multiuser: false — confirm FIFO is used regardless of session_queue_mode.

Run the new unit tests:

pytest tests/app/services/session_queue/test_session_queue_dequeue.py -v

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)
Original prompt

This section details on the original issue you should resolve

<issue_title>[enhancement]: Round-robin generation sessions across users</issue_title>
<issue_description>### Is there an existing issue for this?

  • I have searched the existing issues

Contact Details

No response

What should this feature add?

Right now, when running in multiuser mode, the session manager takes jobs off the queue in a FIFO manner. However, when multiple users are working with the same backend, a greedy user can enqueue 100 rendering jobs, forcing all other users to wait their turn.

This feature would change the queue logic such that the jobs for each active user are dequeued in such a way that each user is served in turn. That is, if there are users A, B and C, and their jobs are queued like this:

A job 1
A job 2
B job 1
C job 1
C job 2
A job 3

They will be processed in this order:

A job 1
B job 1
C job 1
A job 2
C job 2
A job 3

The dequeueing behavior should be controlled by a configuration variable session_queueing with one of the values 'FIFO' (traditional behavior) or round_robin (new behavior).

If multiuser mode is active, then round_robin is the default. If in single user mode, use FIFO and ignore the round_robin option.

Alternatives

No response

Additional Content

No response</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

- Add SESSION_QUEUE_MODE type and session_queue_mode config field
- Modify dequeue() to support round-robin ordering when multiuser mode
  is active, serving each user in turn based on last-served timestamp
- Add tests for FIFO and round-robin dequeue behavior

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Copilot AI changed the title [WIP] Add round-robin generation sessions across users feat: Round-robin session queue scheduling across users Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[enhancement]: Round-robin generation sessions across users

2 participants