Skip to content

Add union query support#2146

Open
seladb wants to merge 13 commits intotortoise:developfrom
seladb:add-union
Open

Add union query support#2146
seladb wants to merge 13 commits intotortoise:developfrom
seladb:add-union

Conversation

@seladb
Copy link
Contributor

@seladb seladb commented Mar 17, 2026

Add union query support in an API similar to Django's API

Addresses these issues:
#1507
#2056 (partially)

Description

Add union query support:

qs1 = Tournament.filter(name="T1").only("id")
qs2 = Reporter.filter(name="R1").only("id")

result = await qs1.union(qs2)

As opposed to Django, this API supports result of multiple models (in Django all results will be instances of the first model).
However, similar to Django, the requested field names and types should be the same across all queried models. This is because of how DBs support union queries.

Motivation and Context

It addresses the following issues:
#1507
#2056 (partially)

Also - it brings more feature parity with Django.

How Has This Been Tested?

Added test coverage for all written code. It was tested with MySQL, Postgres and SQLite.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added the changelog accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@codspeed-hq
Copy link

codspeed-hq bot commented Mar 17, 2026

Merging this PR will not alter performance

✅ 24 untouched benchmarks


Comparing seladb:add-union (ba5973e) with develop (e6c7f64)

Open in CodSpeed

assert result == expected


@requireCapability(dialect=NotEQ("mssql"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MSSql limit syntax is different but it's not currently supported in _SetOperation which has its own implementation of _limit_sql() instead of using the dialect SQL. I guess we can leave it as a TODO for later

https://github.com/tortoise/pypika-tortoise/blob/378f6145f7529d88fa58510851d80454b742406e/pypika_tortoise/queries.py#L672

@seladb seladb marked this pull request as ready for review March 18, 2026 09:10
Copy link

@themavik themavik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the changes — the implementation is clean and follows the existing patterns.

@classmethod
def _get_selects(cls, qs: QuerySet[Model] | UnionQuery[Model]) -> list[str]:
return [
select.name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That will probably fail on query with .annotate(..) - if we don't support them we would need some clean error about that, not attribute error


self._union_query = self._union_query.orderby(field_name, order=order)

if self._limit is not None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we support offset too in such case? or there are some limitations?

union = self._clone()
union._models = {*union._models, *(qs.model for qs in other_qs)}
union._qs = union._qs + other_qs
union._all = all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would overwrite existing _all from previous union call - is it intended behavior?


.order_by('name', '-id')

Supports ordering by related models too.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it really support ordering by related models?

self._selects = self._get_selects(qs)
else:
if self._get_selects(qs) != self._selects:
raise ValueError("Union queries must have the same select fields")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should be ParamsError

Return count of objects in union query.
"""
self._choose_db_if_not_chosen()
self._make_query()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is issue here, that count mutates query, and if you will have code like

asyncio.gather(union_query, union_query.count()) - it will apply _make_query two times on same, which could fail

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants