Skip to content

feat: add toArray() for includes subqueries#1295

Merged
kevin-dp merged 14 commits intokevin/includesfrom
kevin/includes-to-array
Mar 12, 2026
Merged

feat: add toArray() for includes subqueries#1295
kevin-dp merged 14 commits intokevin/includesfrom
kevin/includes-to-array

Conversation

@kevin-dp
Copy link
Contributor

Summary

  • Adds toArray() wrapper that converts child Collections to plain Array<T> on parent rows
  • When children change, the parent row is re-emitted with a fresh array snapshot
  • Supports ordering and limits on toArray subqueries

Test plan

  • Basic toArray produces arrays, not Collections
  • Empty parents get []
  • Adding/removing a child re-emits parent with updated array
  • Array respects ORDER BY
  • Ordered toArray with LIMIT applied per parent
  • Existing Collection-based includes still work unchanged
  • Full db test suite passes with no regressions

🤖 Generated with Claude Code

@changeset-bot
Copy link

changeset-bot bot commented Feb 25, 2026

🦋 Changeset detected

Latest commit: eb8ac7d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages
Name Type
@tanstack/db Patch
@tanstack/angular-db Patch
@tanstack/electric-db-collection Patch
@tanstack/offline-transactions Patch
@tanstack/powersync-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Feb 25, 2026

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1295

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1295

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1295

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1295

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1295

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1295

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1295

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1295

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1295

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1295

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1295

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1295

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1295

commit: 2087b61

@github-actions
Copy link
Contributor

github-actions bot commented Feb 25, 2026

Size Change: +440 B (+0.46%)

Total Size: 96.5 kB

Filename Size Change
./packages/db/dist/esm/index.js 2.71 kB +11 B (+0.41%)
./packages/db/dist/esm/query/builder/functions.js 792 B +59 B (+8.05%) 🔍
./packages/db/dist/esm/query/builder/index.js 4.65 kB +76 B (+1.66%)
./packages/db/dist/esm/query/compiler/index.js 2.69 kB +19 B (+0.71%)
./packages/db/dist/esm/query/ir.js 757 B +19 B (+2.57%)
./packages/db/dist/esm/query/live/collection-config-builder.js 7.91 kB +256 B (+3.35%)
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.39 kB
./packages/db/dist/esm/collection/changes.js 1.22 kB
./packages/db/dist/esm/collection/events.js 388 B
./packages/db/dist/esm/collection/index.js 3.32 kB
./packages/db/dist/esm/collection/indexes.js 1.1 kB
./packages/db/dist/esm/collection/lifecycle.js 1.75 kB
./packages/db/dist/esm/collection/mutations.js 2.34 kB
./packages/db/dist/esm/collection/state.js 3.49 kB
./packages/db/dist/esm/collection/subscription.js 3.71 kB
./packages/db/dist/esm/collection/sync.js 2.41 kB
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.7 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/indexes/auto-index.js 742 B
./packages/db/dist/esm/indexes/base-index.js 766 B
./packages/db/dist/esm/indexes/btree-index.js 2.17 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.1 kB
./packages/db/dist/esm/indexes/reverse-index.js 538 B
./packages/db/dist/esm/local-only.js 808 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 1.05 kB
./packages/db/dist/esm/query/compiler/evaluators.js 1.43 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 2.24 kB
./packages/db/dist/esm/query/compiler/joins.js 2.11 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.5 kB
./packages/db/dist/esm/query/compiler/select.js 1.11 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/collection-subscriber.js 2.42 kB
./packages/db/dist/esm/query/live/internal.js 145 B
./packages/db/dist/esm/query/optimizer.js 2.62 kB
./packages/db/dist/esm/query/predicate-utils.js 2.97 kB
./packages/db/dist/esm/query/subset-dedupe.js 921 B
./packages/db/dist/esm/scheduler.js 1.3 kB
./packages/db/dist/esm/SortedMap.js 1.3 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 924 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/comparison.js 952 B
./packages/db/dist/esm/utils/cursor.js 457 B
./packages/db/dist/esm/utils/index-optimization.js 1.51 kB
./packages/db/dist/esm/utils/type-guards.js 157 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Feb 25, 2026

Size Change: 0 B

Total Size: 3.7 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.34 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 559 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

@kevin-dp kevin-dp force-pushed the kevin/includes-to-array branch 5 times, most recently from 65948a8 to b152564 Compare February 25, 2026 13:46
@samwillis
Copy link
Collaborator

Review from GPT5.4:


Findings

  1. packages/db/src/query/builder/functions.ts, packages/db/src/query/builder/types.ts

toArray() isn’t wired into the result type inference, so the new public API loses type safety even though the runtime behavior works.

Right now toArray() returns a non-generic ToArrayWrapper, and ResultTypeFromSelect never maps that wrapper to Array<GetResult<...>>. As a result, a projection like issues: toArray(...) won’t infer as Issue[] on the parent row, which is a user-visible regression for the feature this PR is adding.

I think this needs a dedicated type-level branch in the select result inference so the wrapper unwraps to the child query result array type.

  1. packages/db/tests/query/includes.test.ts, packages/db/tests/query/*.test-d.ts

The runtime coverage here is good, but there’s no .test-d.ts coverage for the new toArray() API. That leaves the typing regression above completely unguarded.

A focused type test around something like issues: toArray(q.from(...).select(...)) and an assertion that the parent result field is Array<...> would catch this immediately and help keep the public API stable.

@samwillis
Copy link
Collaborator

This is looking very good, other than the type issue that GPT5.4 found above, the other thing we need to do is ensure there are tests that validate the reactivity model:

  • non-toArray subscribe to the main result collection and validate that changes that result in a child change don't trigger a change event on the parent.
  • toArray same, but validate that there is an event on the child collection.

I believe these tests will pass, but we should have them as they are a core design feature.

@kevin-dp kevin-dp force-pushed the kevin/includes-to-array branch 2 times, most recently from 3aa7796 to 8859843 Compare March 9, 2026 14:24
@kevin-dp kevin-dp requested a review from samwillis March 9, 2026 15:29
kevin-dp and others added 13 commits March 10, 2026 10:20
toArray() wraps an includes subquery so the parent row contains
Array<T> instead of Collection<T>. When children change, the parent
row is re-emitted with a fresh array snapshot.

- Add ToArrayWrapper class and toArray() function
- Add materializeAsArray flag to IncludesSubquery IR node
- Detect ToArrayWrapper in builder, pass flag through compiler
- Re-emit parent rows on child changes for toArray entries
- Add SelectValue type support for ToArrayWrapper
- Add tests for basic toArray, reactivity, ordering, and limits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make ToArrayWrapper generic so it carries the child query result type,
and add a ToArrayWrapper branch in ResultTypeFromSelect to unwrap it
to Array<T>.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The toArray re-emit in flushIncludesState mutated parent items in-place
before writing them through parentSyncMethods.begin/write/commit.
Since commitPendingTransactions captures "previous visible state" by
reading syncedData.get(key) — which returns the already-mutated object
— deepEquals always returned true and suppressed the change event.

Replace the sync methods pattern with direct event emission: capture a
shallow copy before mutation (for previousValue), mutate in-place (so
collection.get() works), and emit UPDATE events directly via the parent
collection's changes manager.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Test the reactive model difference between Collection and toArray includes:
- Collection includes: child change does NOT re-emit the parent row
  (the child Collection updates in place)
- toArray includes: child change DOES re-emit the parent row
  (the parent row is re-emitted with the updated array snapshot)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@kevin-dp kevin-dp force-pushed the kevin/includes-to-array branch from f491fd4 to 4c41062 Compare March 10, 2026 09:20
Copy link
Collaborator

@samwillis samwillis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@kevin-dp kevin-dp merged commit 1d81fa3 into kevin/includes Mar 12, 2026
6 checks passed
@kevin-dp kevin-dp deleted the kevin/includes-to-array branch March 12, 2026 12:06
kevin-dp added a commit that referenced this pull request Mar 12, 2026
Resolve conflicts to combine parent-ref-filter support (parentFilters,
parentProjection) with materializeAsArray (toArray) feature from PR #1295.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants