Move game store discovery into main-process scanners by halgari · Pull Request #22011 · Nexus-Mods/Vortex

halgari · 2026-03-30T20:23:20Z

Summary

This PR replaces the old renderer-side game store extension model with a main-process discovery system backed by DuckDB query data.

The new flow is:

main process initializes store scanners for Steam, GOG, Epic, Xbox, Origin, Uplay, and registry
scanner results are written into a store_games DuckDB table
the generic query IPC layer exposes all_store_games to the renderer
renderer-side discovery and launch helpers consume query-backed store data instead of store-specific extension objects
explicit dirty-table invalidation keeps query consumers up to date after discovery writes

Changes

1. Main-process discovery and query plumbing

Adds DiscoveryCoordinator, IStoreScanner, and store-specific scanners under src/main/src/games/scanners
Adds command IPC support for discovery.start
Adds generic query IPC (query:execute and query:dirty) and wires discovery init into main persistence startup
Adds store_games setup/select SQL and generated query types
Stores discovery results in DuckDB and invalidates store_games after each successful run

2. CLI support

Adds a cli:list-games entry point and bundle in @vortex/main
The CLI runs discovery, queries all_store_games, and prints store, id, name, and install path
Uses a CLI-safe logging shim so the bundle does not depend on Electron logging setup

3. Renderer migration to query-backed stores

Adds a small query client for renderer-side cached query execution with dirty-query invalidation
Reworks GameStoreHelper to load store data from all_store_games rather than calling legacy store implementations
Reworks StarterInfo and game mode discovery to use the new query-backed helper methods
startQuickDiscovery() now triggers a main-process scan and then consumes the query results

4. Standardized game finders and store-specific cleanup

Removes bundled store-specific extensions for GOG, Origin, Uplay, and Xbox
Updates game extensions to use standardized queryArgs / GameStoreHelper.findBy*() lookups instead of store-specific utility modules and direct launcher objects
This simplifies the game finder path and standardizes store detection across extensions instead of each store shipping its own independent lookup/cache logic
Also removes renderer API exports and hooks that only existed to support the old store-extension model

Breaking Changes

This branch changes extension-facing behavior in ways that can break extensions which rely on the old store registration model.

Notable breakages:

registerGameStore is removed from IExtensionContext
bundled extensions/gamestore-* packages are removed for GOG, Origin, Uplay, and Xbox
renderer-side code should no longer assume store-specific launcher objects exist or can be fetched/registered dynamically
extensions that imported or depended on old store-specific utilities/exports need to migrate to the standardized query-backed GameStoreHelper and queryArgs flow

In practice, any out-of-tree/community extension that still expects to register a custom game store or talk to the removed bundled store extensions will need an update.

insomnious · 2026-03-30T20:37:57Z

Do we have any data on how many extensions (if any) are affected? Maybe @erri120 extension's analysis?

halgari · 2026-03-30T20:58:28Z

Would be good to find out. All the ones in our codebase are very old, and the small changes I included fixed those. Any extension that uses "query args" is fine and I think most do.

halgari · 2026-03-31T01:57:42Z

I considered updating the extensions that used the util.* endpoints to use queryArgs, we can still do that, but it seems like a bit of scope creep. I could be convinced otherwise however.

erri120 · 2026-03-31T07:28:42Z

Do we have any data on how many extensions (if any) are affected? Maybe @erri120 extension's analysis?

Based on all extensions (data snapshot from last month):

registerGameStore got removed and is not called by any extension,
util.steam got removed but is used 33 times by extensions, see export.csv for usage,
util.epicGamesLauncher got removed but is used 7 times by extensions, see export.csv for usage,
util.GameNotFound got removed but is used 3 times by extensions, see export.csv for usage.

Extensions are using util.steam.findByName and util.steam.findByAppId, to keep compatibility I recommend having a util.steam stub that redirects findByName and findByAppId to an actual implementation. Annotate it with @deprecated and make a note in the API changelog that this will be removed soon ™️. We can also let the extension authors know directly about this change. Same with util.epicGamesLauncher which has util.epicGamesLauncher.findByAppId, util.epicGamesLauncher.findByName, and util.epicGamesLauncher.isGameInstalled usages in extensions.

Queries used for verifying API changes:

-- checks for usage of 'registerGameStore'
SELECT
  extensionId,
  json_extract_string(calls.value, '$.method') AS method
FROM extensions,
  json_each(value, '$.analysis') AS file,
  json_each(json_extract(file.value, '$.apiUsage.calls')) AS calls
WHERE method = 'registerGameStore'

-- checks for explicit steam usage
SELECT
  extensionId,
  file.key AS file,
  json_extract_string(component.value, '$.component') AS component
FROM
  extensions,
  json_each (extensions.value, '$.analysis') AS file,
  json_each (json_extract(file.value, '$.vortexApiUsage.components')) AS component
WHERE
  extensionId NOT LIKE 'github-Nexus-Mods-vortex-games%' AND
  json_extract_string(component.value, '$.component') ILIKE '%steam%'
ORDER BY
  component ASC,
  extensionId ASC,
  file ASC;

erri120 · 2026-03-31T07:37:12Z

src/shared/src/types/preload.ts

+/** Store game entry as returned from main-process discovery */
+export interface IStoreGameRow {
+  store_type: string;
+  store_id: string;
+  install_path: string;
+  name: string | null;
+  store_metadata: string | null;
+}


Is this misplaced? Doesn't seem to be used as a preload type.

yes, I can remove it now, we used to send game store results across the IPC, but now it's more "find" based instead of querying for all games in one call

erri120 · 2026-03-31T07:39:57Z

src/renderer/src/extensions/gamemode_management/util/discovery.ts

+      ? Bluebird.all([
+          Bluebird.resolve(matchStoreGames(game.queryArgs, storeGames)),
+          game.queryArgs.registry !== undefined
+            ? GameStoreHelper.find({ registry: game.queryArgs.registry })
+            : Bluebird.resolve([]),


matchStoreGames returns IGameStoreEntry[] the method isn't async so we don't need Bluebird here.

erri120 · 2026-03-31T07:40:29Z

src/renderer/src/extensions/gamemode_management/GameModeManager.test.ts

New tests should use vitest.

erri120 · 2026-03-31T07:40:38Z

src/renderer/src/extensions/gamemode_management/util/discoveryQueries.test.ts

New tests should use vitest.

erri120 · 2026-03-31T08:19:24Z

src/renderer/src/util/queryClient.ts

+  force?: boolean;
+}
+
+function toQueryError(err: unknown): Error {


Should have custom QueryError type at this point.

erri120 · 2026-03-31T08:28:18Z

src/renderer/src/util/GameStoreHelper.ts

+    const entry = {
+      appid: row.store_id,
+      gamePath: row.install_path,
+      name: row.name ?? "",
+      gameStoreId: row.store_type,
+      priority: this.storePriority(row.store_type),
+    } as IGameStoreEntry & Record<string, unknown>;


All keys are known since they come from IStoreGameMetadata.

Suggested change

const entry = {

appid: row.store_id,

gamePath: row.install_path,

name: row.name ?? "",

gameStoreId: row.store_type,

priority: this.storePriority(row.store_type),

} as IGameStoreEntry & Record<string, unknown>;

const entry: IGameStoreEntry & Partial<Record<keyof IStoreGameMetadata, unknown>> = {

appid: row.store_id,

gamePath: row.install_path,

name: row.name ?? "",

gameStoreId: row.store_type,

priority: this.storePriority(row.store_type),

};

erri120 · 2026-03-31T08:29:20Z

src/renderer/src/util/GameStoreHelper.ts

+    return toBlue(async (): Promise<void> => {
+      if (!appInfo) {
+        throw new ProcessCanceled("appInfo is undefined/null");
+      }


No bluebird.

erri120 · 2026-03-31T08:30:58Z

src/renderer/src/util/GameStoreHelper.ts

+  private launchGOGGame(api: IExtensionApi, appInfo: any): Bluebird<void> {
+    return toBlue(async (): Promise<void> => {


No bluebird.

erri120 · 2026-03-31T08:32:57Z

src/renderer/src/util/GameStoreHelper.ts

Lots of bluebird references that can be removed. Public facing API should return PromiseLike<T> for compatibility and implementations can use native Promise<T>.

erri120 · 2026-03-31T08:40:27Z

src/main/src/games/scanners/SteamScanner.ts

+    // eslint-disable-next-line no-restricted-imports
+    require("node:fs").statSync(libraryFoldersPath);


No reason to use require here, can just do import { statSync } from"node:fs"; at the top if you need it...

erri120

Important points:

API changes break extensions, instead use stubs that redirect and mark them as deprecated.
No new CLI. Our build process is still a mess and I really don't want to complicate it even more with a second entry point. This PR is already large enough, we don't need a CLI in here.

Code quality:

no new Jest tests, new tests should use Vitest,
no Bluebird resolve code, use PromiseLike<T> for API and native Promise<T> for implementations

Phase 0 of Game Adaptors architecture: all store scanners (Steam, GOG, Epic, Xbox, Origin, Uplay, Registry) now run in the main process and write results to a DuckDB store_games table. A query watcher pushes updates to the renderer via IPC. Infrastructure complete: - DuckDB store_games table + all_store_games query - IStoreScanner interface + DiscoveryCoordinator - 7 scanner implementations migrated from renderer - IPC channels (discovery:start, discovery:get-store-games, etc.) - Preload API exposure (window.api.discovery.*) - Renderer quickDiscovery() refactored to use main-process data - Query type generation updated for non-pivot tables - SQL files copied to out/ during build Known issues to fix: - Scanner output contains undefined values causing DuckDB write failures - Debounce prevents IPC-triggered rescan after initial scan failure - Legacy fallback (GameStoreHelper) has timing issues when stores aren't loaded yet Design spec: docs/superpowers/specs/2026-03-26-game-adaptors-design.md Linear: Game Discovery project (APP-163 through APP-171, APP-178-179)

…ed compat - Add CLI bundle step to src/main/build.mjs that compiles list-games.ts into out/cli-list-games.cjs with logging alias swap - Add cli:list-games pnpm script to src/main/package.json - Fix DiscoveryCoordinator to use a safeAllSettled helper that builds plain {status, value} objects, working around turbowalk's global Promise=bluebird patch which made Promise.allSettled return bluebird inspection objects - Add ON CONFLICT DO UPDATE to store_games INSERT to handle duplicate store_id entries (e.g. Steam games appearing in multiple library paths with different case)

Replace per-query IPC channels with a generic system. The main process broadcasts dirty query names via query:dirty after invalidation, and the renderer can execute any named query via query:execute. Removes discovery-specific query IPC (discovery:get-store-games, discovery:store-games-updated) in favor of the generic channels.

@deprecated

…nd stubs Extensions using the legacy game store API will continue to work via shims that delegate to util.GameStoreHelper with the appropriate storeId. Annotated @deprecated to signal migration path.

…ion unnecessary)

- Remove misplaced IStoreGameRow interface from preload.ts (already defined in discovery.ts where it belongs) - Add QueryError class to queryClient.ts so callers can distinguish query failures from other errors via instanceof - Replace require("node:fs").statSync with top-level import in SteamScanner.ts (removes eslint-disable comment)

…ive Promise - Change all public method signatures from Bluebird<T> to PromiseLike<T> so extensions consuming the API get a stable interface compatible with any Promise implementation - Convert private methods (runtimeLaunchStore, launchURI, launchGOGGame, launchXboxGame) from toBlue() wrappers to native async functions - Fix rowToEntry type cast: use Partial<Record<keyof IStoreGameMetadata, unknown>> instead of Record<string, unknown> for better type safety - Remove Bluebird and toBlue imports (no longer needed)

Move GameModeManager and discoveryQueries tests from Jest __tests__/ directories to colocated Vitest test files. Replace jest.fn()/jest.mock() with vi.fn()/vi.mock() and add explicit vitest imports.

matchStoreGames is synchronous so no Bluebird.resolve() wrapper needed. Replace Bluebird.all with Promise.all and convert to async/await.

github-actions · 2026-04-01T07:23:37Z

This PR has conflicts. You need to rebase the PR before it can be merged.

Aragas · 2026-04-01T10:51:44Z

I absolutely agree with that we need to have stubs, but I would expand it to have stubs for any API that was exposed by us and got removed. I believe just redirecting everything should not be an issue.
Ideally we need to also add at the same time a better API that a renderer extension could use to get the same data in a proper way. If it's already exposed by DuckDB alone, I guess just having the ability to query data is enough then

Just deprecate the stubs and show the proper way to do it in the future

halgari requested a review from a team as a code owner March 30, 2026 20:23

erri120 reviewed Mar 31, 2026

View reviewed changes

src/renderer/src/extensions/gamemode_management/GameModeManager.test.ts

Copy link
Copy Markdown

Member

erri120 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New tests should use vitest.

erri120 reviewed Mar 31, 2026

View reviewed changes

src/renderer/src/extensions/gamemode_management/util/discoveryQueries.test.ts

Copy link
Copy Markdown

Member

erri120 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New tests should use vitest.

erri120 reviewed Mar 31, 2026

View reviewed changes

erri120 requested changes Mar 31, 2026

View reviewed changes

halgari added 14 commits March 31, 2026 11:20

feat(cli): add minimal stderr logging module for CLI bundle

8b3342d

feat(cli): add list-games entry point

4d47643

refactor: move discovery IPC to cqrs flow

537ed3f

refactor: add renderer query client

234088e

Migrate store lookups and remove legacy gamestore extensions

91be6bb

fix: remove discovery rescan debounce

6b97991

fix: parameterize discovery table writes

02ad464

chore: remove game adaptor design doc

9cbc773

feat: include store ids in list-games output

5c8cb2b

fix: align query ipc payload types

73b615f

compat: re-export util.steam, util.epicGamesLauncher, util.GameNotFou…

63ecf2f

…nd stubs Extensions using the legacy game store API will continue to work via shims that delegate to util.GameStoreHelper with the appropriate storeId. Annotated @deprecated to signal migration path.

halgari added 4 commits March 31, 2026 11:20

revert: restore extensions/ to pre-PR state (compat stubs make migrat…

c62e2a9

…ion unnecessary)

test(game-scanners): convert __tests__/ to Vitest colocated tests

d2ce0b8

Move GameModeManager and discoveryQueries tests from Jest __tests__/ directories to colocated Vitest test files. Replace jest.fn()/jest.mock() with vi.fn()/vi.mock() and add explicit vitest imports.

halgari force-pushed the halgari/game-scanners branch from fcd23d6 to d2ce0b8 Compare March 31, 2026 17:20

fix(game-scanners): convert queryByArgs from Bluebird to native async

19608fa

matchStoreGames is synchronous so no Bluebird.resolve() wrapper needed. Replace Bluebird.all with Promise.all and convert to async/await.

github-actions bot added the PR: needs rebase label Apr 1, 2026

		private launchGOGGame(api: IExtensionApi, appInfo: any): Bluebird<void> {
		return toBlue(async (): Promise<void> => {

		// eslint-disable-next-line no-restricted-imports
		require("node:fs").statSync(libraryFoldersPath);

Conversation

halgari commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

1. Main-process discovery and query plumbing

2. CLI support

3. Renderer migration to query-backed stores

4. Standardized game finders and store-specific cleanup

Breaking Changes

Uh oh!

insomnious commented Mar 30, 2026

Uh oh!

halgari commented Mar 30, 2026

Uh oh!

halgari commented Mar 31, 2026

Uh oh!

erri120 commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

erri120 Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

erri120 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

Aragas commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

halgari commented Mar 30, 2026 •

edited

Loading

erri120 commented Mar 31, 2026 •

edited

Loading

erri120 Mar 31, 2026 •

edited

Loading

Aragas commented Apr 1, 2026 •

edited

Loading