Malformed NodeJS Packages

[CalmDownVal]

Hello, I'm running into issues trying to use this extension in NodeJS. I identified multiple problems:

1. The sqlite-vec package has broken platform/arch checks

The sqlite-vec package acting as the "router" for the platform-specific ones currently checks these combinations:

const supportedPlatforms = [["macos","aarch64"],["linux","aarch64"],["macos","x86_64"],["windows","x86_64"],["linux","x86_64"]];

This is checked against Node's process.platform and process.arch respectively. Sadly, all of these are wrong... As a result, an error is thrown before any attempts at loading the suitable package are even made. First I thought maybe these were Deno or Bun specifics, but they all seem to match Node:

• Node process.platform docs
• Node process.arch docs
• Deno process.platform docs
• Deno process.arch docs
• Bun docs don't mention much, instead linking back to Node's docs ... but I assume platform and arch should stick to the spec

To address this:

• "windows" should be "win32"
• "macos" should be "darwin"
• "aarch64" should be "arm64"
• "x86_64" should be "x64"

2. The sqlite-vec package manually composes the path to vec0.* file

While this may work in classic setups where everything is unpacked into node_modules, in stricter setups like Plug'n'Play (Yarn, pnpm, Bun does something similar, ...), this is very likely to fail and compose a path to nowhere.

The correct way, I believe, would be to ask the module resolver itself to provide the path via APIs like import.meta.resolve or require.resolve, which should abstract away all of the unknowns.

3. The platform-specific packages declare non-existent exports

E.g. the sqlite-vec-windows-x64 package declares:

  "main": "./index.cjs",
  "module": "./index.mjs",
  "types": "./index.d.ts",
  "exports": {
    ".": {
      "require": "./index.cjs",
      "import": "./index.mjs",
      "types": "./index.d.ts"
    }
  }

...and none of these files actually exist within the published NPM package. What's worse, the vec0.* file is not declared anywhere, so resolvers refuse to see it. I believe the package should simply declare this:

  "exports": {
    "./vec0.dll": {
      "default": "./vec0.dll"
    }
  }

With the above change, calling import.meta.resolve("sqlite-vec-windows-x64/vec0.dll") (ESM) or require.resolve("sqlite-vec-windows-x64/vec0.dll") (CJS) cleanly resolves the path to the binary, wherever the package manager decided to put it.

---

After applying the mentioned changes via yarn patch I'm now up and running.

I noticed similar open issues here already, but with next to no information, so I decided to open one with some research. Hope this can help improve the NPM bundles, cheers!

[asg017]

Hey @CalmDownVal , thank you for the detailed report! Do you think you can try out v0.1.8-alpha.1 and let me know if it works?

Big thanks for that exports tip btw, didnt know you could point that to shared object files!

[CalmDownVal]

Hello @asg017 thank you for the lighting fast response and action! I can happily confirm that the alpha version does work flawlessly in my project now. Both in development where Yarn PnP interferes in resolution and in production builds where it's just raw Node with the usual unpacked node_modules dir alongside it.

I tested logging what getLoadablePath() comes up with and in both cases it resolves the exact correct paths, perfect!!

Tested on Windows (x64) and MacOS (arm64), both fine! I don't have a working linux box I could quickly test this on right now.

Btw. I think these issues may be related and could possibly be resolved with this fix:

• Error when running in nodejs #99
• cannot find compiled sqlite-vec (breaking change betseen 0.1.2-alpha.9 and 0.1.2-alpha.10) #107
• Node.JS Library path not correctly resolved in gitlab CI. #217
• Fix Deno bindings when there's no node_modules #188 ... possible but not sure

[asg017]

thanks for confirming and providing those related issues! i'll push a proper non-alpha release later today and close them out