PGlite bindings for other runtimes?

[samkhawase]

I'm trying to run and access PGLite through a golang program via wasmer-go. However it fails with the error regardgin env not instantiated. I assume that there is no support for other runtimes (#40)

Is there a way I could contribute to adding support for more runtimes?

$ wazero run postgres.wasm                                                                                                                                                                                         
error instantiating wasm binary: module[env] not instantiated

$ wasmtime postgres.wasm                                                                                                                                                                                           
Error: failed to run main module `postgres.wasm`

Caused by:
    0: failed to instantiate "postgres.wasm"
    1: unknown import: `env::invoke_ii` has not been defined

[e253]

A .wasm file will declare a bunch of functions it needs to execute properly. WASI and WASIX are standard sets of such functions. A pure WASI/WASIX web assembly program can be run in a WASI/WASIX runtime like wasmer without changes. At the moment, PGlite ships a wasm binary with a JS file that provides many custom Emscripten non-standard functions that wasmer doesn't have.

// postgres.js (generated)
var wasmImports = {
//...
      emscripten_date_now: _emscripten_date_now,
      emscripten_dispatch_result: emscripten_dispatch_result,
      emscripten_get_heap_max: _emscripten_get_heap_max,
      emscripten_get_now: _emscripten_get_now,
      emscripten_get_now_res: _emscripten_get_now_res,
      emscripten_memcpy_js: _emscripten_memcpy_js,
      emscripten_resize_heap: _emscripten_resize_heap,
      environ_get: _environ_get,
      environ_sizes_get: _environ_sizes_get,
      exit: _exit,
      fd_close: _fd_close,
      fd_fdstat_get: _fd_fdstat_get,
      fd_pread: _fd_pread,
      fd_pwrite: _fd_pwrite,
      fd_read: _fd_read,
      fd_seek: _fd_seek,
      fd_sync: _fd_sync,
      fd_write: _fd_write,
// ...
}

A WASIX (note the x, not WASI) build of pglite shouldn't be too tough because it has a very broad API. You link the wasix libc and that libc will compile imports in the .wasm binary that wasmer understands. You could even compile totally vanilla postgres to wasm32-wasix-none. I'd love to use a wasix pg instead of bloated, ram hungry docker pg! https://wasix.org/docs/language-guide/c/usage.

WASI should be possible through Emscripten by using the -sWASM_STANDALONE=1 linker flag. It can work, but will take some finagling to pass tests.

[sohaha]

Similarly have such a demand.

[ggnery]

invoke_* are js functions used by javascript to support longjmp and exceptions. To remove the try compiling and linking with

-s SUPPORT_LONGJMP=0 and -s DISABLE_EXCEPTION_CATCHING=1

try reading this for more information: https://stackoverflow.com/questions/45511465/purpose-of-invoke-functions-generated-by-emscripten

[pmp-p]

if golang has a wasi runtime you could try that experimental pglite wasi build embededd in python, sorrry can't help for the wrapper part i don't speak Go but since it uses only some files it should not be hard to translate from python

pglite-embed-python3-poc.zip

[sgosiaco]

@pmp-p Thanks for the experimental wasi build. With it I was able to get a REPL setup in Go using wazero as the runtime.

Sample code available here https://github.com/sgosiaco/pglite-go

Hopefully this helps other people out who were looking to do something similar

[morlay]

Hi @sgosiaco.
I played with your demo. but I cannot find a way to get query results in golang.
Do you have some suggestion to figure it out?

[tw1nk]

@morlay I've recreated the @sgosiaco experiment here: https://github.com/tw1nk/go-pglite

From what my tests the pglite-wasi binary from the zip file seems to be extremely limited in functionality basically only support the tests and some REPL functionality. No query results are returned from the pglite-wasi binary it only shows up as debug output.

as you can see in my code-base I have tried adding network support to be able to connect a regular Postgres client (or library) however that doesn't seem to work, the pglite-wasi binary complains about invalid data if I use the interactive_write method. Otherwise I don't get anything.

I might have completely misunderstood something in how to hook this up though. So if @pmp-p could give any hints on what I'm doing wrong, Or maybe some information on how to build the wasi-merge branch to get maybe a newer version of the pglite-wasi binary I would be very happy

[morlay]

the pglite-wasi binary complains about invalid data if I use the interactive_write method

I got the same issue, when trying to wrap the pglite-wasi as a sql driver.

same logic from https://github.com/electric-sql/pglite/blob/main/packages/pglite/src/pglite.ts#L586
but seems pglite-wasi not have same behaviers as pglite-wasm

[dkegel-fastly]

I found this before I found https://github.com/electric-sql/pglite-bindings ... which is where I suspect that draft zipped build came from.