Proposal: Portable bytecode backend and compiler as a library

[0syf3r]

Since Zig is a pretty lightweight language and the compiler will support a diverse range of backends, I think it's time that we considered offering the compiler as a separate library that doesn't have to deal with extraneuos thing like file handling or caching.

So what's the use of such a separation? Well, this leads to my second proposal: a portable bytecode backend.

I propose that Zig support outputting bytecode for custom or just a extremely simple virtual machine suitable for embedding and running on top of another programs, kind of like WASM or Lua bytecode. This wll allow Zig to be used both as a compiled language AND a scripting language for the same project with no performance cost and maximum code sharing. Imagine running your Zig code as scripts for hot reloading and rapid prototyping and then choosing to compile it for release. The amount of Zig code being compiled and running as scripts can be mixed and matched in any proportation with no code duplication, something that’s impossible with the traditional way like Zig+Lua.

With both these proprosals code like this will become reality:

Compiler = @import("zig-compiler")
file = load("user-script.zig")
output = ...
Compiler.compile(file, &output, {.backend=StandardVirtualMachine});
//Now 'ouput' can be run with any implementation of a runtime for SimpleVirtualMachine.
//The Zig project doesn't even have to provide one and the users can even use a custom one
//for high-performance or memory-constrainted environments.
DebugRuntime runtime = ...
runtime.setStackSize(...)
runtime.setDynamicAllocator(...)
runtime.setPanicCallback(...)
runtime.run(output);

Finally, this is not something that is radical or alien to the Zig project since backends as diverse as calculators, old consoles, and WASM, and gpus are planned, with WASM and SPIR V(#2683 ) basically implementing all the features of this proposal anyway. This just makes it so that WASM and SPIR V are not special and allows anyone to write runtimes for Zig code.

Once this proposal is implemented then Zig will probably become the best language for video game development since scripting can be added at no cost. And as a bonus, we can choose make something like zig.js that can run Zig code without relaying on asm.js or WASM support.

Oh did I mention that REPL and !#/usr/bin/zig a la Python and Nim will become trivial?

[0syf3r]

Let me just emphasize this: The WASM and SPIR V(#2683 ) backends will basically do all the work anyway, this just allows us to target VMs other than browsers. The rest of the proposal basically comes for free then.

WASM Backend:
zig -> wasm bytecode -> browsers
SPIR V Backend:
zig -> spir v bytecode -> graphic cards

In both cases:
zig -> vm bytecode -> vm

This proposal:
zig ----> wasm, spir v, zig standard bytecode(maybe more than one) ----> User-provided VM for each specific use cases (high performance, low memory footprint, debugging, game scripting, sandboxing, embedded devices, terminal scripting, etc)

(As an aside, we could even generate WASM and SPIR V bytecode through a translator:
zig -> zig standard bytecode -> bytecode translator -> wasm, spir v
But I have no idea if this is possible, so we can ignore this part is entirely optional).

[0syf3r]

This will also make testing edge cases easier, like we can test on a runtime with a small stack size or an unorthodox memory model.

[RossComputerGuy]

Hot reload would be great to have for my Flutter-like GUI toolkit. Would love to see this.