WebAssembly and Web Worker Experiments/Evaluation

[robertsoakes]

Last year, I had a brief conversation via IRC with Joel about optimizing portions of the code base by re-implementing some of the code in ASM.js and making use of web workers. It's been a while, but I'm interested in re-engaging that conversation.

The company I work for (Guru Labs) uses noVNC in a training/collaboration application and it generally works very well. We have one view, though, where it's possible that 40 or 50 simultaneous sessions might be displayed at once. If all of the machines on the page become active at once, though, it can get really laggy and we've started to get some customer complaints about the interface being "slow".

Seems like a good time to move forward on those ideas, and I've got an engineer who is interested in taking this on as a project.

As a first step, I would like to re-implement some of the routines as a drop-in ASM.js module and then compare how much of a speedup we get. From there, the next step might be to start looking methods for putting that processing in a web worker.

Which I guess brings me to the point: do you have any feedback on where we might start to experiment? We can always spend quality time with the profiler, but having some candidates on portions of the code that might stand for a rework using ASM.js would be greatly appreciated.

[kanaka]

Profiling is never a bad idea. You're definitely going to want to get tests/vnc_perf.html working again. @DirectXMan12 updated tests/vnc_playback.html to get it working and the changes to get tests/vnc_perf.html to work too are probably pretty similar. The tests/vnc_perf.html page is similar but provides quite a bit more detail timing information (per encoding for one thing). It's super useful for performance related work. Both vnc_playback.html and vnc_perf.html playback RFB recordings made with websockify. vnc_playback is more oriented towards demo and vnc_perf towards testing.

In terms areas of code to target for asm.js conversion. The two places that come to mind as probably having the biggest real-world impact are all of include/jsunzip.js and display_tight in include/rfb.js. In addition to likely being in the hot path for most users of noVNC, those pieces of code have the additional benefit of being highly numeric and likely to hugely benefit from asm.js optimizations.

Regarding Web Workers. Now that firefox 38 support websockets in web workers, this is definitely the time to begin exploring that. At a high-level, I think much of the functionality in include/rfb.js, include/websock.js, and the routines in include/display.js that call createImageData and putImageData should run within a web worker. You'll definitely want to look into structured cloning and transferable objects (both forms of zero copy designed for web workers but with varying browser support) for getting the imageData from the web worker to the main thread for display (see https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers#Transferring_data_to_and_from_workers_further_details)

One thing to explore is how double-buffering should work in this context. @samhed has been discussing adding double-buffering support to noVNC:#518 If we add double buffering support it's going to be worth doing some performance comparisons to see whether it's more efficient to:

• keep the second buffer (an imageData array the size of the display context) in the web worker and send it out to the main thread whole (using structured cloning to send it) which displays it into the visible canvas all at once.
• have the second buffer on the main thread (either an imageData array the size of the display context or a hidden canvas, not sure which is more efficient), and then have the worker send the smaller updates to the main thread which copies them into the second buffer and then displays the second buffer contents all at once.

I would definitely focus on the asm.js work first as I think that may give the biggest bang for the buck across the board (single noVNC and multiple noVNC instances on a single page). Also, asm.js will be less invasive to the current modular structure of noVNC. Supporting web workers will likely require some restructuring.

[DirectXMan12]

[ Whoops, @kanaka beat me to replying as I was typing this up. Anyway...]

Generally, the parts that need to be fast are the various encoding handlers -- check out the TIGHT encoding handlers, for instance. The performance of ASM.js, when I last checked several months ago, was very dependent on the browser -- it was faster in Firefox, but slower in Chrome, for instance. That's something to consider.

For web workers, you've got a few options -- you could put the encoding handlers in one worker, the websocket receive logic in another, etc. You could simple stick most of noVNC in a worker, and then send updates to the main page for drawing, which should offer improvements if you have multiple noVNC instances on the same page. Finally, you might be able to have a thread-pool-like setup for encoding handlers, but you'd have to add logic to avoid overlapping areas from being reordered, and you'd have to check whether or not the performance increase from parallelization justified any delay due to communication between workers.

[kanaka]

I've not done much direct experimentation myself, however, from what I've been reading, asm.js provides a significant performance boost on the last versions Firefox, Chrome, Safari, and Edge. However, currently I think it's only Firefox and Edge that do full AOT compilation on asm.js so they get the biggest boost by a fair margin. However, at least with Chrome, it does look for "asm.js" hinted code and enables TurboFan mode for that code. In fact, it might be worth looking at the Octance 2.0 benchmark code because it has an asm.js zlib benchmark.

Links:

• https://hacks.mozilla.org/2015/03/asm-speedups-everywhere/
• https://hacks.mozilla.org/2014/05/asm-js-performance-improvements-in-the-latest-version-of-firefox-make-games-fly/
• http://blog.chromium.org/2015/07/revving-up-javascript-performance-with.html

For the zlib piece, we might want to consider using emscripten to cross-compile the C zlib library. That's what the Octane benchmark does. See the comment at the top of https://github.com/chromium/octane/blob/master/zlib-data.js

[DirectXMan12]

http://blog.chromium.org/2015/07/revving-up-javascript-performance-with.html

Huh, that's new. Sweet.

For the zlib piece, we might want to consider using emscripten to cross-compile the C zlib library. That's what the Octane benchmark does. See the comment at the top of https://github.com/chromium/octane/blob/master/zlib-data.js

Just be aware that we need support for the streaming input/immediate output functionality that zlib provides -- the other emscripten-compiled zlib implementations that I found couldn't cope with that properly.

Our new current one (pako) is actually pretty good -- it looks like it's roughly to a by-hand translation of the C code in a C like manner, with some of the optimizations.

[kanaka]

@DirectXMan12 yep, I think now is definitely the time for exploring asm.js. It's working well across the leading edge of browser releases. From things I've read, it usually has a positive impact even on older browser versions that don't have direct support because it usually means less chance of the code hitting the de-optimizer and it tends to reduce GC because it forces you to use pre-allocated binary arrays for entire functions. There may be some ways of using it that result in worse performance. E.g here is one example where using "use asm" results in worse perf. So I asked Alon Zakai about it to see if he has any comment on that. Regardless, we'll definitely want to test and compare. If we find ways that asm.js results in worse perf (even though it AOT compiles properly), that would still be really valuable information for the asm.js community. Worth at least one blog post :-)

[robertsoakes]

Thank you everyone for your feedback. This is really helpful in deciding where to go first.

As recommended by Joel, I think we'll start working with asm.js and try and get vnc_perf.html working. I'll have some time later next week to spend on that and Dax (who has expressed interest in also working on this) should be able to start in a week or so. When I've got vnc_perf.html working, I will submit a pull request.

From there, I'll start looking at ASM.js enhancements for jsunzip and display_tight. I've got an ASM.js toolchain working, and should be ready to get started. It seems like jsunzip is pretty self-contained, and drop-in replacement shouldn't be too bad. I'll probably move to that next.

(Side question: emscpripten is a complete PITA on Linux, which is what I use for development. Everything has to be compiled from source. Is anyone aware of a precompiled binary for anything but Windows? Or at least a precompiled clang that is compatible?)

Any ideas on how you would want to structure the ASM.js code? Because of emscripten, it may make sense to create a separate repo with C/C++ tests and include that into noVNC as a submodule.

[kanaka]

@robertsoakes it probably makes sense to compile the zip library with emscripten, but display routines should probably just be ported to asm.js manually. They are fairly small and they are fairly specific to noVNC/RFB so it probably makes sense to hand code them.

In terms of pre-built clang for Linux, you might consider using a docker container with things pre-built. For example, https://hub.docker.com/r/rsmmr/clang/ (github repo here: https://github.com/rsmmr/install-clang) has clang 3.5 pre-built in a container. You could then volume mount the noVNC directory (probably using -u to change to your current user ID rather than root). Something like this run from your novnc repository (where 1000 is your user ID):

docker run --rm -it -u 1000 -v `pwd`:/novnc -w /novnc rsmmr/clang bash

[DirectXMan12]

@robertsoakes : note that we don't currently use jsunzip.js any more -- we use inflator.js instead, which is based on Pako (just so you don't go trying to actually optimize jsunzip itself).

[kanaka]

After some WebAssembly discussion here #1108 (comment) I had a long plane flight back from Tokyo last week and decided to do a little bit of WebAssembly evaluation. I've pushed a branch to https://github.com/novnc/noVNC/tree/wasm with the results of some experimentation using a rust to wasm pipeline (see the README.md in that directory for build/test process). Here are some high level takeaways:

• Build with release for testing performance. debug builds (default) are 10x slower.
• I initially had a function on the rust size that took a reference to a canvas 2D context, created a Uint8ClampedArray which it updated and then called a wrapped version of putImageData on the JS side with that data. This was pretty slow. Based on some examples found online I switched to calling a rust memory allocation function from JS, passing a reference to that memory into the rust/wasm code, doing the updates directly on it, and then doing the putImageData call on that memory from the JS side. This was quite fast.
• I have two test systems: an desktop with 8 3.2 GHz i7 960 processors and Radeon HD 7770 and a laptop with 8 2.7 GHz i7-6820HQ processors and Intel HD 530 both running Chrome 68. The following table shows some performance measures for both systems at different canvas sizes. This includes calculating a unique color value on the Rust/Wasm side for all 2 million pixels based on the the x/y coordinate plus frame number (i.e. not trivially optimizable), and then doing a putImageData for the full canvas. The two last columns show an average amount of time spent in the rust/wasm code vs the putImageData time on the JS side.

System	Canvas	FPS	rust/wasm	putImageData
i7 960	1024x768	60	2ms	1ms
i7 960	2048x1024	60	7ms	2ms
i7 960	2048x2048	46	13ms	9ms
i7 960	3840x2160	28	24ms	13ms
i7-6820HQ	1024x768	60	3ms	1ms
i7-6820HQ	2048x1024	60	5ms	8ms
i7-6820HQ	2048x2048	53	10ms	11ms
i7-6820HQ	3840x2160	36	18ms	13ms

• Note that the above numbers are for recalculating and redrawing every pixel of the canvas. In practice, the frame buffer updates will only recalculate a percentage of the pixels. However, I suspect that doing putImageData for the full canvas will still be the most optimal approach because there is no typed array to typed array blit operation. The closest thing is the 2d context drawImage, but that requires a full canvas or image as the source which can't be shared with webassembly (at least not yet).
• Interesting to note that the laptop (newer processor but slower clock speed) appears to do better with the rust/wasm code whereas the desktop (older/faster processor and dedicated graphics card) seems to do a touch better with putImageData (at least at the smaller sizes).
• As @DirectXMan12 noted in the comments from the other issue, in order to gain full performance we will need to the memory for both the receive queue and the image data to be shared rather than copied to get full performance advantage with WebAssembly. My next experiments are to try and integrate this in rough form into novnc so that the websock receive queue (rQ) is shared between JS and webassembly and implement RAW/COPYRECT decode/rendering on the wasm side (disabling other encodings).

I think this ticket is as good as any to track experiments and exploration for WebAssembly so I've renamed the issue from "ASM.js optimizations + web worker support" to "WebAssembly and Web Worker Experiments/Evaluation". After the Spectre/Meltdown issue, ShareArrayBuffers were disabled by all the major vendors. Browser vendors are now either beginning to re-enable (Chrome 68) or discussing re-enabling SharedArrayBuffersWeb. Using Web Workers won't really be viable until SABs are enabled broadly again.

[DirectXMan12]

@kanaka I'm curious here as to the actual performance impact that we'd be looking at in real-world noVNC -- my profiling the other day implied that we're not actually spending a ton of time in non-js code, but I'd like to see some other recordings to profile. We should probably build up a library of recordings for the purpose of performance profiling.

Reading one of your comments gave me some ideas for how to tweak some of my own experiments, though, so I might have some more improvements to my own experiment soonish