Multithreading Issue

[Pebaz]

It seems that when trying to use a Nimpy library from multiple Python threads, an error crashes the Python interpreter:

SIGSEGV: Illegal storage access. (Attempt to read from nil?)

This error originates from compiled Nim code. The information I have on it so far:

• If a Nim extension is imported in the global scope, it can only be used from the main thread
• If an extension has been imported into the global scope, you cannot use it from another Python thread
• If an extension is imported into a running function's scope, it will still be bound to sys.modules so when another thread tries to access it it will crash with the above error
• When using an extension in a non-main thread only, it will work (since it is only one thread)
• When using a Nim extension with Flask within a route handler, it will crash with the above error since each handler is executed in its own thread. Even handling subsequent requests after the initial one will crash even though it is on the same thread ID
• I tried duplicating a Nim extension .so per thread, creating a unique Spec and Loader object and it successfully imports, but for some reason it still crashes likely due to the underlying C extension exposing the same module name (PyInit_<mod name>)
• The only solution I can come up with is to compile a separate Nim extension with a unique name per thread that should be used. This is just straight up hacky, however. A better solution probably exists.

Use this Nim module as a reference for the next 2 Python examples:

# calc.nim

import nimpy

proc add(a, b: int): int {.exportpy.} =
    return a + b

Here is a code example that you can use the reproduce the problem:

import threading, nimporter

def foo():
    import calc
    print(calc.add(2, 20))

threading.Thread(target=foo).start()
threading.Thread(target=foo).start()

print('Here!')

Another example:

from flask import Flask
import nimporter

app = Flask(__name__)

@app.route('/')
def hello_world():
    # Importing here instead of in global scope works for 1 request
    import calc
    return 'Hello ' + str(calc.add(2, 20))

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Unfortunately, it should be noted that this issue is beyond my current (2020) skillset so any assistance is appreciated if it will contribute towards a resolution.

[Benjamin-Lee]

For reference, I have this solved using

# testing.nim
import nimpy

proc multiplier*(a: int, b: int): int {.exportpy.} =
    var x = 0
    echo a, " ",  b

    # noop to check CPU usage
    for i in 0..100000000000:
        x = x + a + b
        
    return a * b

Python code:

import multiprocessing
import testing


# square function
def square(x):
    return testing.multiplier(
        x,
        x,
    )


if __name__ == "__main__":

    # multiprocessing pool object
    pool = multiprocessing.Pool()

    # pool object with number of element
    pool = multiprocessing.Pool(processes=6)

    # input list
    inputs = [0, 1, 2, 3, 4]

    # map the function to the list and pass
    # function and input list as arguments
    outputs = pool.map(square, inputs)

    # Print input list
    print("Input: {}".format(inputs))

    # Print output list
    print("Output: {}".format(outputs))

Output:

$ python test.py
0 0
1 1
2 2
3 3
4 4
Input: [0, 1, 2, 3, 4]
Output: [0, 1, 4, 9, 16]

Proof it works:

[Iapetus-11]

Any update on this?

[Pebaz]

Hi @Iapetus-11, from what I understand, this doesn't affect Linux users and there's a workaround on MacOS listed above, but Windows still suffers from this I believe. Since most people will run Flask from Linux VMs or containers, hopefully this is not too big of an issue.

[Iapetus-11]

• The workaround is not a workaround, multiprocessing is very different than threading in Python. Multiprocessing spins up new Python interpreters whereas threading does not and code execution in threads is limited by the GIL.
  • Also, using multiprocessing with nimporter does actually appear to work on windows, probably because it's seperate Py interpreters. Test code and output here: https://gist.github.com/Iapetus-11/2ae39ef8d490546771d3550b0b79cbdc
• Also, if I change the usage of multiprocessing.Pool to something like ThreadPoolExecutor or if I even just use threading.Thread, when calling nim code, my program crashes with no output or any indication of error. I did notice that the errorlevel variable in command prompt is set to -1073741819 after the crash, if that helps at all.
• I have mostly moved to docker and mac but it's nice to write code which works everywhere.

Anyways, thank you for writing such a great library. If you need any help testing or need any additional information from my tests please let me know.

[Pebaz]

True that, yeah multiprocessing works since it's presumably using IPC with different processes so they each load their own version of the Nim extension dynamic library. I'm sure this has to do with executable memory pages on Windows but I'm not sure if it's something that needs to be set when Python is compiled or a Python CLI flag or even a Nim compile flag. I need to get with the Nimpy guy again and see if he has any ideas on how to proceed. 😁

[Pebaz]

@Iapetus-11, if you're interested, I've got an in-progress complete rewrite of Nimporter that I've needed someone to help test: https://github.com/Pebaz/nimporter/releases/tag/v2.0.0rc

The reason I haven't merged it so far is because it diverges from the current implementation of Nimporter v1 significantly but in return, it solidifies on the original idea of Nimporter which was to make Python + Nim projects dead-simple.

Basically, I'm looking for reasons why it wouldn't work. One example is that pure-Nim libraries are the easiest to implement but what about wrapping something like Raylib with Nim and exposing it to Python? Would that work given the new v2 semantics on all platforms? Also note that the platforms listed in v2 need to be added onto, I just wanted to cover Win32, MacOS, and Linux initially.

[Iapetus-11]

I need to get with the Nimpy guy again and see if he has any ideas on how to proceed. 😁

That'd be awesome

Basically, I'm looking for reasons why it wouldn't work. One example is that pure-Nim libraries are the easiest to implement but what about wrapping something like Raylib with Nim and exposing it to Python? Would that work given the new v2 semantics on all platforms? Also note that the platforms listed in v2 need to be added onto, I just wanted to cover Win32, MacOS, and Linux initially.

I'm not familiar with RayLib but I do have a project which uses nimpy's raw py buffers api and several other projects that use pure nim (and nimporter) I could test with later today.