Issue using RefCell with nested rayon threads

[jdmarsh]

Hello, I am getting an "already borrowed" error when using nested rayon threads and a RefCell object. I have a repro here

What I don't understand is why the creation of a new independent thread pool, with threads that don't access the RefCell in any way, they just do a println, are causing an extra borrow to even be attempted? Is this a bug or am I missing something when creating this thread pool?

n.b. If I just use a regular thread, instead of a rayon thread, everything works as expected as seen here

[cuviper]

I believe this is yet another instance of work-stealing reentrancy like #592. Any time a rayon thread is blocked in a rayon call, even across thread pools, it will try to run other work while it is waiting.

So in this case, if you wait on rayon while holding that borrow_mut() ref, then you may work-steal another call of that par_iter that tries to borrow_mut again on the same thread-local.

[jdmarsh]

Is there a way around this? I'm developing a library where I want to use my own thread pool, but then that means no one can call my library from within a rayon thread.

[cuviper]

Generally, don't hold a non-reentrant resource when making a rayon call.

I've thought about various ways of disabling work-stealing during "critical sections" like this, but my fear is that it will become a different kind of footgun, where it would be easy to get all threads in the pool stuck waiting for "someone" to pick up pending work.