Unpickleable exceptions inside a setup_context will result in out of memory errors #369
Description
Originally reported by: itziakos (Bitbucket: itziakos, GitHub: itziakos)
The problem is demonstrated by the following example code
from setuptools.sandbox import setup_context
from pkg_resources import DistributionNotFound
with setup_context(' '):
raise DistributionNotFound('Memory more memory')
Inside the setup_context context manager, if an exception takes place, is captured and saved in pickled format using UnpickleableException.dumps classmethod. The dumps classmethod will first try to pickle the exception. In the example above the pickling attempt will fail because inside the setup_context the DistributionNotFound is not pickleable (note that the call to hide_modules will have removed the pkg_resources module from sys.modules). On a failed picking attempt the dumps method will wrap the DistributionNotFound inside an UnpickleableException and try again calling the dumps classmethod with the new exception. Unfortunately inside the setup_context an UnpickleableException is also unpickleable because setuptools is not in sys.modules. From this point the dumps classmethod will be called recursively trying to pickle the UnpickleableException, until the system runs out of memory.
I was able to replicate the memory problems using both python 2.7 and 3.4