I would like to use multiprocessing in my jupyter notebook and AFAIK, the multiprocess package is the way to to that. My function does quite a lot of stuff, so it calls multiple subfunctions, but this throws a NameError for the name of the subfunction, unless the subfunctions are defined in the function beein called.
import multiprocess as mp
def test_int():
return 1
#This throws import multiprocess as mp
def test_int():
return 1
#This throws "NameError: name 'test_int' is not defined"
def test():
return test_int()
#This works
def test2():
def test2_int():
return 2
return test2_int()
if __name__ == '__main__':
with mp.Pool(20) as pool:
for _ in range(20):
res = pool.apply_async(test, [])
print(res.get())
As the functions are defined before the process is started and even before the called function is defined, I don't see a reason why this wouldn't work, but it doesn't.
The second varient works for now, but gets messy fast with more and bigger functions, so is there a way to make this cleaner?
Linux Mint. But I didn't test it injupyter-notebookjupyter-notebookand it works for me too.