All the documentation I have read mentioned that the Pool should be surrounded by a check that it’s in main. Otherwise, there is potential for an infinite loop.
What I see Online to Do:
if __name__ == "__main__": with Pool(processes=5) as pool: output = pool.starmap(test_func, list(tuples))
However, I am running the multiprocessing library in a sub-module (not ‘main‘) and not having any errors (note running it thru a Juypter-notebook). I do have an infinite loop when trying to run some integration tests I made with the unittest library.
How can I run the multiprocessing step in a sub-module?
How I am Currently Doing It:
with Pool(processes=5) as pool: output = pool.starmap(test_func, list(tuples))
Source: Python Questions