Python Multiprocessing Module Freeze_support() Line Can Be Obmitted Error Windows

I am trying to experiment with multiprocessing in python. I have initially written this test code on a Ubuntu system and it exectued fine. When I try to run the same thing in windo

Solution 1:

You most likely want to call one of the processes and then the other and don't do the p1.join() p2.join() because that will just make the main program halt until the subprocesses finish. This will work:

from multiprocessing import Process


defprocess1(text):
    returnprint(text)

defprocess2(text):
    returnprint(text)

defmain():

   p1 = Process(target=process1, args=('Test1', ))
    p2 = Process(target=process2, args=('Test2', ))
    p1.start()
    p2.start()

if __name__ == '__main__':
    main()

Solution 2:

You must set the multiprocessing Contexts and start methods

For my case, I had to utilize the context 'fork'

ctx = multiprocessing.get_context('fork')
work_queue    = ctx.Queue()
results_queue = ctx.Queue()
...
 workers = get_worker_processes(
    _process_data,
    (task_function, work_queue, results_queue),
    nproc=nproc,
)          
.....
workers = [
    ctx.Process(target=f, args=args) for _ in range(num_procs)
]   

Follow the guidance given in the Python multiprocessing documentation for the link referenced. Notice the change in defaults.

Contexts and start methods Depending on the platform, multiprocessing supports three ways to start a process. These start methods are

spawn The parent process starts a fresh python interpreter process. The child process will only inherit those resources necessary to run the process object’s run() method. In particular, unnecessary file descriptors and handles from the parent process will not be inherited. Starting a process using this method is rather slow compared to using fork or forkserver.

Available on Unix and Windows. The default on Windows and macOS.

fork The parent process uses os.fork() to fork the Python interpreter. The child process, when it begins, is effectively identical to the parent process. All resources of the parent are inherited by the child process. Note that safely forking a multithreaded process is problematic.

Available on Unix only. The default on Unix.

forkserver When the program starts and selects the forkserver start method, a server process is started. From then on, whenever a new process is needed, the parent process connects to the server and requests that it fork a new process. The fork server process is single threaded so it is safe for it to use os.fork(). No unnecessary resources are inherited.

Available on Unix platforms which support passing file descriptors over Unix pipes.

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

Changed in version 3.4: spawn added on all unix platforms, and forkserver added for some unix platforms. Child processes no longer inherit all of the parents inheritable handles on Windows.

I also found an attempt by some to resolve, and I think the resolution be better to inform the developer to set the context and start methods. Discussion found in issue at bugs.python.org/issue42949

The problem for me started on MacOS Python 3.8 when I updated my conda env and python was updated to 3.8.10.

envs/myenv/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main raise RuntimeError('''

some have had success with setting

if __name__ == '__main__': __spec__ = None