Python Multiprocessing – is it creating two threads per process

This program starts two processes. It appears that there may actually be two threads created for each process. One, which is the mainline code of the spawned python program and the other being the execution of the function that was called to start the process. The program output shows that both the mainline of the program running in each process as well as running the called function specified when spawning the process.

The mainline of each process ends with the displaying of the process ID. Each of the two spawned tasks also terminate by displaying their process ID. The log shows the mainline messages be printed first, then the spawned function is called after that.


from multiprocessing import Process
import os
import psutil

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', psutil.Process().name())

if __name__ == '__main__':
    print ('nMain process ID:', os.getpid(), 'n')
    
    p = Process(target=f, name='process-1', args=('passed-name1',))
    p.start()
    p.join()
    print ('nAfter 1st join, PID:', os.getpid(),'n')

    p = Process(target=f, name='process-2', args=('passed-name2',))
    p.start()
    p.join()
    print ('nAfter 2 joins, PID:',os.getpid(),'n')
    
print ('End mainline code for PID',os.getpid())

I expected the message ‘Executed at task end’ to appear when the process completes its operation. But I find it executing BEFORE it runs the mainline code of the process.

Here is the output of this program:


Main process ID: 5528

End mainline code for PID: 42368

Starting process ID: 42368
I am process with argument:  passed-name1
parent:  5528
process name:  python.exe

After 1st join, PID: 5528

End mainline code for PID: 17764

Starting process ID: 17764
I am process with argument:  passed-name2
parent:  5528
process name:  python.exe

After 2 joins, PID: 5528

End mainline code for PID: 5528

Question 1: So does the Process() function both spawn the entire program as well as running the named function as a separate process?

Question 2: Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

Now here is another wrinkle is this multiprocessing dilemma. If you add the code

exit(0)

as the last line of the code, after the printed termination message, then the main line of the two spawned tasks do not execute at all. Here is the output of the code if the exit(0) is added as the last line:


Main process ID: 36996

End mainline code PID: 39976

After 1st join, PID: 36996

End mainline code PID: 28324

After 2 joins, PID: 36996

End mainline code PID: 36996

So the mainline code of each process ran, but the function called by the Process() function, did not.

I also noted that if instead of calling exit(0), I import time and call time.sleep(5) instead, the End of mainline message appears then the mainline code of the spawned task is executed after the 5 second delay. I would have expected the called function to be started immediately. This seems confusing as to executing what looks like two threads for the process. Or perhaps it executes the mainline followed by the called function after that. But important to understand as the code becomes more complex into mainline versus the function called in the Process() call.

So does the Process() function both spawn the entire program as well as running the named function as a separate process?

No, Process() function doesn’t spawn the entire program, instead, it creates separated process which executes only particular function. In your case – f(name).
But after multiprocessing.Process() a new process import code from module from which it was defined (__main__ in your case), that’s why seems like all entire code run again, but instead executes only code inside if __name__ == '__main__'.

Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

psutil.Process().name() prints name of executer file. (its python.exe for all Python processes). To print name of process you could use multiprocessing.current_process().name inside of your function:

from multiprocessing import Process, current_process
import os

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', current_process().name)

Details about exit(0) and exit(5)

When you calling exit(0) it cause immediate termination of a program and all processes of program, so f(name) doesn’t have enough time to run. But with time.sleep(5) it does have enough time. That’s why you could see output of f(name)

Filed under: Kiến thức lập trình - @ 09:45

Thẻ: pythonmultiprocessingpython-multiprocessing

Python Multiprocessing – is it creating two threads per process


from multiprocessing import Process
import os
import psutil

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', psutil.Process().name())

if __name__ == '__main__':
    print ('nMain process ID:', os.getpid(), 'n')
    
    p = Process(target=f, name='process-1', args=('passed-name1',))
    p.start()
    p.join()
    print ('nAfter 1st join, PID:', os.getpid(),'n')

    p = Process(target=f, name='process-2', args=('passed-name2',))
    p.start()
    p.join()
    print ('nAfter 2 joins, PID:',os.getpid(),'n')
    
print ('End mainline code for PID',os.getpid())

I expected the message ‘Executed at task end’ to appear when the process completes its operation. But I find it executing BEFORE it runs the mainline code of the process.

Here is the output of this program:


Main process ID: 5528

End mainline code for PID: 42368

Starting process ID: 42368
I am process with argument:  passed-name1
parent:  5528
process name:  python.exe

After 1st join, PID: 5528

End mainline code for PID: 17764

Starting process ID: 17764
I am process with argument:  passed-name2
parent:  5528
process name:  python.exe

After 2 joins, PID: 5528

End mainline code for PID: 5528

Question 1: So does the Process() function both spawn the entire program as well as running the named function as a separate process?

Question 2: Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

Now here is another wrinkle is this multiprocessing dilemma. If you add the code

exit(0)


Main process ID: 36996

End mainline code PID: 39976

After 1st join, PID: 36996

End mainline code PID: 28324

After 2 joins, PID: 36996

End mainline code PID: 36996

So the mainline code of each process ran, but the function called by the Process() function, did not.

So does the Process() function both spawn the entire program as well as running the named function as a separate process?

Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

from multiprocessing import Process, current_process
import os

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', current_process().name)

Details about exit(0) and exit(5)

Filed under: Kiến thức lập trình - @ 09:45

Thẻ: pythonmultiprocessingpython-multiprocessing

Python Multiprocessing – is it creating two threads per process


from multiprocessing import Process
import os
import psutil

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', psutil.Process().name())

if __name__ == '__main__':
    print ('nMain process ID:', os.getpid(), 'n')
    
    p = Process(target=f, name='process-1', args=('passed-name1',))
    p.start()
    p.join()
    print ('nAfter 1st join, PID:', os.getpid(),'n')

    p = Process(target=f, name='process-2', args=('passed-name2',))
    p.start()
    p.join()
    print ('nAfter 2 joins, PID:',os.getpid(),'n')
    
print ('End mainline code for PID',os.getpid())

I expected the message ‘Executed at task end’ to appear when the process completes its operation. But I find it executing BEFORE it runs the mainline code of the process.

Here is the output of this program:


Main process ID: 5528

End mainline code for PID: 42368

Starting process ID: 42368
I am process with argument:  passed-name1
parent:  5528
process name:  python.exe

After 1st join, PID: 5528

End mainline code for PID: 17764

Starting process ID: 17764
I am process with argument:  passed-name2
parent:  5528
process name:  python.exe

After 2 joins, PID: 5528

End mainline code for PID: 5528

Question 1: So does the Process() function both spawn the entire program as well as running the named function as a separate process?

Question 2: Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

Now here is another wrinkle is this multiprocessing dilemma. If you add the code

exit(0)


Main process ID: 36996

End mainline code PID: 39976

After 1st join, PID: 36996

End mainline code PID: 28324

After 2 joins, PID: 36996

End mainline code PID: 36996

So the mainline code of each process ran, but the function called by the Process() function, did not.

So does the Process() function both spawn the entire program as well as running the named function as a separate process?

Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

from multiprocessing import Process, current_process
import os

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', current_process().name)

Details about exit(0) and exit(5)

Filed under: Kiến thức lập trình - @ 09:45

Thẻ: pythonmultiprocessingpython-multiprocessing

Python Multiprocessing – is it creating two threads per process


from multiprocessing import Process
import os
import psutil

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', psutil.Process().name())

if __name__ == '__main__':
    print ('nMain process ID:', os.getpid(), 'n')
    
    p = Process(target=f, name='process-1', args=('passed-name1',))
    p.start()
    p.join()
    print ('nAfter 1st join, PID:', os.getpid(),'n')

    p = Process(target=f, name='process-2', args=('passed-name2',))
    p.start()
    p.join()
    print ('nAfter 2 joins, PID:',os.getpid(),'n')
    
print ('End mainline code for PID',os.getpid())

I expected the message ‘Executed at task end’ to appear when the process completes its operation. But I find it executing BEFORE it runs the mainline code of the process.

Here is the output of this program:


Main process ID: 5528

End mainline code for PID: 42368

Starting process ID: 42368
I am process with argument:  passed-name1
parent:  5528
process name:  python.exe

After 1st join, PID: 5528

End mainline code for PID: 17764

Starting process ID: 17764
I am process with argument:  passed-name2
parent:  5528
process name:  python.exe

After 2 joins, PID: 5528

End mainline code for PID: 5528

Question 1: So does the Process() function both spawn the entire program as well as running the named function as a separate process?

Question 2: Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

Now here is another wrinkle is this multiprocessing dilemma. If you add the code

exit(0)


Main process ID: 36996

End mainline code PID: 39976

After 1st join, PID: 36996

End mainline code PID: 28324

After 2 joins, PID: 36996

End mainline code PID: 36996

So the mainline code of each process ran, but the function called by the Process() function, did not.

So does the Process() function both spawn the entire program as well as running the named function as a separate process?

Why is the process named ‘python.exe’ when I specifically named each process with a name to call it?

from multiprocessing import Process, current_process
import os

def f(name):
    print('nStarting process ID:', os.getpid())
    print ('I am process with argument: ', name)
    print ('parent: ', os.getppid())
    print ('process name: ', current_process().name)

Details about exit(0) and exit(5)

Filed under: Kiến thức lập trình - @ 09:45

Thẻ: pythonmultiprocessingpython-multiprocessing

Thiết kế website giá rẻ

Danh mục

Python Multiprocessing – is it creating two threads per process

Details about exit(0) and exit(5)

Python Multiprocessing – is it creating two threads per process

Details about exit(0) and exit(5)

Python Multiprocessing – is it creating two threads per process

Details about exit(0) and exit(5)

Python Multiprocessing – is it creating two threads per process

Details about exit(0) and exit(5)