I'm pretty sure that the meaning of this is that Python doesn't handle this stuff well - dealing with it is a pain. I would love Python to the end of time if it just had multithreading that worked.
I've always loved robotics so I focused on learning that. I've worked on destkop applications, on drones, and now on exoskeletons! Web dev looks scary to me but there is a lot of potential there.
Location
France
Education
Master of Engineering
Work
Critical Embedded Software engineer at Wandercraft
Meaning that when you launch multiple threads in the same python interpreter they are actually locked on one core and can't be really parallelized. For some applications it's not a real issue, when your threads spend most of the time sleeping/waiting for IO for example. But when you need real parallel computing on multiple cores you can't do it with threads.
Python provides the "multiprocessing" module to workaround this, launching code in separate processes. It makes it a bit harder but not impossible to do real multi core parallelization.
Thanks for your explanation! So if I'm understanding correctly the only real way to do parallel processing is to have multiple python processes separate from each other. But that means the data used by each process is sort of isolated, right?
I've always loved robotics so I focused on learning that. I've worked on destkop applications, on drones, and now on exoskeletons! Web dev looks scary to me but there is a lot of potential there.
Location
France
Education
Master of Engineering
Work
Critical Embedded Software engineer at Wandercraft
Yes, the multiprocessing module provides some tools to exchange this data: docs.python.org/2/library/multipro... but your data needs to be easily serializable which is not always the case.
@antontsvil
, I personally think the article is a bit misleading becuase it mixes different details under umbrella terms but @loki
already explained what's going on in Python :)
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Great write up! Could you elaborate on what you mean by Python not having "true" multiprocessor support?
I'm pretty sure that the meaning of this is that Python doesn't handle this stuff well - dealing with it is a pain. I would love Python to the end of time if it just had multithreading that worked.
Python has what is called the "Global Interpreter Lock", you can read more here: docs.python.org/3/c-api/init.html#...
Meaning that when you launch multiple threads in the same python interpreter they are actually locked on one core and can't be really parallelized. For some applications it's not a real issue, when your threads spend most of the time sleeping/waiting for IO for example. But when you need real parallel computing on multiple cores you can't do it with threads.
Python provides the "multiprocessing" module to workaround this, launching code in separate processes. It makes it a bit harder but not impossible to do real multi core parallelization.
Thanks for your explanation! So if I'm understanding correctly the only real way to do parallel processing is to have multiple python processes separate from each other. But that means the data used by each process is sort of isolated, right?
Yes, the multiprocessing module provides some tools to exchange this data: docs.python.org/2/library/multipro... but your data needs to be easily serializable which is not always the case.
@antontsvil , I personally think the article is a bit misleading becuase it mixes different details under umbrella terms but @loki already explained what's going on in Python :)