I am making a program that primarily deals with heavy data mining and analysis. It is primarily a back-end type algorithm, with no GUI elements or console windows seen as it runs. Occasionally though, it will need to graph something or get very special input from the user. However, I initially didn’t want to build the application integrated into the GUI as I feel that would be overly complicated. Instead I had wanted to just make my algorithm separate from any GUI elements, and if they were needed, then spawn them in, they do their job, and then exit; they are additions to the base program, not necessary for it to work.
So, the logical conclusion is to make my data processing happen in the main thread, and my GUI occupy another thread when necessary. Why do I need two threads? Because the GUI requires me to run an event loop so everything stays in check.
The specific graphing / GUI utility I have picked for this task is PyQt5, but that should be besides the point (most of them have the same basic mechanics).
It turns out though, that GUI elements do not like being run in a non-main thread (see here). It is possible, as you can see here, but the communication from the main thread and the secondary thread is nasty. I don’t have to pass information as much as I have to actually interact with the GUI object’s methods. I don’t have to do any actual GUI widget manipulation from another thread, just access non-GUI methods belonging to the GUI object.
This must be a common design, but I have no idea how to implement it. Keep in mind, when I say ‘data processing’ section, it is vastly more complicated than that. Thus, building a GUI around it, and having everything being done inside of signals and slots seems to make it a massive mess.
So here are the general questions. I am not looking for code, but rather general pointers.
-
How would you design a program like this?
-
What would each thread look like?
-
How do they interact in a way that doesn’t anger the GUI?
6
First things first. Modern OSs and even programming languages allow the manipulation of GUI only from the main thread. For many reasons this is an absolute must and in fact it is hardly an option.
Then, the concerns you are describing are being solved by developing applications under design patterns. There are many approaches here with the most popular the MVP, MVC and MVVM patterns. They all allow you to separate the model (your data fetching and manipulation) from what and how you present things to the user.
3
There are several approaches possible, and to get the GUI/calculation with multithreading right, there is be more necessary than a few paragraphs here on “Programmers” can teach your.
However, the first thing I would typically ask here is: do you really need multithreading? Since you are writing a program that primarily deals with heavy data mining and analysis, I guess there is database available. You data mining program may run in one process, without any GUI, and your GUI program in a separate process. The interprocess communication might be done by using the DB. That way, it might be possible to avoid any need for multithreading in the GUI at all.
That way, you can keep a strong separation between the processing part and the GUI part, your processing part won’t get intermixed with any signal/slot mechanics, and it is not even mandatory to use Python for the GUI part (but if you like it, you still can.)
2