Decoupling Components – Design

I have created a a number of non-trivial scripts in Python that do some long running reporting. At first these reports started off as scripts with a config file. Then I added in the a database component to persist data. Now I’m building a GUI to sit on top of all of them.

My question relates to design. Each time I add an additional requirement, I find myself wishing I had decoupled the components of these scripts more than I did. When I consider how to recfactor my code to be more modular, I find that I need to pass quite a few references around the code, to the point that class initializers and function signatures have a ton of parameters. Also I find myself having parameters that are simply “pass-through”, in that one object needs them only to pass to another object, doing nothing with them.

What is the best way to approach modularity in Python while minimizing the need to pass references all over the place?

I understand this is very conceptual and will vary by design, but I’m looking for guiding principles.

I don’t tend to run into this problem when I’m trying to modularize. So clearly I am doing something different, but without seeing what you’re doing, I can only guess what.

So I’ll limit myself to general platitudes.

First of all, when writing a number of related scripts, it is usually best to build small scripts and a common library. This type of design tends to survive multiple refactors better.

Second, if you have a lot of parameters to a function, use named parameters and sensible defaults. (See http://docs.python.org/release/1.5.1p1/tut/keywordArgs.html if you don’t know how to use named parameters.)

Third, there is nothing wrong with having methods in classes that just pass stuff through to another class. As long as the classes really serve different purposes, that is fine.

Fourth, there is everything wrong with having a ton of classes around simply because it serves some aesthetic purpose. For each and every class, unless you can explain exactly what having that class is doing to make your life easier, you can remove it.

Fifth, the book Code Complete has a tremendous amount of very detailed advice on everything from the proper naming of variables to effective modularization. If you have not worked your way through it, I highly recommend that exercise.

Sixth, if you can you really, really want a mentor that you trust to start reviewing your code. I would be willing to bet that what you most need to be taught is something you would never see (which is why you need help with it) which is fairly obvious to a more experienced person. Teaching someone without being able to see what they are doing is no more effective than driving a car without being able to see the road.

When you use inversion of control techniques such as dependency injection you can give classes only the components they need to do the job so they do not have to worry about what their dependents depend on. This will greatly reduce the number of parameters you are passing to functions, reduce the complexity of your scripts, and introduce loose coupling between components. To give a good example in psudo code, say we are building a car.

You could pass all the parts of a car into the constructor, but then it has to know about the piston, coil and spark plugs even though it doesn’t need to know about them. It only needs them to assemble the engine.

Car(Tires tires, Pistons piston, Coil coil)
{
    SparkPlugs sparkPlugs = new SparkPlugs(piston, coil)
    Engine = new Engine(sparkPlugs, tires)
    Tires = tires;
}

A better way to do it is to build the engine beforehand and pass it to the car. Then the car only needs to know about the parts it uses

Car(Engine e, Tires t)
{
    Engine = e;
    Tires = t;
}

That’s an interesting question, I have done exactly the same thing but in Ruby. Basically my reporting solution evolved like this:

A temporary script to show management what is going on. The script only ran as a console app, so I copy and pasted the resulting data into Excel and added some nice formatting and graphs.
The temporary solution became a weekly report and since vacation was near, I added an HTML output through a separate script. The commonly used functions were put into a common library file.
Some business software producing the input changed but the reports were popular. At that point I ported the report to Go and changed from a procedural/functional style to an Object Oriented style. (Go became the lingua franca at my job, you can do the same thing in Python as well)

I assume you are in my world at step 1. or 2. As you have seen yourself, modularization is needed. Some patterns I remember when doing that:

passing many parameters repeatedly to a function? Create a class that can carry all those parameters. Probably some functions could become methods of that class
go abstract: I did many grouping, counting and aggregation operations. I created classes (types) that perform those operations for many possible table formats. If you have such a transformation class, you can define transformation functions/lambdas (functional!) for the data types you need. So in a nutshell: remove all the business from the problem at just think of the data types and the transformations you need.

For the second point: as you are using Python, I think Numpy is a perfect tool for that job. It has some very powerful transformation tools for tabular data.

If each of these components uses some subset of all possible references, you might want to consider creating a “controller” object that has a handle to all the references. You would then pass the controller to the functions, and the functions call pull out the references that they need.

For example, instead of:

def foo(a,b,c):
    print "a:", a
    bar(b,c):

def bar(b,c):
    print "b:", b
    baz(c)

foo("this is a", "this is b", "this is c")

You would do:

class Controller(object):
    def __init__(a,b,c):
        self.a = a
        self.b = b
        self.c = c
    }

def foo(controller):
    print "a:", controller["a"]
    bar(controller)

def bar(controller):
    print "b:", controller["b"]
    baz(controller)

controller = Controller("this is a", "this is b", "this is c")
foo(controller)

You have correctly realized that modularization is key for a well maintained software. Without a specific problem I will have to limit it to a general answer.

Structure

Each module has one function. That being said there are a few classes that represent the external interface for the module and others that are only used internally. Those that serve as an interface need to get everything they require for an action via parameter. If the parameters are non-trivial types, you should use an interface for the parameter type. That way the class only knows about a certain public interface of the objects it receives but nothing about the specific class.

Some modules need other modules. There is a simple approach to this. You have tools, services, materials and values (see tools and materials approach). Tools can use every type of class. A service can only use other services, materials and values. A material can only use materials and values and finally a value can only use values.

The values represent immutable values in the subject matter. In the banking area this might be an amount of money. Though the amount itself might be variable (you can have 100, 200, x bucks), the value is not. You don’t change 100 to 200. You simply take the value representation of 200.
The materials represent things in the subject matter (like a bank account).

Services are used to change materials and operate subject matter operations (e.g. a lending service that is used to lend out movies). They also contain the business logic to determine if a certain operation is valid.
The tools are the interaction point with the user. They mostly work on one material or service. There can be sub-tools. The tools are the interface to the GUI (be it desktop based of web based).

That being said you can ensure that each module contains only one kind of classes and one responsibility.

Dependency Injection

But how does Dependency Injection now work without polluting other classes? It is allowed to use new statements within tools classes to create sub-tools. As tools are mostly technology oriented, it is not very easy to test it automatically (as they require other classes to do the heavy work). Important is that every dependency to another module (be it services, materials or values) is given via parameter.

In the end you need one class that ties the modules together, initializes the services needed in the software and gives them to the appropriate classes via parameter injection (be it in constructor or setter method).

The tools by the way get their needed object references (if not via parameter) from the GUI, so there is not much relaying of references through the system.

Filed under: softwareengineering - @ 20:27

Thẻ: design, python

Thiết kế website giá rẻ

Danh mục

Decoupling Components – Design

Structure

Dependency Injection