Write progress from a long running process

We have some long running processes which could run for as long as an hour up to 6 hours.

Some of the processes are started by hand and the user wants to see what the progress of the process is.

Without getting into to much details of the actual details, you could think of:

Fetching data
Update client info
remove false data
create external files
create pdf

This process of course consists of loads of different classes, and sometimes I want those individual classes to write progress.

For example: When I am only interested in high overview progress, I could write the progress from the top level facade, but I also want to write progress in the service which creates the pdf’s, so that class and it’s subclasses also have to write progress.

My question is: how can I write progress from all those classes regarding the same ‘base’ process without passing some kind of processid from class to class?

What I thought of is creating a loggingservice class which takes care of the logging and passing that from class to class, but that seems so nasty to hand every single class the logging service.

Second I thought of is to create a static loggingservice class for the logging (which I find common in applications for the normal error logging) but then I still have to pass the processId around

Third I thought of creating a factory to be called from every class, but then I still have to pass the processId around

Fourth I thought of injecting the loggingservice class in every class, but then I still have to pass the processId around and it seems like a wrong kind of dependency for a class to have, a dependency for a logger.

So I am looking for either number 5, or reasons why 1,2,3 or 4 is a good idea anyway and the downsides I see are not relevant or important.

I usually do this by letting every long-running class implement an interface, e.g. LongRunning, with a single method reportProgress(). Your logging service would simply be notified of the existence of such long runners, periodically query their status, and then report on the general state of the world in whatever way it sees fit.

Your business code still has to be interleaved with logging statements that way, but otherwise your existing classes don’t have to adapt to anything external, and that’s the most important thing in my view: not changing business code that already does its job for extraneous reasons.

You are correct that all of these solutions dirty your code with logging / reporting code that breaks separation of concerns. This sort of problem is exactly why Aspect Oriented Programming exists:

Wikipedia (emphasis mine):

Aspect-oriented programming entails breaking down program logic into distinct parts (so-called concerns, cohesive areas of functionality). Nearly all programming paradigms support some level of grouping and encapsulation of concerns into separate, independent entities by providing abstractions (e.g., functions, procedures, modules, classes, methods) that can be used for implementing, abstracting and composing these concerns. Some concerns “cut across” multiple abstractions in a program, and defy these forms of implementation. These concerns are called cross-cutting concerns.

Also relevant: Cross-cutting concern

Essentially, you want something that can touch upon all aspects of your application in a particular scenario. In your case, such logging is a cross-cutting concern.

I can’t help you with any implementation details as your question doesn’t mention a language or a platform. But many languages offer Aspect Oriented Programming natively, or there are frameworks that assist with this process.

Examples:

Java: Guice AOP
Python: Pytilities
Ruby: Aquarium
PHP: Go!

Instead of thinking of logging (i.e. your application telling some logging system, perhaps using syslog(3) on Linux, what is going on) you might think of it differently: your application could be a server for requests querying its state.

Then you might even make your app some JSONRPC server (and you’ll have also to code the tiny command-line client to query it).

You could also have your app becoming a specialized web server (e.g. using some HTTP server library like libonion, etc… see this list) and be queriable thru usual browser technologies.

These days, for an application which can run an entire working day, I would enable some web server technologies inside it. Even a cheap printer is today able to give its status thru some web interface.

Your internal classes might give their state e.g. by updating some “global” variables or application wide data, etc… (Of course use mutexes for synchronization issues).

The aspect-oriented perspective from durron597’s answer is still relevant. If your application is a big C or C++ code compiled by GCC, you might consider using MELT to customize the compiler for such (aspect-oriented) purposes. But that would be compiler specific, would take a week of effort, etc.

BTW, I am not sure that an externally queriable state should be fine-grained. I believe that updating that state every dozen of seconds is enough (I’m not thinking of periodical upgrade, but of coarse-grained state). Then you probably don’t have to touch a big lot of code. YMMV.

Whatever you do, you have to design it quite early. The wikipages on continuations, application checkpointing, persistence might be relevant.

I know this is an old post but anyways … to address the concerns of ‘dirtying’ code in the accepted answer, consider the visitor pattern.

Update reportProgress() method to take a visitor param which contains the code specific to reporting for each concrete instance, then the concrete instance calls the appropriate visitor method.

This way you can keep your business logic code separate from your logging / reporting code. You’d also be able to provide different logging implementations as different visitors.

Filed under: softwareengineering - @ 17:28

Thẻ: logging, monitoring

Thiết kế website giá rẻ

Danh mục

Write progress from a long running process