Caching factory design

I have a factory class XFactory that creates objects of class X. Instances of X are very large, so the main purpose of the factory is to cache them, as transparently to the client code as possible. Objects of class X are immutable, so the following code seems reasonable:

# module xfactory.py
import x
class XFactory:
  _registry = {}

  def get_x(self, arg1, arg2, use_cache = True):
    if use_cache:
      hash_id = hash((arg1, arg2))
      if hash_id in _registry:
        return _registry[hash_id]
    obj = x.X(arg1, arg2)
    _registry[hash_id] = obj
    return obj

# module x.py
class X:
  # ...

Is it a good pattern? (I know it’s not the actual Factory Pattern.) Is there anything I should change?

Now, I find that sometimes I want to cache X objects to disk. I’ll use pickle for that purpose, and store as values in the _registry the filenames of the pickled objects instead of references to the objects. Of course, _registry itself would have to be stored persistently (perhaps in a pickle file of its own, in a text file, in a database, or simply by giving pickle files the filenames that contain hash_id).

Except now the validity of the cached object depends not only on the parameters passed to get_x(), but also on the version of the code that created these objects.

Strictly speaking, even a memory-cached object could become invalid if someone modifies x.py or any of its dependencies, and reloads it while the program is running. So far I ignored this danger since it seems unlikely for my application. But I certainly cannot ignore it when my objects are cached to persistent storage.

What can I do? I suppose I could make the hash_id more robust by calculating hash of a tuple that contains arguments arg1 and arg2, as well as the filename and last modified date for x.py and every module and data file that it (recursively) depends on. To help delete cache files that won’t ever be useful again, I’d add to the _registry the unhashed representation of the modified dates for each record.

But even this solution isn’t 100% safe since theoretically someone might load a module dynamically, and I wouldn’t know about it from statically analyzing the source code. If I go all out and assume every file in the project is a dependency, the mechanism will still break if some module grabs data from an external website, etc.).

In addition, the frequency of changes in x.py and its dependencies is quite high, leading to heavy cache invalidation.

Thus, I figured I might as well give up some safety, and only invalidate the cache only when there is an obvious mismatch. This means that class X would have a class-level cache validation identifier that should be changed whenever the developer believes a change happened that should invalidate the cache. (With multiple developers, a separate invalidation identifier is required for each.) This identifier is hashed along with arg1 and arg2 and becomes part of the hash keys stored in _registry.

Since developers may forget to update the validation identifier or not realize that they invalidated existing cache, it would seem better to add another validation mechanism: class X can have a method that returns all the known “traits” of X. For instance, if X is a table, I might add the names of all the columns. The hash calculation will include the traits as well.

I can write this code, but I am afraid that I’m missing something important; and I’m also wondering if perhaps there’s a framework or package that can do all of this stuff already. Ideally, I’d like to combine in-memory and disk-based caching.

EDIT:

It may seem that my needs can be served well by a pool pattern. On further investigation, however, it’s not the case. I thought I’d list the differences:

Can an object be used by multiple clients?
- Pool: No, each object needs to be checked out and then checked in when no longer needed. The precise mechanism may be complicated.
- XFactory: Yes. Objects are immutable, and can be used by infinitely many clients at once. There’s never a need to create a second copy of the same object.
Does pool size need to be controlled?
- Pool: Often, yes. If so, the strategy to do so may be quite complicated.
- XFactory: No. An object must be delivered on demand to the client, and if an existing object is unsuitable, a new one needs to be created.
Are all objects freely substitutable?
- Pool: Yes, the objects are typically freely substitutable (or if not, it’s trivial to check which object the client needs).
- XFactory: Absolutely not, and it’s very hard to find out if a given object can service a given client request. It depends on whether an existing object is available that was created with (a) the same arguments and (b) the same version of the source code. Part (b) cannot be verified by XFactory, so it asks the client to help. The client fulfills this responsibility in two ways. First, the client may increment any of its several designated internal version counters (one per developer). This cannot happen in runtime, only a developer may change these counters when he believes that the source code change makes existing objects unusable. Second, a client will return some invariants about the objects that it needs, and XFactory will verify that these invariants are not violated before serving the object to the client. If any of these checks fail, XFactory will create and deliver a new object.
Does performance impact need careful analysis?
- Pool: Yes, in some cases a pool actually hurts performance if the overhead of object management is greater than the overhead of object creation/destruction.
- XFactory: No. The computation costs of the objects in question is known to be very high, and loading them from memory or from disk is without doubt superior than recalculating them from scratch.
When are objects destroyed?
- Pool: When the pool is shut down. Perhaps it might also destroy objects if told to (partially) release resources or if certain objects have not been used for a while.
- XFactory: Whenever an object was created with the version of the source code that is no longer current, as evidenced by either invariant violation or counter mismatch. The process of locating and destroying such objects at the right time is quite complicated. In addition, time-based invalidation of all objects may be implemented to reduce the accumulated risks of using invalid objects. Since XFactory is never certain that it is the sole owner of an object, such invalidation is best achieved by an additional “version counter” in the client objects, which is incremented programmatically on a periodic basis, rather than by a developer.
What special considerations exist for multithreaded environment?
- Pool: Has to avoid collisions in object checking out / checking in (don’t want to check out an object to two clients)
- XFactory: Has to avoid collision in object creation (don’t want to create two objects based on two identical requests)
What needs to be done if client does not release an object?
- Pool: It may want to make the object available to others after waiting for some time.
- XFactory: Not applicable. Clients do not notify XFactory about when they are done with the object.
Do objects need to be modified?
- Pool: May have to be reset to the default state before being reused.
- XFactory: No, the objects are immutable.
Are there any special considerations related to persistence of objects?
- Pool: Typically not. A pool is about saving the cost of object creation, so all the objects are kept in memory (reading from disk would defeat the purpose).
- XFactory: Yes, XFactory is about saving the cost of performing complex calculations, so storing pre-calculated objects on disk makes sense. As a result XFactory needs to deal with the typical problems of persistent storage; e.g. at initialization, it needs to connect to persistent storage, obtain from it the metadata about which objects are currently available there, and be ready to load them into memory if requested. And object may be in one of three states: “doesn’t exist”, “exists on disk”, “exists in memory”. While XFactory is running, the state may change only in one direction (to the right in this sequence).

In summary, the pool’s complexity is in items 1, 2, 4, 6, and possibly 5, 7, 8. The XFactory complexity is in items 3, 6, 9. The only overlap is item 6, and it’s really not the core function of either pool or XFactory, but rather a constraint on the design that is common to any pattern that needs to work in a multithreaded environment.

Your concerns are very much valid and they tell me your original easy caching solution is eventually becoming part of your architecture which naturally brings a new level of issues as you described yourself.

A good architectural solution to caching is to use annotations combined with IoC which solves several problems you described. For example:

Allow you to better control the life-cycle of your cached objects
Allow you to replace the caching behavior easily by changing the annotations (instead of changing the implementation)
Let you easily configure a multi-layered cache where you could be storing in memory then disk cache for example
Let you define the key (hash) for each method in the annotation itself

In my projects (Java or C#) I use Spring caching annotations. You can find a brief description here.

IoC is a key concept in this solution because it allows you to configure your caching system anyway you want.

In order to implement a similar solution in Python you have to find how to use annotations and search for a IoC container that allows you to build proxies. That’s how the annotations work in order to intercept all method calls and provide you this particular solution for caching.

The way I see it the cache is fine – X is not.

IMHO de-serialization of single instances should not be an issue of the cache. It is a task for the according class. The main issue here is that this class is changing frequently. I suggest to separate the concern of caching instances and the concern of de-serializing the object. The latter has to be improved so that X can de-serialize older formats as well. This might be very tricky and expensive. If it is too expensive, you must ask yourself do you really need to load old versions as long as X is changing frequently?

BTW a version identifier seems mandatory. Without further knowledge of the structure of X I can only make some guesses, but the structure of X seems to be logically modular (e.g. you spoke of traits). If so, maybe it would help to make this structure explicit.

Filed under: softwareengineering - @ 05:42

Thẻ: caching, design, design-patterns, persistence, python

Thiết kế website giá rẻ

Danh mục

Caching factory design