Event Driven Programming: A sequence of unfortunate events

I have a very basic game loop whose primary purpose is to check for updates & changes to a list.

I have contemplated using event driven programming to replace the game loop/list idea with an ObservableCollection, however I just have this big cloud of doubt on event driven programming. I’m posing these questions to those with experience with event driven programming:

What are good ways to test & build an event driven programming design?

That is, how will I know that I cannot run into “a sequence of unfortunate events”? I want to avoid the events I didn’t plan for.
Are there unit tests schemes to test event driven programming?

Observable collections are found at:

http://msdn.microsoft.com/en-us/library/ms668604.aspx

I will leave aside the wisdom (or not) of this particuar design for a game engine, I will address the core concern about the testability of event-driven designs.

When you state

I want to avoid the events I didn’t plan for.

I presume you mean a sequence that you did not plan for, like Create/Delete/Update instead of the expected Create/Update/Delete, where the former might try and update a non-existant object and therefore crash or otherwise fail.

The entire nature of event-driven programming has to actually presume a near-random order of events, and so effectively your pre- and post- conditions become very important. Since you cannot fully qualify the order of events, you need to be robust against accidental ordering. So long as your Delete leaves no stray pointers, then a delayed Update should unambiguously fail, or at least have some well-defined behavior.

As far as unit-testing goes, most of the common suggestions would still apply. Generate test cases for various boundary conditions, as well as common cases to try and achieve appropriate coverage and interaction testing.

Perhaps in your game, I should be able to PickUp an item, which would add it to my inventory. If I attempt to PickUp an item, which was just destroyed for some reason (perhaps an ogre stepped on it), I should not somehow get an undamaged object due to a race condition. Either I got the object before it was crushed, or it is now gone, but nothing inbetween. If each event is atomic with respect to game state, then you won’t get a partial object someplace it should not be.

Good Luck

I used XSLT as a general purpose programming language for 6 years at work. XSLT is not exactly “event” driven, but it is input-driven – meaning that there is no pre-defined order of execution or limited number of paths that the execution of the code can take. Rather, each node in the input data tree triggers some code to be executed. It’s totally data driven, which I imagine is similar to your event-driven model.

Anyone who has spent a few years writing software for a large organization knows that there is no end to the variety of data that gets into big systems. Every time you account for one condition, a new one pops up that no-one ever imagined. When your input data defines the order of execution of your code, then the paths you need to test are equal to the total number of data conditions that your code can be exposed to – a number which grows daily. The people using XSLT to write batch data crunching processes would be paged repeatedly at home, at night, pretty much every night as new data conditions were “discovered” by the program.

When you let your data drive your software, you will often run into a “sequence of unfortunate events.” Fault tolerance is a big deal. Emergent behaviors rule the day. Limits are your friends. Obviously for a game universe to feel real, the fewer limits, the better. You must choose your limits very wisely. After all, a game is art, and an important part of any artistic statement is its limits.

Speaking of art, John Cage wrote a piece of music, “Four minutes and thirty three seconds” where the performer sits, making no sound, for that length of time. The point of this artistic composition, as far as I’m concerned, is not the picture, but the frame. Everything has limits. You are making a game about one thing, not another thing. It is one game, not another game. The better you can choose limits, and the more appropriate those limits are for your art/game/project, the easier it’s going to be, and the better the experience will be for your users/gamers.

All software (like all music and art) has limits. Choosing those limits wisely may be the single most determining factor of the success of any software design. But with event- or data-driven programming, this is perhaps even more critical, if such a thing is possible, because every bit of non-essential flexibility that you allow in such a system will punish your development team with bugs and unimaginably complicated testing. First make as many limits on your event-driven system as possible, and only remove or extend a limit when doing so has a major, positive effect on the system as a whole.

Unit testing (generally black-box testing) usually takes the form of, “given certain inputs, does the software being tested return the expected outputs.” For a very versatile event-driven system, you need a lot of variety in those inputs. Maybe you could capture all the interesting kinds of input data that you’ve seen in your game so far and make tests from those. Maybe you can add a new test whenever you encounter a new and interesting data condition. But with a sufficiently complicated system, there is no way to test every data condition, only the currently known conditions.

Maybe it would be good to build that kind of sanity testing (bounds-checking, things null or not null, no exceptions, etc.) into some sort of logging utility that would alert you when your method sees something new in the wild?

Hopefully a statistician will answer your question too and tell you how to cover a high percentage of the possible data conditions in a meaningful way. But until then, choosing the limits of your data- or event-driven software wisely is the only way I know to control the complexity of a meaningful set of unit tests.

I’d echo some of Glen’s answer here. Event-driven programming, taken to a deep level, can be the worst debugging nightmare ever. You also can’t effectively unit test against it, since it’s almost always temporal in nature with side effects that aren’t violating object-level invariants, but broad system-level invariants (the individual objects can be in a valid state, but combined together aren’t in the expected state).

A lot of the complexity of large-scale systems in trying to reason about their correctness to me is the combination of side effects and event-driven programming. Side effects are always a tripping point, and event-driven programming tends to make it a mystery about what actually is going to happen when an event is triggered. The two combined, when looking down at millions of lines of code and call stacks which are 45 levels deep, can make the most deadly concoction.

Maintainability in terms of being able to effectively make changes is one aspect, but correctness and reliability is a huge other one. It’s hard to get an overall sense of correctness when the control flow is going wildly all over the place with deeply nested function calls, and further if those function calls are not known in advance (involving a dynamic dispatch, as in the case of events). Observers of observers of observers is not a pleasant thing here.

… and this graph is simplified. When you’re dealing with really complex systems spanning millions of lines of code that go all out on events in this way, running into that unfortunate series of events doesn’t seem like a probability, but an inevitability if there is any possible way your system can break if things are called in an arbitrary order.

I’ve worked with some developers who thought exception-handling was obfuscating control flow with how throwing would branch potentially way down the call stack. Pshaw, that’s just a linear branch unwinding the stack. Try to guess where some arbitrary event triggered will actually lead us (the full comprehensive graph) from a control flow perspective in a very complex system, and that tends to be an absolute ??? until we trace into the whole thing in its entirety.

The only strategy I’ve found to fight against this is:

Do broad integration testing. Protect yourself. Unit testing is largely useless here, as events are often attached in a site-specific way and it’s only through a very unusual combination of events that we run into bugs. It needs to be broad integration testing to cover the possibilities of these unfortunate series of events.
Assert liberally. Assertions in the middle of production code and outside of testing are kind of like a poor man’s “on-the-site” testing, but in these kinds of cases which are so hard to test for, running into an assertion failure can be a lifesaver as opposed to just some odd side effect which leads to more odd side effects which lead to more for the person debugging without actually bringing the process to a jarring halt.
Strive for a shallower call stack (not so much to the point of epic functions that are their own debugging nightmare, but there’s something to be said for centralized code in one function that processes things in a straight top-to-bottom fashion with possibly some loops here and there). Don’t recursively trigger events if you can help it.
Whenever possible, swallow up events and turn it into like a concurrent event-queue, waking up a thread and processed in one central loop. It helps flatten the whole thing and kind of centralize the timing of when events occur. It also fights against “event spam processing” inefficiencies where you might otherwise be tempted to do expensive work redundantly when two or more events could just lead to one update of some data structure or a screen refresh, e.g.
Continued from #4, invert push paradigms to more pull paradigms. There are ways to implement pull paradigms without burning up cycles needlessly polling, as with a concurrent queue which often uses condition variables to wake up a thread when the queue is non-empty.
Favor immutability whenever possible. Side effects to central data structures are difficult to reason about, and ever more so when there is no sanity or predictability to the order in which they happen which is especially true in a complex event-driven system where you might have no idea what actually gets called when a button is clicked until you trace into the entire graph and map the control flow in a diagram.
I haven’t tried this, but it might be a fine idea to make it impossible in advance to trigger recursive events. Instead, for example, a recursive attempt to trigger an event might defer the actual signal/notification until the processing of the current event is completely finished. This will not necessarily protect against an unfortunate series of events, but will keep the call stack shallower which makes tracing easier (as well as simplifying the conceptual graph). As Glen said, the more constraints and limitations you impose on your event system upfront, the easier your life will generally become. “Uber-flexible event system” should be a warning sign raising red flags rather than something that we applaud.

Filed under: softwareengineering - @ 20:27

Thẻ: c++, event-programming, object-oriented, testing, unit-testing

Thiết kế website giá rẻ

Danh mục

Event Driven Programming: A sequence of unfortunate events