Correct, timely and sufficient business intelligence is a very important building block in an enterprise. But gambling systems present us with some difficulties. In this article I'll take a look at how a business can harness much more information, even in near-real time. This article discusses business intelligence in the context of gambling networks but can easily be extended to other domains as well. 

Pinpointing the Problem

Traditional business intelligence is usually performed over historical data and centers round the concept of data mining. As events are propagated through a system they are also written to a database, which can be queried and processed in retrospect to gain insights in user behavior, assist in fraud management, etc.

In many fields this is sufficient, but as a gaming system grows, the sheer number of events in the system makes it impractical to use normal data warehouse techniques as the amount of data produced every second quickly becomes too cumbersome. Fortunately, recent efforts in complex event processing, or CEP, and staged computing offers a way forward.

The Event Cloud

Let's imagine a large enterprise system, there are a number of servers all connected to each other, to databases and to other integrations. Every second, thousands of events are generated in the system in response to user actions, forming a large cloud of events.

Different subsets of the system will be concerned with different events – a game for example will handle only the events appropriate for itself and its players – and through this specialization of separate modules the events are partitioned, and each event processed only to instantly be replaced by a new one.

Even now we're starting to see the problem: how can we make sense of this event cloud? It is too large to write to a database in its entirety, and as the events are of a great variety of types it is far from clear how we can combine events to form a coherent picture.

Before we go further, it is all too easy to slip into a language so abstract and completely deprived of specifics that the end text is almost intelligible, so let's put down a simple example: After initiating a campaign for a subset of our users with the goal of getting them to play more of a specific game we need to evaluate the outcome. In other words we'd like to know: if a user is a member of the campaign subset when he logs in, and if he is we'd like to know his “click path”, in other words the order which the user navigates in the system after logging in, and finally if he ultimately ends up playing the game more.

In order to evaluate our campaign example there are a number of modules and components involved which may not know how, or indeed want to, interact with each other. We need information from the system authenticating the user, we possibly need information from the game client for the click path, and we need information from the game server themselves. And this is information spread over time, so that any individual snapshot of the event cloud will not be of help; we need to track aspects of the event cloud as it progresses ever forward in time.

Complex Event Processing

Complex Event Processing, or CEP, is primarily an event processing concept that deals with the task of processing multiple events from en event cloud with the goal of identifying the meaningful events within it. CEP employs techniques such as detection of complex patterns, event correlation and abstraction, event hierarchies, and relationships between events such as causality, membership, and timing.

Crucially, CEP allows for patterns to be recognized and correlated in an event flow over time, in so called sliding time windows. In our example we could use CEP to correlate the disparate events we need over a period of time starting from when the user logs into the system.

Another example might be in order to highlight the benefits of CEP, detecting login problems in the system: Imagine all login events as a subset of our event cloud. CEP allows us to do pattern detection on this subset over a moving time window, so that we can express almost in plain text the following: “raise an alert if the number of login attempts the last ten minutes differs significantly from the average ten minutes the last twenty four hours”. Such an alert would signify a technical problem in the system which must be investigated and is usually hard to detect. There could be network problems, an integration could be non-responsive, the database down on its knees, etc.

Staged Computing

In order to utilize the data, and to tap the event cloud, we need another abstraction. We don't want to spend CPU cycles on the actual machines running our enterprise. Rather, we want to hand off a subset of the event cloud to another process where we can analyze and correlate the data.

Staged computing is the concept of processing as a chain of independent stages that perform computing tasks isolated from one another. After an event is generated it is passed through a chain of processors that each perform a subtask. For example, a login event may first pass through the stage using CEP to determine the system login status as described above, then handed off to another stage using CEP and an IP database to check for “impossible travel” indicating an account used by multiple users, then on to even more filtering.

One of the main advantages of staged computing is that it is easy to scale and maintain. As different stages usually are separated by message queues or communicate with stateless sessions which can easily be load balanced, a staged computing setup for our real time business intelligence would be easy to extend and maintain.

Putting It All Together

Now let's combine our new knowledge of complex event processing and staged computing, what could we actually use this for? Some examples may be:

• Fraud Detection - Event processing over a sliding window of time can be used to detect subtle patterns in client behavior. This could be used to detect chip dumping, collusion etc.
• Game Experience Management - A client management application that would use transient game events to build a picture of the current client experience and tracking promotion drive impact.
• Process Supervision - In large enterprise gambling installation correlation of individual systems or services may present a non-trivial problem. Using CEP, problem areas can be detected earlier and a large scale picture of the system health can be updated in real-time.

There's a sea of data and knowledge available in the event cloud of every major gambling installation that could be crucial to the success and endurance of the business. The problem is how to get our hands on it. I believe complex event processing and staged computing is one of the answers we've been looking for. Using these techniques could be the game changer that takes a business just that one step ahead of the competition.

Lars J. Nilsson is a founder and Executive Vice President of Cubeia Ltd. He's an autodidact programmer, former opera singer, and lunch time poet. You can contact him at lars.j.nilsson in Cubeia's domain.