The information and communications technology (ICT) sector has become the backbone of our society by connecting people and devices with each other. Since related visions of the future, such as the Internet of Things (IoT), are now being driven forward by industry at a rapid pace, the demands on ICT technologies, which made this development possible in the first place, are increasing by leaps and bounds.

As a result of these requirements, existing ICT architectures are becoming more and more complex and increasingly challenging to maintain and monitor manually. For this reason, automated approaches are moving to the centre of research to monitor ICT structures, present information in a user-friendly manner, and automate workflows. The challenges in this context are primarily due to the high dynamics with which the relevant communication networks, cloud centre structures and associated applications are constantly growing and changing. In order to address these challenges, the changes that IT infrastructures and the applications running on them are subject to must be made traceable. For example, to be able to react to undesired constellations and sequences of events.

Hence, methods and tools are needed for the efficient development of monitors that recognize unusual sequences of (communication) events and monitor and evaluate structural changes in the underlying ICT structures. Currently, approaches for detecting critical sequences of communication events, on the one hand, and approaches for monitoring the associated ICT structures, on the other hand, have been developed separately. For example, the approach presented in [1] is designed to detect events such as a significant increase in the number of connections to a database server in a particular network in a specific time interval. For this purpose, streams of atomic events can be analyzed with so-called complex event processing (CEP) tools. In contrast, the approach presented in [2] for monitoring structural changes can detect the migration of a database containing personal data from EU citizens to a provider outside the EU as an unwanted structural change. However, the pattern matching techniques used in this context completely ignore the time dimension. In practice, this is not sufficient since often, only certain sequences of structure-changing events in an ICT system are safety-critical. For example, the storage of personal data of EU citizens in a particular database and access to this database by a customer who is not certified as “trustworthy” is only security-critical if this access occurs after storing the personal data.

In this research project, we research the event- and structure-based monitoring of ICT systems using Model-Driven Engineering (MDE) techniques to develop ICT monitors at a high level of abstraction. This approach is illustrated in more detail in Figure 1. Here, incremental graph pattern matching (IGPM) techniques are applied to detect structural events, such as the appearance or disappearance of certain configuration patterns, in an ICT scenario efficiently. This approach is combined with a well-established CEP tool, which makes it possible to efficiently search streams of events for occurrences of critical sequences in certain time windows and to process them in the form of so-called complex events. The information derived this way is made available to the user and can be used to react automatically to complex events.

The goal of the project is to combine model-driven engineering (MDE), incremental graph pattern matching (IGPM) and complex event processing (CEP) in order to implement a novel monitoring approach.

The three main aspects of the approach are described as follows:

(1) Modeling

In the project, models are used to capture the respective current state of an ICT system to be monitored at a suitable level of abstraction. For this purpose, so-called metamodels are specified that describe the structure of ICT system models. This offers the advantage that developers can agree on a clear specification of the problem domain, which improves development processes and reduces frequent sources of error.

(2) Incremental Graph Pattern Matching - Detecting structural changes

Based on the representation of the aspects of the ICT system under investigation, interesting substructures and structural changes can be described in the next step. Graph pattern matching is a technique that is used for this purpose in various domains [4]. The models are considered as graphs (consisting of nodes and edges), on which we search for the appearance or disappearance of certain graph patterns. For example, a graph pattern could describe the structural relationship between a database and its physical medium. It is assumed that the models or graphs under consideration reflect the current state of the monitored ICT system, i.e., they are themselves subject to a continuous adaptation process. For the case mentioned above, this means that a previously found occurrence of this pattern can disappear, and a new one can appear with the same database. The new pattern occurrence can then, for example, be examined to determine whether the database has been migrated. For this reason, it is recommended to use so-called incremental graph pattern matching techniques (IGPM techniques), which do not start a new pattern matching process after each graph change. Their incremental approach is based on maintaining index structures for all occurrences of all relevant subpatterns of the graph patterns to be monitored. Each atomic graph change results in an update of the corresponding index structure, leading to more manageable runtime characteristics compared to traditional pattern matching approaches. Thus, the occurrence or disappearance of patterns of interest to the administration of an ICT system can be efficiently monitored and reported continuously. Well-known IGPM tools include Viatra [5] and Democles [6] or the recently developed HiPE.

(3) Complex Event Processing - Analysis of event sequences

The result of analysing the adaptation process of an ICT system with an IGPM tool is a stream of events that can provide valuable information about the dynamics of the ICT system under study. The combination of several atomic events, which have a certain temporal correlation, to complex events can be accomplished by complex event processing techniques. Modern CEP tools, such as Apama, can examine event streams for various dependencies and in real-time. The dependencies can be causal (if-then) but also temporal (if-before). They place high demands on modern CEP tools, which have to examine ordered streams of events for these dependencies in a performant way. In this research project, CEP is used to analyse critical sequences of events that arise from structural changes to ICT models detected using IGPM techniques. We derive complex events from these (atomic) events that correspond, for example, to violations of data protection rules in cloud applications.

Approach

The resulting approach combines MDE techniques and IGPM with state-of-the-art CEP tools. This creates a synergy between IGPM and CEP since the atomic events detected by IGPM techniques already contain non-trivial structural information. Thus, the complex events derived from them gain even more significance. The results obtained in this way are subsequently used to react to events in an automated manner and detect unwanted situations in the ICT scenario under investigation and trigger appropriate defensive measures. Figure 2 shows the interaction of IGPM and CEP as an example. Here, an ICT model (e.g., a cloud) is continuously changing due to some events. Such an event can be, for example, the storage of data of European citizens in a database (1) or the migration of a database abroad (2). These events are detected by an IGPM tool and passed to a CEP tool. This CEP tool, in turn, is subsequently used to examine the event sequence for complex events. In this example, such a complex event would be a temporal connection between the two events (1) and (2) since a database on which data of European citizens was stored may no longer be migrated abroad.

GrapeL: Combining Graph Pattern Matching and Complex Event Processing [7]

The GrapeL framework represents a novel approach to combine graph pattern matching and complex event processing. We use GrapeL as a textual language glueing IGPM and CEP together to generate solutions that integrate both approaches to benefit from their synergy. GrapeL is integrated into eMoflon, a state-of-the-art graph transformation tool, in which we implemented a flight and booking scenario as a proof-of-concept.

The MEMIK project has been funded by the German Federal Ministry of Education and Research as part of the Software Campus initiative from 2017 to 2021.