Integrated Model-based Testing of Continuously Evolving Software Product Lines (IMoTEP)
Automation engineers are regularly faced with the problem to maintain long living safety critical process control software families. A promising paradigm for developing this type of software efficiently is (dynamic) software product line (SPL) engineering; it supports the systematic development of software product families of similar applications. However, existing SPL approaches offer only little support for integrated quality assurance and evolution of software over time. In particular, there exist only few approaches and no concise methodology for testing evolving SPLs and their applications until now. For automation systems, there is a particular need for such a support, since here modifications often have to be conducted under hard safety constraints—sometimes even in a running system.
We hence propose an approach for managed dynamic SPL evolution in the automation engineering domain with a particular focus on efficient model-based testing techniques. SPL test suites needed for (1) systematic offline testing of representative sets of products in their development environments and (2) online testing of reconfigurating products in their runtime environments are generated and incrementally updated using a mixture of model checking, constraint solving, and model transformation techniques. Integrated general purpose feature modeling and domain-specific (test) modeling languages are used together with standard model- and new feature model/interaction-based coverage criteria for that purpose. The involved models and their metamodels as well as the associated model processing algorithms are used both at design and runtime. The overall aim is to support both predefined reconfigurations of products and unforeseen evolution of full product lines with a specific focus on the adaptation of their test suites.
Second Funding Phase (IMoTEP 2)
Today’s automation engineers are faced with new challenges concerning the development and maintenance of modern production systems as, e.g., pointed out in the Industry 4.0 manifesto. This includes the maintenance of often very long living, safety-critical, real-time control software with a rapidly growing degree of distribution as well as variability and adaptability at design and runtime. A promising paradigm for developing this type of software efficiently is Dynamic Software Product Line (DSPL) engineering combined with model-based testing (MBT) techniques; DSPL engineering supports the development of families of (self-)adaptive software systems, whereas MBT offers the appropriate means for systematically deriving test suites from test models and related coverage criteria.
In the first funding period of the IMoTEP project, we laid the conceptual foundation for a comprehensive variant-aware MBT methodology with significant achievements concerning, e.g., the adaptation of model-checking techniques for DSPL test suite generation purposes, the integration of domain-specific modeling languages in a tooling framework, the delta-based handling of SPL evolution scenarios, and the usage of multi-objective optimization techniques for test case selection and test suite minimization purposes. In the second funding phase we will integrate our delta-based techniques for the evolution of SPL versions and their related test cases with our variant-aware approach for the model-based generation of a DSPL test suite that puts specific focus on runtime reconfiguration activities. The resulting tool-assisted methodology will offer unified support for the incremental co-evolution of all DSPL testing artifacts and regression testing strategies for mastering anticipated and unanticipated changes of a long-living DSPL.
Based on our experiences with a real-world case study from an industrial partner and the PPU case study of the SPP we will develop automation and application domain-specific (model) coverage and mutation-based test selection criteria in order to increase the efficiency and effectiveness of generated test suites. Furthermore, the developed DSPL test suite maintenance methodology will be extended to take scenarios of distributed components with real-time constraints for their normal and reconfiguration behavior into account. The IMoTEP tooling will be extended accordingly, relying on meta-modeling, model transformation techniques in turn for the integration of a whole range of variability and modeling languages used in the automation engineering domain. In addition, we will continue our efforts to select and adapt advanced model-checking techniques and tools for our DSPL test suite generation and maintenance purposes. Last but not least we will continue our efforts to apply the developed methodology in the context of the above mentioned two case studies, complemented by controlled experiments studying the usability of the developed approach.