Benny Akesson

Today, we had a double whammy! Two excellent master students from the University of Amsterdam that conducted their projects as internships with TNO-ESI have defended their work on the same day. Both projects were conducted in the context of an applied research project conducted in partnership between TNO-ESI and Thales and share a common context. As cyber-physical systems are getting increasingly complex and software-intensive, the industry is looking for new design methodologies to increase engineering productivity. One promising direction is to automatically synthesize systems from a set of requirements. This may involve selecting the components to integrate and mapping and deploying software on available hardware nodes. The design space for system synthesis is huge, but can be efficiently navigated using a design-space exploration tool that iteratively optimizes the design.

One challenge for such a design-space exploration tool is to quickly estimate the performance of selected software components for a candidate mapping to hardware nodes. Jan Przystal’s thesis Early Software Performance Prediction in Cyber-Physical System Design tackles exactly this challenge. Building on the Bubble Up method, in particular on the thesis of our previous student Bruno Dzikowski, his thesis investigates how the performance of software components can be predicted early in the design process, before a complete system has been built or deployed. Rather than relying on time-consuming simulations or exhaustive testing of all possible deployment configurations, the Bubble Up method individually characterizes software components by how sensitive they are to contention in shared resources and how much contention they create themselves. These individual characterizations allow performance predictions to be made quickly for any pair of  applications, as shown in the figure below. Jan’s work significantly improved the prediction accuracy and reduced the profiling time compared to the baseline. In addition, he extended the baseline approach to cover realistic deployment scenarios with many co-located applications. He also took the first steps beyond sharing only memory resources by investigating how sharing CPU resources affects application performance and how these effects can be incorporated into the prediction framework. Ultimately, his work brings automated system synthesis one step closer by enabling design-space exploration tools to rapidly assess the performance implications of alternative software-to-hardware mappings.

The work of Toine van Wonderen addresses a quite different aspect of system synthesis. It observes that design-space exploration evaluates many, many, possible candidate system configurations and provides an optimized solution, but it does not teach developers anything about what actually makes a particular configuration good or bad. To this end, his thesis Interpretable DSE – Extracting Design Principles from the Automated Synthesis of Hierarchical dCPS aims to automatically extract design principles from the design-space exploration process. In his work, Toine developed a methodology that first converts synthesized system configurations into a novel interpretable representation and then uses machine-learning techniques to learn the relationships between design decisions and resulting system quality attributes. By applying explainable AI techniques, in particular SHAP (SHapley Additive exPlanations), his approach identifies which design choices contribute most to a solution’s quality, as shown in the SHAP beeswarm plot below. Rather than treating the outcome of design-space exploration as a black box, his work enables engineers to understand why certain solutions emerge and what design principles can be derived from them. Evaluated on a complex case study from the defense domain, the approach demonstrated that valuable architectural insights can be extracted automatically from the large volumes of data generated during design-space exploration. In this way, Toine’s work complements the optimization capabilities of automated system synthesis by turning exploration results into actionable knowledge that can support future design decisions and deepen engineers’ understanding of complex system architectures.