Category: Master Students

Literature Study on Performance Prediction

Posted on by Benny Akesson

Today, Jan Przystal presented his literature study Performance Prediction for Microservice-Based Cyber-Physical Systems: A Cross-Domain Literature Review. The study reviews existing methods for predicting performance in Cyber-Physical Systems (CPS) during the early design phase. It focuses on how resource contention, caused by shared hardware such as caches, memory, and I/O, leads to performance degradation, and why this makes Design Space Exploration (DSE) challenging for complex distributed CPS. The study surveys simulation-based approaches, profiling techniques, and interference prediction methods originating from both CPS and cloud computing, and evaluates their suitability for fast and scalable performance estimation. It concludes that while current methods can provide useful approximations, none fully meet the need for a quick, accurate, and scalable prediction approach for large CPS design spaces, highlighting the need for further research in this area. These are challenges that Jan will address during his master project, building on our earlier work with Bruno Dzikowski in this area.

It was nice to see that there was broad interest in Jan’s presentation not only from the performance team at TNO-ESI, but also among some of its industry partners, as well as experts in performance prediction from University of Amsterdam, Eindhoven University of Technology, and University of Twente. That certainly made the Q&A session afterwards spicier, although Jan confidently answered most of the questions.  Congratulations Jan on work well done!

Bruno Dzikowski Successfully Defends Master’s Thesis on Performance Prediction

Posted on by Benny Akesson

Yesterday, Bruno Dzikowski successfully defended his master’s thesis titled “Practical Recommendations for Accurately Predicting Performance Degradation Caused by Memory Contention” The thesis addresses the challenge of predicting performance in microservice-based architectures for cyber-physical systems (CPS) running on multi-core platforms, where resource contention significantly impacts accuracy. Existing methods model interference sensitivity and contentiousness but lack practical implementation guidelines.

 

Bruno’s work introduces a compositional performance prediction framework with three key contributions: 1) a validated contentiousness profiling component, 2) an analysis of how system configuration affects prediction accuracy, and 3) the design and implementation of an experimental testbed. Tested across 195 co-location scenarios, the approach achieves high accuracy (median error ≈ 1.4%), demonstrating its effectiveness for forecasting microservice performance.

We are very proud of the excellent research Bruno conducted during his internship with TNO-ESI, which resulted in an outstanding thesis that was confidently presented and defended. We thanks Bruno for the excellent collaboration and wish him all the best for his future career.

Hyperheuristic Optimization in Cyber-Physical System Design

Posted on by Benny Akesson

Today, we proudly celebrate Lars van der Water’s successful defense of his Master’s thesis, Exploring Vast Design Spaces with Hyperheuristics: Theoretical Foundations and Autotuning Implementation, at the University of Amsterdam. This work has been conducted in connection with the DSE2.0 project, a research collaboration between University of Amsterdam, Leiden University, and ASML, co-funded by NWO and TNO-ESI as a part of the Mastering Complexity (MasCot) Program.

Lars’ thesis addresses the growing complexity in designing Distributed Cyber-Physical Systems, which are increasingly vital to infrastructure and industry. Traditional Design Space Exploration methods struggle with scalability, algorithm selection, and parameter tuning, creating a bottleneck in efficient exploration of system designs. To overcome this, this work explores hyperheuristics (HHs) as a higher-level domain-agnostic approach to automate the selection and tuning of metaheuristics.  Key contributions include a modular framework for integrating HH strategies, and empirical insights into the trade-offs between performance, effort, and computational cost in autotuning. Experiments show promising results for auto-tuning of simpler meta-heuristic search algorithms like Gravitational Search and Particle Swarm Optimization, but revealing limitations with more complex ones like Genetic Algorithms.

We sincerely thank Lars for the excellent collaboration and wish him all the best in the next chapter of his career!

William Ford Successfully Defends Master Thesis on Network Delay Models for dCPS

Posted on by Benny Akesson

On Wednesday, William Ford, a master student from VU/UvA defended his master thesis “Network Delay Model Creation and Validation for Design Space Exploration of Distributed Cyber-Physical Systems“. This thesis was executed in the context of the MasCot project DSE2.0 and was supervised by Benny and Faezeh Sadat Saadatmand, PhD student at Leiden University.

William’s thesis focuses on improving the development process of complex distributed cyber-physical systems (dCPS), such as the equipment developed by high-tech companies like ASML, Canon Production Printing, and Philips. Building physical prototypes for these systems is complex and costly, so the thesis explores automated and scalable model-based Design Space Exploration (DSE) as a solution. The research addresses the challenge of modeling network delays in dCPS, aiming to create models that balance speed and accuracy for DSE purposes. The methodology includes formalizing network topology and traffic concepts, resulting in an open-source framework for synthetic network generation called GeNSim. Three analytical network delay models—Constant Delay, Constant Bandwidth, and Latency-Rate, and a simulation-based approach using the INET framework—are proposed and evaluated synthetic networks and an industry case study at ASML. The findings reveal that each model has its strengths and weaknesses, with no single model meeting all requirements perfectly. Therefore, a multi-step modeling approach is suggested to leverage the strengths and mitigate the weaknesses of the different models.

William confidently presented his thesis. In particular, the committee was very happy with the Q&A session after the presentation, which resulted in a lively back and forth with interesting questions and answers. Having defended his thesis, William can now apply for his diploma and graduate. We thank William for his contributions to the DSE2.0 research and wish him all the best with his future career.

Master’s Thesis Explores User Behavior’s Impact on Digital Service Energy Consumption

Posted on by Benny Akesson

Just before the end of summer, Nsidibe Onoyom Bassey, master student at the Vrije Universiteit Amsterdam, has successfully defended her thesis “Impact of Users’ Behavior on Digital Service Energy Consumption“. Congratulations on the defense and completing your studies Nsidibe!

This work was supervised by Ana Lucia Varbanescu and myself in the context of our research project Energy Labels for Digital Services, which studies the energy consumption of applications distributed over the compute continuum. In particular, the research addresses the growing concerns over energy consumption in the ICT sector, which poses challenges to achieving net-zero emissions. While ICT solutions are often seen as efficient and low-cost, their energy impact is significant, particularly due to the high demand for digital services, such as online shopping. Energy consumption in the digital domain is largely driven by hardware, software, and infrastructure, but the role of user behavior in influencing this consumption is often overlooked. The thesis focuses on understanding how user behavior affects energy consumption in digital services, using a commonly used open-source online shop implemented as microservices as a case study. The energy consumption on both the client and server side is studied and experiments are conducted with different client browsers, user interactions, and number of users. Based on the experiments, an analytical model is proposed to estimate the energy impact of user behavior on the server side and recommendations are made to both users and developers for how to limit energy consumption.

Master Thesis Project Leads to Conference Publication on Microservice Architecture Anti-Patterns at SEAA 2024

Posted on by Benny Akesson

I am delighted to announce that our paper, “Graph-based Anti-Pattern Detection in Microservice Applications,” has been accepted for publication at the 50th Euromicro Conference Series on Software Engineering and Advanced Applications (SEAA). This paper stems from Amund Lunke Røhne’s master thesis project, which he conducted as an internship with TNO-ESI under the supervision of myself and Ben Pronk. This achievement showcases how exceptional work by master students can lead to publications in established conferences.

Our paper addresses a significant challenge in the evolution of microservice applications: as the microservice architecture evolves, architectural anti-patterns may emerge. These anti-patterns are challenging to detect and manage due to their informal natural language definitions and the lack of automated tools. To tackle this, we propose an automated methodology for detecting architectural anti-patterns related to microservice dependencies. A key component of this methodology is the novel Granular Hardware Utilization-Based Service Dependency Graph (GHUBS) model, which is automatically inferred from telemetry data. We have formalized three commonly known anti-patterns and developed algorithms to detect them within the GHUBS model. This methodology is supported by an open-source tool that automatically identifies and visualizes these anti-patterns. We validated our approach using both synthetic data and a case study of a popular microservice benchmarking suite, demonstrating successful detection of the formalized anti-patterns.

Congratulations to Amund on the acceptance of your paper! Your work has made both TNO-ESI and the Software Engineering program at the University of Amsterdam very proud!

Merrick Oost-Rosengren Successfully Defends Thesis on Early Component Verification using Colored Petri Nets

Posted on by Benny Akesson

Just before the summer holidays, another master student has finished his project. This time, it is Merrick Oost-Rosengren who successfully defended his thesis “Formal Verification of Components through Mirroring of Coloured Petri Nets“. Parts of this work was done as an internship with TNO-ESI in collaboration with Thales.

This research addresses a challenge in distributed component-based systems, where different components are developed by different teams, perhaps even different organizations, over time. The problem is that when components are ultimately integrated, their interactions may cause deadlocks, livelock, or unbounded memory behavior. Fixing such problems late in the development process is very costly. An alternative approach is to model components, or component interfaces, early in the design process and use model checking to verify the behavior of the component and its interactions. However, we may not know which components it will interact with yet. Perhaps they have not yet been developed?

The thesis addresses this challenge by proposing a methodology and corresponding tool chain, where components as modelled as Colored Petri Nets from which a verification model, a mirror of the component that captures relevant possible behaviors of interacting components, is automatically generated. As a part of the methodology, the thesis proposes a new class of Colored Petri Nets called Mirrorable Open Colored Petri Nets. This class combines features of existing Colored Petri Nets and Open Petri Nets, and also adds extra semantics to allow the component to be mirrored. It also describes methods for mirroring such a net and fusing the mirror with the original component, such that the components and its interactions can be verified using reachability analysis.

We congratulate Merrick on his successful defense and wish him a lovely summer!

The Journey from Offline to Online Conformance Checking for Microservice Applications

Posted on by Benny Akesson

Ricardo Andrade has successfully defended his master thesis “Real-Time Conformance Checking for Microservice Applications“. This thesis was done in the context of the ArchViews project together with Thales. The academic supervisor was ESI colleague and TU/e professor Johan Lukkien and the daily supervision at ESI was done by myself and Ben Pronk.

The thesis addresses the shift from monolithic architectures to microservice architectures in order to manage the complexities and dependencies that emerge as systems grow and incorporate new features. A significant gap identified in the management of microservice applications is the lack of effective conformance checking techniques that can verify whether the execution of microservices aligns with their specification. To address this, the thesis proposes an innovative solution by developing an online conformance checker specifically designed for microservice applications. The project begins with the creation of an offline conformance checker that evaluates conformance using execution traces and sequence diagrams. The work then progresses to an online conformance checker, significantly improving performance and delivering conformance results within approximately 30 seconds per trace. This rapid response time meets the requirement for swift identification and correction of non-conforming sequences, thereby offering a practical and effective tool for managing microservice applications.

Ricardo presented his work very well using beautifully prepared slides. He confidently answered questions from the audience and the examination committee and left the session with a good grade. Ricardo is now moving on from his studies to start his career at CGI. We wish him the best of luck in his future career.

Jesse Liauw-A-Fong Defends Master’s Thesis on Local Anomaly Detection in Smart Public Transport Vehicles

Posted on by Benny Akesson

Yesterday, Jesse Liauw-A-Fong, a student of the Master of Software Engineering program at UvA, defended his thesis Local Anomaly Detection in Smart Public Transport Vehicles. This research was conducted externally at a company called Ximedes. Jessie’s research is addressing the problem of detecting anomalies, such as a loss of cloud connection, in Smart Public Transport Vehicles (SPTV), such as buses, trams, and metros, comprising many complex heterogeneous systems. It emphasizes the importance of local, context-aware anomaly detection due to the dynamic nature of SPTVs and explores the generalization of anomaly detection, particularly addressing performance, normal region, and quality challenges. The research proposes a unified data collection framework comparing agent-based and agent-less methods, advocating for an agent-based approach for its adaptability and integration ease. It also quantitatively evaluates three local anomaly detection algorithms on real data from a specific bus line. We thank Jessie for his contributions to our research and wish him the best of luck in his future career.

Master Thesis Shines Light on Hardware Dimensioning for Cyber-Physical Systems

Posted on by Benny Akesson

On Wednesday, Marijn Vollaard defended his master thesis “Hardware Dimensioning for Microservice-based Cyber-Physical Systems: A Profiling and Performance Prediction Method” at the University of Amsterdam. This research has been supervised by Ben Pronk and myself as a part of a project with TNO-ESI.

The thesis addresses the problem of determining the number of homogeneous compute nodes needed for a particular variant of a cyber-physical system to meet its timing requirements. This is important in early discussions with customers and bidding processes, since it affects the size and cost of the resulting system. To this end, the thesis proposes a structured hardware dimensioning methodology comprising a profiling method and a performance prediction method. The four novel contributions of the thesis are: 1) A component-based profiling method, 2) a performance prediction method, 3) a structured hardware dimensioning methodology, and 4) validation of the approach, using a case study that represents a prototype of a CPS. Experimental evaluations on the case study show that the predicted performance differs from measurements on the application by at most 20%, which is satisfactory for hardware dimensioning decisions for new product variants.

The defense went well and Marijn confidently presented his story and convincingly answered the questions of the audience. The examination committee, impressed by his work, awarded his thesis a well-deserved grade of 8. As we bid farewell to Marijn, embarking on his next career adventure, we also extend our heartfelt congratulations. He certainly has much to be proud of. We wish him all the best on his travels and in his future pursuits.