Tag: performance prediction

Literature Study on Performance Prediction

Posted on by Benny Akesson

Today, Jan Przystal presented his literature study Performance Prediction for Microservice-Based Cyber-Physical Systems: A Cross-Domain Literature Review. The study reviews existing methods for predicting performance in Cyber-Physical Systems (CPS) during the early design phase. It focuses on how resource contention, caused by shared hardware such as caches, memory, and I/O, leads to performance degradation, and why this makes Design Space Exploration (DSE) challenging for complex distributed CPS. The study surveys simulation-based approaches, profiling techniques, and interference prediction methods originating from both CPS and cloud computing, and evaluates their suitability for fast and scalable performance estimation. It concludes that while current methods can provide useful approximations, none fully meet the need for a quick, accurate, and scalable prediction approach for large CPS design spaces, highlighting the need for further research in this area. These are challenges that Jan will address during his master project, building on our earlier work with Bruno Dzikowski in this area.

It was nice to see that there was broad interest in Jan’s presentation not only from the performance team at TNO-ESI, but also among some of its industry partners, as well as experts in performance prediction from University of Amsterdam, Eindhoven University of Technology, and University of Twente. That certainly made the Q&A session afterwards spicier, although Jan confidently answered most of the questions.  Congratulations Jan on work well done!

Bruno Dzikowski Successfully Defends Master’s Thesis on Performance Prediction

Posted on by Benny Akesson

Yesterday, Bruno Dzikowski successfully defended his master’s thesis titled “Practical Recommendations for Accurately Predicting Performance Degradation Caused by Memory Contention” The thesis addresses the challenge of predicting performance in microservice-based architectures for cyber-physical systems (CPS) running on multi-core platforms, where resource contention significantly impacts accuracy. Existing methods model interference sensitivity and contentiousness but lack practical implementation guidelines.

 

Bruno’s work introduces a compositional performance prediction framework with three key contributions: 1) a validated contentiousness profiling component, 2) an analysis of how system configuration affects prediction accuracy, and 3) the design and implementation of an experimental testbed. Tested across 195 co-location scenarios, the approach achieves high accuracy (median error ≈ 1.4%), demonstrating its effectiveness for forecasting microservice performance.

We are very proud of the excellent research Bruno conducted during his internship with TNO-ESI, which resulted in an outstanding thesis that was confidently presented and defended. We thanks Bruno for the excellent collaboration and wish him all the best for his future career.

Celebrating Dr. Panos Giannakopoulos’ Dissertation Defense

Posted on by Benny Akesson

Congratulations to the newly minted Dr. Panagiotis (Panos) Giannakopoulos, who has successfully defended his dissertation, “Predictable Application Performance in Resource Clusters” The dissertation tackles the challenge of meeting strict Round-Trip Time (RTT) deadlines for time-sensitive applications in heterogeneous, resource-constrained edge environments by developing lightweight, accurate performance predictors. These predictors leverage selected system metrics and machine learning models to anticipate execution time and variability, enabling proactive scheduling and load balancing that improve efficiency and reduce resource waste, with demonstrated success on Electron Microscopy workloads in Kubernetes-based clusters.

This research was conducted as part of the NWO ADAPTOR project, co-funded by Thermo Fisher Scientific and ASTRON. I have had the pleasure of serving on the user committee for this project over the past couple of years and was honored to be invited to join the Ph.D. committee. Over the years, Panos has presented his work at TNO-ESI several times in various settings and was also invited to share his insights at Thales. Panos will now continue his work as a postdoctoral researcher at TU/e.

Master Thesis Shines Light on Hardware Dimensioning for Cyber-Physical Systems

Posted on by Benny Akesson

On Wednesday, Marijn Vollaard defended his master thesis “Hardware Dimensioning for Microservice-based Cyber-Physical Systems: A Profiling and Performance Prediction Method” at the University of Amsterdam. This research has been supervised by Ben Pronk and myself as a part of a project with TNO-ESI.

The thesis addresses the problem of determining the number of homogeneous compute nodes needed for a particular variant of a cyber-physical system to meet its timing requirements. This is important in early discussions with customers and bidding processes, since it affects the size and cost of the resulting system. To this end, the thesis proposes a structured hardware dimensioning methodology comprising a profiling method and a performance prediction method. The four novel contributions of the thesis are: 1) A component-based profiling method, 2) a performance prediction method, 3) a structured hardware dimensioning methodology, and 4) validation of the approach, using a case study that represents a prototype of a CPS. Experimental evaluations on the case study show that the predicted performance differs from measurements on the application by at most 20%, which is satisfactory for hardware dimensioning decisions for new product variants.

The defense went well and Marijn confidently presented his story and convincingly answered the questions of the audience. The examination committee, impressed by his work, awarded his thesis a well-deserved grade of 8. As we bid farewell to Marijn, embarking on his next career adventure, we also extend our heartfelt congratulations. He certainly has much to be proud of. We wish him all the best on his travels and in his future pursuits.

Master’s Student Marijn Vollaard Shines with Study on Hardware Dimensioning for Microservice Applications in Cyber-Physical Systems

Posted on by Benny Akesson

Our master’s student, Marijn Vollaard, has achieved a significant milestone by completing and presenting his literature study titled “Hardware Dimensioning for Microservice Applications in Cyber-Physical Systems: Current Directions and Challenges” The study addresses the challenge of dimensioning the number of compute nodes required to meet the performance demands of microservice-based applications in cyber-physical systems. It thoroughly reviews an extensive body of literature on application and system profiling, performance prediction, and design-space exploration to establish the current state of knowledge in this field. The survey culminates in a discussion about how the surveyed literature applies to microservice applications, the cyber-physical systems context, and the problem of hardware dimensioning. Overall, this is a nice piece of work with a lot of references presented in a systematic way. Congratulations to Marijn for his great effort!”