It has been almost a year since we published the first survey-based study in the area of real-time systems at the Real-time Systems Symposium (RTSS) back in December 2020. The paper was entitled “An Empirical Survey-based Study into Industry Practice in Real-time Systems” and had the ambitious goal of addressing the gap between academic research and industry practice through an empirical survey-based study into industry practice. The survey had five objectives: 1) Establish whether timing predictability is of concern to the real-time embedded systems industry, 2) Identify relevant industrial problem contexts, including hardware platforms, middleware, and software, 3) Determine which methods and tools are used to achieve timing predictability, 4) Establish which techniques and tools are used to satisfy real-time requirements, and 5) Determine trends for future real-time systems. The survey was completed by 120 industry practitioners in the field of real-time embedded systems, which allowed interesting observations and insights to be made about the characteristics of the systems being developed today and important trends for the future.
Now, almost one year later, I am happy to announce that an extended version of our RTSS paper has been accepted for publication in the Real-time Systems journal. The title of the article is “A Comprehensive Survey of Industry Practice in Real-Time Systems“. The main extensions in the article include: 1) a discussion of potential threats to validity of the survey and its results, as well as the steps taken to mitigate them, 2) a statistical analysis and discussion of the results of the survey, in the context of its five objectives, 3) a discussion of the results of a quiz aimed at determining if the aggregate findings of the survey are common knowledge in the real-time systems community. In addition, more aggregated data from the survey has been released, allowing interested readers to further delve into the similarities and differences between the avionics, automotive, and consumer electronic domains.
It is my pleasure to announce our project proposal entitled “Zero-Waste Computing: Energy Labels for Digital Services” has been granted for the Science and Design PhD program at the University of Amsterdam. Ana Lucia Varbanescu is the main applicant for this project, with Anuj Pathania and myself as co-applicants. The project proposal was supported by Surf, ESI (TNO), Barcelona Supercomputing Center, and ASTRON.
The project addresses the issue that digital services are getting increasingly prevalent in society and are vital to the Dutch economy, already reaching 60% of GDP. However, they come with a significant, rapidly-increasing energy cost, raising sustainability concerns, since a mid-size datacenter alone consumes as much energy as a small town. However, datacenters are only the final link in a digital chain. Users interacting with devices — mobile phones, tablets, or laptops — trigger entire digital chains, combining multiple communicating computing layers and data transfers: from the device itself, through the edge, to the datacenter. Each layer has its own computing infrastructure (see figure). At each layer, decisions are made about how, where and when applications are running and/or data are transferred. These decisions have a significant impact on the user-perceived quality-of-service (QoS), but also on the energy consumption – per layer, and for the entire digital chain. The energy footprint of different devices along the chain might be known, but the actual energy consumed by the application is unknown, because it depends on infrastructure choices, and on user QoS requirements, and on mapping decisions made on the edge and in the datacenter. Thus, the energy efficiency, i.e., the amount of energy consumed to perform the actual task at hand, is largely unknown, for most digital chains.
We argue that the first step to reduce waste in computing is to quantify the energy efficiency of end-to-end digital chains. Our project focuses on designing an integrated framework (i.e., the methods, metrics, and tools) for this quantification effort. Specifically, we aim to define a reference architecture of digital chains, use it to define an analytical digital-chain energy-efficiency model that exposes the factors that impact energy efficiency along the chain, and support it with a high-level functional simulator to assess different operational scenarios and parameters that affect the energy efficiency of digital chains.
This is a small project funding only a single PhD student. More momentum is required to further advance this area and make a step from only monitoring the energy consumption of digital chains to also include actuation, e.g. energy minimization through workload redistribution, subject to performance constraints. We are currently looking for interested parties to collaborate with us on this topic in future project proposals.
ESI (TNO) was featured in the latest episode (Season 4 Episode 1) of Nederland Maakt Het, a program on RTL Z about Dutch organizations that develop of apply innovative technologies. In the segment, Wouter Leibbrandt, the Research and Operations director at ESI, explains that the Netherlands has a powerful high-tech industry, which is important to its competitiveness and earning power. To stay at the top and continue to develop excellent products in light of increasing system complexity, it is important to invest in research and development of new design methodologies. Big high-tech companies do this in an open innovation environment to address the challenges they face together. ESI is the applied research organization and knowledge partner that brings the industry and academic parties together into an eco-system to facilitate this.
In my role as part-time professor at UvA, I explain my view on open innovation and how universities contribute and get value from the eco-system. In the Embedded Software and Systems course at the University of Amsterdam, which is an academic partner of ESI since 2021, I discuss the increasing system complexity with my students and teach model-based engineering methodologies to help them address this challenge. I also supervise students that want to contribute to solving the complexity problem by doing their thesis project in with ESI or in industry. Lastly, Hein Otto Folkerts, the (former) head of Research at ASML, provides the industry view and explains the value of open innovation to ASML, one of the big high-tech companies in the Eindhoven region.
For those of you that missed the episode, it is available for online viewing on RTL XL. The segment about ESI starts at 14m30s and last for about 4 minutes. ESI also has a version of this segment in its own house style that is used for promotional purposes. This version is available here:
Today was the official project kick-off for the research project “Design Space Exploration 2.0: Towards Optimal Design of Complex, Distributed Cyber Physical Systems”. This project is a part of the Partnership Program Mastering Complexity (MasCot), funded by NWO Domain Applied and Engineering Sciences (AES) together with ESI (TNO). The University of Amsterdam and Leiden University are the academic partners, spearheaded by Andy Pimentel and Todor Stefanov. The carrying industrial partner is ASML, but with Philips, Siemens and ESI as parts of the user committee.
The main goal of the project is to extend existing methods for design-space exploration, often developed for on-chip systems, to cover complex distributed cyber-physical systems (dCPS), such as the lithography machines made by ASML. Designers of such systems need quick answers to so-called “what-if” questions with respect to possible design decisions/choices and their consequences on non-functional properties, such as system performance and cost. This calls for efficient and scalable system level design space exploration (DSE) methods that integrate appropriate application workload and system architectures models, simulation and optimization techniques, as well as supporting tools to facilitate the exploration of a wide range of design decisions. However, such DSE technology for complex dCPS does currently not exist. This projects hence tries to answer the question of how perform efficient and effective DSE for complex, distributed cyber-physical systems.
In today’s kick-off meeting, all stakeholders in the project had an opportunity to introduce themselves and refamiliarize themselves with the project and its goals. The two PhD students who will be working on the project, Marius and Faezeh, from UvA and Leiden, respectively, also gave a brief overview of the work they had done in the first three months of the project, which included a literature review and generation of high-level simulation models for different parameter settings.
I am directly involved in this project through my part-time appointment at UvA. As Marius’ second promotor, I will help him on his journey towards a PhD. I also have an interest in this project as an ESI Research Fellow and part of the MasCot Core Team. In this capacity, I am happy to help linking this project to ESI’s applied research projects, in particular at ASML, to exploit possible synergies, and to stimulate exchanges with other projects in the MasCot program.
ESI has just made a press release to announce that both Thales and the University of Amsterdam (UvA) has joined as partners in its open-innovation ecosystem. ESI’s ecosystem, based on open innovation, plays an important role in maintaining the leading competitive position of the Dutch high-tech industry. Together with universities and partner companies, ESI develops methodologies and tooling that are in line with the vision and needs of the high-tech industry, making use of the latest insights from universities. In an industry-as-a-lab setting, system engineering methodologies are developed, tested and validated on site at and with partners.
With the addition of UvA and Thales, ESI’s ecosystem now has more industrial and academic partners than ever before, which shows great promise in difficult times. Personally, I am very happy to see that the university where I work decided to further invest in its collaboration with ESI and join the partner board. Similarly, Thales is the company I have worked with in applied research projects for the past five years, and it pleases me that they see the benefits of this collaboration.
The fall semester of the very special academic year 2020/2021 is over. Most of the students following the Master of Software Engineering program at the University of Amsterdam have just completed my course Embedded Software and Systems (ESS). The ESS course had changed in a three important ways this year.
Firstly, a generic lecture about Petri Nets was changed to a series about two lectures, explaining how Petri Nets can be used to model and analyze software interfaces and components. Part of the material for this course was reused from the course Modelling and Analysis of Component-based Systems (MOANA-CBS), developed together with Thales targeting an industrial audience. These new lectures also prime students nicely for a lecture about the DYNAMICS project, a research collaboration between ESI and Thales. This allows me to show how these models and analyses can be used in practice to address problems related to software evolution by detecting incompatibilities and generating adapters when updating software interfaces. A generic lecture about the data-flow model of computation was removed to create room for this new material, but I am happy to teach fewer modelling formalisms and have more time to go in depth and show how they can be used to solve industrial problems. A nice result of this change to the course is that three master students have accepted thesis projects in the area of modelling and analysis of software components and interfaces in collaboration with ESI under the supervision of myself and my colleague Debjyoti Bera.
Secondly, the course project was redeveloped this year. Previously, students used Mathworks Stateflow to program Lego Mindstorm EV3 rovers to follow a line, avoid obstacles, and count objects. However, this project felt a bit too much like a toy and there were technical problems with both rovers and tools that were hard to overcome and limited the education experience. In particular, it was not easily possible to see or influence how code was generated for the Lego Mindstorm robots, which felt like a missed opportunity when teaching model-based engineering.
Two bachelor students did their theses in spring to evaluate the suitability of using the TurtleBot3 Burger robot, both in reality and in simulation using Gazebo, in the course. In addition, Stateflow was exchanged for Yakindu Statechart Tools, which is easier to use and gives us the flexibility we need in code generation. The new application developed in the project is to use Yakindu to program the TurtleBot to autonomously drive through a maze and map it.
Lastly, the COVID-19 pandemic required the entire course to be taught online. As a result, used a blended learning approach and prerecorded the lectures so that the students could watch them when they wanted to. Online interactive sessions were added to the course where the students could ask questions about the lectures, and participate in quizzes and group discussions. Online teaching meant that the students did not have access to the four physical TurtleBots that we had purchased. Luckily, the newly developed course project could be done with simulations in Gazebo. Below is a demo from one of the groups that very successfully solved the assignment.
The ESS course is continuously evolving and maturing and next year will be no different. Most importantly, we hope that the pandemic will be over by then and that we can put our three physical TurtleBots to good use.
It is my great pleasure to announce that our paper “An Empirical Survey-based Study into Industry Practice in Real-time Systems” has appeared at the 41st IEEE Real-Time Systems Symposium (RTSS). The paper presents results and observations from a survey of 120 industry practitioners in the field of real-time embedded systems. The survey provides insights into the characteristics of the systems being developed today and identifies important trends for the future. The survey aims to inform both academics and practitioners, helping to avoid divergence between industry practice and fundamental academic research.
This work is a dear pet project of mine that has been a long time in the making. Once I joined ESI (TNO), I started reading papers and attending conferences in the modelling community. I came across empirical survey-based research that systematically investigated industry trends, needs and practices, and that studied adoption and perceived benefits and drawbacks of different technologies and methodologies. I immediately found this line of work incredibly useful as it elevated my understanding of what happened in industry from a collection of anecdotes based on conversations with a few people in a few companies to something that could capture the experience of hundreds of people across industrial domains. I also had the feeling that this line of work provided all the citations I needed for the introduction of my papers, as it helped me position my own work on modelling in a broader industrial reality.
Empirical research is an established research direction in social science, but also in technical fields, such as software engineering and to a lesser extent system engineering. However, there was no work like this in the area of real-time systems. I decided to change this and pitched the idea to Rob Davis, Mitra Nasri, and Geoffrey Nelissen and Sebastian Altmeyer during a meeting in Amsterdam in May 2019. They substantially improved on my ideas and did a lot of very good work and almost a year and a half later, the paper is available for you to read. We could not fit everything we had to say into the RTSS paper, so there is also a supporting technical note entitled “A Comprehensive Survey of Industry Practice in Real-time Systems“.
RTSS Session
A separate session was dedicated to this work on the last day of RTSS 2020. The session began with a 25 minute paper presentation, which is available here:
The paper presentation was followed by a panel discussion involving three industry practitioners from the three main industrial domains covered by the survey: Marcelo Lopez Ruiz (Microsoft), representing the consumer electronics industry, Simon Schliecker (Volkswagen), representing automotive, and Stephen Law (Rolls-Royce), providing an avionics perspective. The panel discussed four key questions relating to the survey results:
Q1. What important characteristics of real-time systems highlighted in the survey results are the most relevant with respect to your industry? And what other important characteristics are there that were not picked up by the survey?
Q2. What are the most relevant trends in real-time systems development in your industry now, and looking ahead over the next 10 years?
Q3. Did anything surprise you in the survey and its results? And why?
Q4. Given the results of the survey, and your own experience, what recommendations would you to give to the academic community? Which areas should we work more or less on? What assumptions should we make or not make?
The opening statements from the panelists related to the four questions was pre-recorded and followed by a live discussion. The pre-recorded part of the panel is available here:
The session finished after one hour, before there was time to take questions from the audience. A separate Zoom room was created for this purpose and to allow the interaction to continue, which it did for another hour! We were very pleased with the interest in this paper and in the session.
Emerging Research Direction
I hope that this work is the first of many empirical research papers in real-time systems. There are many ways to continue with this line of work. First of all, others need to replicate our results to validate that they hold for different populations. For this purpose, we will be happy to transfer the survey we made on SurveyMonkey, such that it can be reused. Secondly, our survey was very broad and covers real-time systems across many application domains. More specific questions could be obtained if the focus was on a single domain, although the main challenge will be finding enough representative participants with a narrow focus. Thirdly, surveys are only one way of conducting empirical research. Another method sometimes used in software engineering is to use interviews, allowing more in-depth questions to be asked. However, the drawback of this method is that it is more time consuming to interview are large number of participants and to encode and analyze the results.
This direction in real-time system research is just emerging and we hope it will grow and become a well-established part of the research conducted in the community. This would help us better understand the industry we are trying to serve and help us close the gap between academic research and industry practice. A first important step is that this direction is recognized by all main conferences and journals in the area of real-time systems and explicitly included in the call for papers. You can play an important part here by helping us communicate the value of empirical research to others in our community and beyond.
It is my great pleasure to announce that I have received my University Teaching Qualification (UTQ), essentially my license to teach. The UTQ (Basiskwalificatie Onderwijs, BKO in Dutch) is a proof of teaching skills for university teaching staff, which allows you to demonstrate a proven ability to develop and teach courses at university level. It is recognized by all Dutch universities.
One of the most important things I have learned during the UTQ trajectory is the constructive alignment method for course design. I already knew that aligning goals, execution, and assessment was an important part of course design, yet challenging to do in practice. I appreciate that the constructive alignment workshops not only communicated the importance of this alignment, but also provided the tools (e.g. specification tables and tips for assessment) to identify and address misalignments. Constructive alignment may not be the only way to approach course design, but I see it as a very useful method and a good starting point from which further exploration can be done.
One of my stated goals was to learn how to make my lectures more interactive. This is something I knew was important to further improve my teaching, but I did not quite know how to achieve. This was addressed by the workshop on activating teaching that came paired with the excellent list of 50 Classroom Assessment Teachniques (CATs) by Angelo and Cross. Based on this material, I have now significantly increased the level of interaction and formative assessment in my course Embedded Software and Systems.
Lastly, the module about organizing teaching was very important to lift my view from my course and start looking more at the program that provides the context, in my case the Master of Software Engineering program. Being aware of this context not only makes my course better, but also the program, resulting in a better educational experience for the students. Interviewing people in the program for this module also provided a trigger to go talk to colleagues and build a network that will help both my research and education at UvA going forward.
In this short two minute presentation, I introduce myself and my fundamental and academic research into design methodologies for cyber-physical systems. I sketch a high-level view of the problem and outline a direction based on model-based engineering in which my previous work into domain-specific languages and analysis non-functional behavior fits. For a more elaborate description of my research, please have a look at my research page.
The paper addresses the problem of satisfying real-time requirements in industrial systems using unpredictable hardware and software, which limit or entirely prevent the application of established real-time analysis techniques. To this end, we propose PReGO, a generative methodology for satisfying real-time requirements in industrial commercial-off-the-shelf (COTS) systems. We report on our experience in applying PReGO to a use-case: a Search & Rescue application running on a fixed-wing drone with COTS components, including an NVIDIA Jetson board and a stock Ubuntu/Linux. We empirically evaluate the impact of each integration step and demonstrate the effectiveness of our methodology in meeting real-time application requirements in terms of deadline misses and energy consumption.
