Proposal Building Dutch Real-time Systems Community Granted

Recently, I submitted a small proposal worth 5K euro to the 4TU.NIRICT Call Community Funding together with Mitra Nasri and Geoffrey Nelissen, both from Eindhoven University of Technology, and Kuan-Hsun Chen from University of Twente. The purpose of the proposal was for creating a Dutch Real-time Systems community and stimulate collaboration both nationally and at the European level. Earlier this week, we were notified that the proposal was granted!

We plan to use the funding for building a Dutch real-time systems community by organizing a workshop in the Netherlands with several invited speakers (around 6) from other European countries, followed by a consolidation event after 3 months. The duration of the workshop will two days, and the target audience is the domain researchers affiliated with the 4TU and UvA. On each workshop day, there will be keynotes, rapid pitch talks, interactive panels, reviews of funding opportunities, and social meetings. The one-day consolidation event, (e.g., three months later) will focus on strengthening the Dutch real-time system community vision and on consolidating the initiatives planned at the workshop.

I look forward to working with Mitra, Geoffrey, and Kuan to organize a strong real-time systems community in the Netherlands through this grant, and through other means.

 

Article about Survey-based Study into Industry Practice in Real-time Systems in Real-time Systems Journal

It has been almost a year since we published the first survey-based study in the area of real-time systems at the Real-time Systems Symposium (RTSS) back in December 2020. The paper was entitled “An Empirical Survey-based Study into Industry Practice in Real-time Systems” and had the ambitious goal of addressing the gap between academic research and industry practice through an empirical survey-based study into industry practice. The survey had five objectives: 1) Establish whether timing predictability is of concern to the real-time embedded systems industry, 2) Identify relevant industrial problem contexts, including hardware platforms, middleware, and software, 3) Determine which methods and tools are used to achieve timing predictability, 4) Establish which techniques and tools are used to satisfy real-time requirements, and 5) Determine trends for future real-time systems. The survey was completed by 120 industry practitioners in the field of real-time embedded systems, which allowed interesting observations and insights to be made about the characteristics of the systems being developed today and important trends for the future.

Now, almost one year later, I am happy to announce that an extended version of our RTSS paper has been accepted for publication in the Real-time Systems journal. The title of the article is “A Comprehensive Survey of Industry Practice in Real-Time Systems“. The main extensions in the article include: 1) a discussion of potential threats to validity of the survey and its results, as well as the steps taken to mitigate them, 2) a statistical analysis and discussion of the results of the survey, in the context of its five objectives, 3) a discussion of the results of a quiz aimed at determining if the aggregate findings of the survey are common knowledge in the real-time systems community. In addition, more aggregated data from the survey has been released, allowing interested readers to further delve into the similarities and differences between the avionics, automotive, and consumer electronic domains.

We hope that you enjoy the article!

An Empirical Survey-based Study into Industry Practice in Real-time Systems @ RTSS 2020

It is my great pleasure to announce that our paper “An Empirical Survey-based Study into Industry Practice in Real-time Systems” has appeared at the 41st IEEE Real-Time Systems Symposium (RTSS). The paper presents results and observations from a survey of 120 industry practitioners in the field of real-time embedded systems. The survey provides insights into the characteristics of the systems being developed today and identifies important trends for the future. The survey aims to inform both academics and practitioners, helping to avoid divergence between industry practice and fundamental academic research.

This work is a dear pet project of mine that has been a long time in the making. Once I joined ESI (TNO), I started reading papers and attending conferences in the modelling community. I came across empirical survey-based research that systematically investigated industry trends, needs and practices, and that studied adoption and perceived benefits and drawbacks of different technologies and methodologies. I immediately found this line of work incredibly useful as it elevated my understanding of what happened in industry from a collection of anecdotes based on conversations with a few people in a few companies to something that could capture the experience of hundreds of people across industrial domains. I also had the feeling that this line of work provided all the citations I needed for the introduction of my papers, as it helped me position my own work on modelling in a broader industrial reality.

Empirical research is an established research direction in social science, but also in technical fields, such as software engineering and to a lesser extent system engineering. However, there was no work like this in the area of real-time systems. I decided to change this and pitched the idea to Rob Davis, Mitra Nasri, and Geoffrey Nelissen and Sebastian Altmeyer during a meeting in Amsterdam in May 2019. They substantially improved on my ideas and did a lot of very good work and almost a year and a half later, the paper is available for you to read. We could not fit everything we had to say into the RTSS paper, so there is also a supporting technical note entitled “A Comprehensive Survey of Industry Practice in Real-time Systems“.

RTSS Session

A separate session was dedicated to this work on the last day of RTSS 2020. The session began with a 25 minute paper presentation, which is available here:

The paper presentation was followed by a panel discussion involving three industry practitioners from the three main industrial domains covered by the survey: Marcelo Lopez Ruiz (Microsoft), representing the consumer electronics industry, Simon Schliecker (Volkswagen), representing automotive, and Stephen Law (Rolls-Royce), providing an avionics perspective. The panel discussed four key questions relating to the survey results:

Q1. What important characteristics of real-time systems highlighted in the survey results are the most relevant with respect to your industry? And what other important characteristics are there that were not picked up by the survey?

Q2. What are the most relevant trends in real-time systems development in your industry now, and looking ahead over the next 10 years?

Q3.  Did anything surprise you in the survey and its results? And why?

Q4. Given the results of the survey, and your own experience, what recommendations would you to give to the academic community? Which areas should we work more or less on? What assumptions should we make or not make?

The opening statements from the panelists related to the four questions was pre-recorded and followed by a live discussion. The pre-recorded part of the panel is available here:

The session finished after one hour, before there was time to take questions from the audience. A separate Zoom room was created for this purpose and to allow the interaction to continue, which it did for another hour! We were very pleased with the interest in this paper and in the session.

Emerging Research Direction

I hope that this work is the first of many empirical research papers in real-time systems. There are many ways to continue with this line of work. First of all, others need to replicate our results to validate that they hold for different populations. For this purpose, we will be happy to transfer the survey we made on SurveyMonkey, such that it can be reused. Secondly, our survey was very broad and covers real-time systems across many application domains. More specific questions could be obtained if the focus was on a single domain, although the main challenge will be finding enough representative participants with a narrow focus. Thirdly, surveys are only one way of conducting empirical research. Another method sometimes used in software engineering is to use interviews, allowing more in-depth questions to be asked. However, the drawback of this method is that it is more time consuming to interview are large number of participants and to encode and analyze the results.

This direction in real-time system research is just emerging and we hope it will grow and become a well-established part of the research conducted in the community. This would help us better understand the industry we are trying to serve and help us close the gap between academic research and industry practice. A first important step is that this direction is recognized by all main conferences and journals in the area of real-time systems and explicitly included in the call for papers. You can play an important part here by helping us communicate the value of empirical research to others in our community and beyond.

Another Article in Real-time Systems Journal

An article entitled “Response Time Analysis of Multiframe Mixed-Criticality Systems with Arbitrary Deadlines” has been accepted for publication in Real-time Systems journal. This work is first authored by Ishfaq Hussain and is another collaboration with my former colleagues at CISTER. The article extends our RTNS 2019 paper “Response Time Analysis of Multiframe Mixed-Criticality Systems” that received both an Outstanding Paper Award and a Best Student Paper Award. The RTNS paper presented a schedulability analysis for the multi-frame mixed-criticality model, extending the static and dynamic analysis techniques for mixed-criticality scheduling and the schedulability analysis for multi-frame task systems.

The accepted journal article extends the RTNS paper by generalizing the proposed schedulability analyses from a constrained-deadline task model to the more general, but also more complex, model with arbitrary deadlines. The corresponding optimal priority assignment for our schedulability analysis is also identified. In experiments with synthetic workloads, the proposed analyses are compared in terms of scheduling success ratio, against the frame-agnostic analyses for the corresponding variants of the Vestal model.

Paper Accepted at GPCE 2020

A paper entitled “PReGO: a Generative Methodology for Satisfying Real-Time Requirements on COTS-based Systems – Definition and Experience Report” was accepted for publication at the 19th International Conference on Generative Programming: Concepts & Experiences (GPCE). This is my first paper in collaboration with my new colleagues at the University of Amsterdam, and it describes work done in the TeamPlay project, a three-year research project funded by the EU Horizon 2020 research and innovation programme.

The paper addresses the problem of satisfying real-time requirements in industrial systems using unpredictable hardware and software, which limit or entirely prevent the application of established real-time analysis techniques. To this end, we propose PReGO, a generative methodology for satisfying real-time requirements in industrial commercial-off-the-shelf (COTS) systems. We report on our experience in applying PReGO to a use-case: a Search & Rescue application running on a fixed-wing drone with COTS components, including an NVIDIA Jetson board and a stock Ubuntu/Linux. We empirically evaluate the impact of each integration step and demonstrate the effectiveness of our methodology in meeting real-time application requirements in terms of deadline misses and energy consumption.

Journal Article Presented at ECRTS 2019

Today, Ali presented our Real-time Systems article “Uneven Memory Regulation for Scheduling IMA Applications on Multi-core Platforms” in the Journal-to-conference (J2C) session at ECRTS.

This article addresses the problem of resource sharing in mixed-criticality systems through temporal isolation by extending the state-of-the-art Single-Core Equivalence (SCE) framework in three ways: 1) we extend the theoretical toolkit for the SCE framework by considering EDF and server-based scheduling, instead of partitioned fixed-priority scheduling, 2) we support uneven memory access budgets on a per-server basis, rather than just on a per-core basis, and 3) we formulate an Integer-Linear Programming Model (ILP) guaranteed to find a feasible mapping of a given set of servers to processors, including their execution time and memory access budgets, if such a mapping exists. Our experiments with synthetic task sets confirm that considerable improvement in schedulability can result from the use of per-server memory access budgets under the SCE framework.

Overall, I greatly appreciate that key conferences in the real-time community are starting to allow journal articles to be presented. This increases the exposure of these works that are often longer and better edited. It is also helpful for researchers at the institutes where conference publications are not considered a relevant KPI. You can argue the validity of this reasoning in areas of computer science where conferences are highly competitive with 20-30% acceptance rates, but it is reality for some researchers. An interesting thing with the MODELS conference is that they collaborate with the SOSYM journal such that some accepted articles in the journal gets a full slot at the conference. This is a nice way to highlight good articles and to appreciate the work done by both authors and reviewers.

Accepted Article about Dynamic Command Scheduling in Real-Time Systems Journal

Today, we congratulate Yonghui Li on his first accepted journal article. The article is entitled “Architecture and Analysis of a Dynamically-Scheduled Real-Time Memory Controller” and has been accepted in the Real-Time Systems journal. The work extends his paper “Dynamic Command Scheduling for Real-Time Memory Controllers” that was presented at ECRTS 2014. The previous conference paper introduced a back-end architecture and scheduling algorithm for a dynamically scheduled SDRAM controller supporting variable transaction sizes and different degrees of bank interleaving. The properties of the back-end was extensively analyzed and worst-case execution times (WCET) of scheduled transactions was derived using two different methods with varying complexity and accuracy.

The newly accepted article extends this work by proposing a corresponding memory controller front-end, along with a complete response time analysis for memory transactions of variable sizes. A key feature of the front-end is that it features a non-work-conserving TDM arbiter, which provides static information about the order in which transactions of different sizes are scheduled, allowing the response time analysis to leverage the flexible WCET analysis of the back-end to provide tighter bounds. In addition, it is shown in which order memory clients with different request sizes should be served to minimize the total response time. The results demonstrate that dynamic command scheduling significantly outperforms our semi-static (pattern-based) approach in the average case, while it performs equally well or better in the worst-case with only a few exceptions.

Article Accepted in Real-Time Systems Journal

A journal article entitled “A Framework for Memory Contention Analysis in Multi-Core Platforms” has been accepted for publication in Real-Time Systems. This article is a collaboration with Dakshina Dasari and Vincent Nelis and is a result from the time I spent with the CISTER-ISEP Research Unit in Porto.

The article proposes a unified framework to bound memory interference in multi-core platforms for a variety of different arbiters, such as time-division multiplexing (TDM), fixed priority, and an unspecified work-conserving arbiter. Our framework clearly demarcates the arbiter-dependent and independent stages in the analysis of interference. The arbiter-dependent phase takes the arbiter and the task memory-traffic pattern as inputs and produces a model of the availability of the bus to a given task. Then, based on the availability of the bus, the arbiter-independent phase determines the worst-case request-release scenario that maximizes the interference experienced by the tasks due to memory contention. We experimentally evaluate the quality of the analysis by comparison with a state-of-the-art TDM analysis approach and consistently showing a considerable reduction in maximum interference.

Article in Real-Time Systems Journal has Appeared

A journal article entitled “Unified overhead-aware schedulability analysis for slot-based task-splitting” has appeared in Real-Time Systems Journal. This article was first-authored by Paulo Baltarejo Sousa during my time at CISTER-ISEP Research Unit in Porto, Portugal and is the result of a collaboration from that time.

The main contribution of the article is a unified scheduling theory for two state-of-the-art slot-based semi-partitioned algorithms, S-EKG and NPS-F. This new theory is based on exact schedulability tests, thus also overcoming many sources of pessimism in existing analyses. Another benefit of the proposed analysis is that it captures overheads, such as interrupts, context switches, and caches, occurring when tasks are deployed on real multi-core platforms. Together, these advantages results in a new efficient and reliable schedulability analysis for slot-based task-splitting algorithms.