Eclipse ESCET 2nd community meeting: minutes of the meeting
Eclipse ESCET community meeting #2 – minutes of the meeting
Date/time: September 26, 2023, 9:00 – 12:00 CEST (UTC+2)
The Eclipse Supervisory Control Engineering Toolkit (Eclipse ESCET™) project provides a model-based approach and toolkit for the development of supervisory controllers. It supports Synthesis-Based Engineering (SBE), an engineering approach to design and implement supervisory controllers that combines model-based engineering with computer-aided design to produce correct-by-construction controllers.
The theme of this second community meeting was the Eclipse ESCET ecosystem. We had presentations from various members of the community, that show both increasing maturity of the technology and interesting new directions that are being explored. A significant part of the meeting was dedicated to discussing ways to further mature and grow the ecosystem, and to expand collaboration.
Following up on my first impression, my conclusion from the meeting is: It was a very vibrant meeting, with a lot of enthusiasm and constructive discussions. The participants were impressed by the presentations, and the progress that was made compared to the previous community meeting. They see the momentum, with more and more collaborations, as well as industrial commitment. The participants are convinced of the value of SBE, as it gives gives competitive advantages over the current way of working, and continuing with the current way of working is not sustainable towards the future. The need to take next steps was also recognized. The proposal to set up ecosystem orchestration, to mature and grow the ecosystem, by making a shared roadmap and jointly executing it, was accepted. The next steps will be started still this year.
Kind regards,
Dennis Hendriks
Eclipse ESCET project lead
Below are more detailed minutes per presentation:
Welcome / Introduction to SBE and update on ESCET
- Slides: Welcome, introduction to SBE, and update on ESCET.pdf
- Presenter: Dennis Hendriks, Eclipse ESCET project lead, TNO-ESI / Radboud University
- Summary: Dennis welcomed everyone to the community meeting. He first presented a short general introduction to SBE and the Eclipse ESCET project. He then shared some general news from the project, including some general announcements, new features from recent releases, and information about the current state of the project’s developer ecosystem.
-
Discussion/questions:
- The Eclipse ESCET project has graduated. What does that mean? → The project has left the ‘incubation’ phase, becoming a ‘mature’ Eclipse Foundation open source project. The graduation review by the Eclipse Foundation focused on three parts: 1) is the tool ‘Eclipse quality’; 2) does the project follow the relevant processes, and does it adhere to the Eclipse Foundation’s principles of openness, transparency and meritocracy; 3) is the project’s ecosystem growing, and does the project actively seek to grow the community.
Synthesis scalability improvements
- Slides: Synthesis scalability improvements.pdf
- Presenter: Dennis Hendriks, Eclipse ESCET project lead, TNO-ESI / Radboud University
- Summary: Scalability of synthesis for complex models was identified during the previous community meeting as a challenge. TNO conducted a short evaluation of the performance improvements for CIF data-based synthesis since the previous community meeting, using the 21 benchmark models shipped with Eclipse ESCET v0.10. Out of the box, using default settings, memory was reduced by 18.5 times and time by 96.9 times, with improvements up to 244.8 times reduction in memory and 1515.3 times in time for some models. This can be improved even further by using non-default settings. Overall, performance has improved quite a bit, but it does depend on the model. Next steps are being considered.
-
Discussion/questions:
- Should users model differently to prevent scalability issues, or can the tool do it itself? → Some patterns are already rewritten in the tool (e.g., state requirements). The CIF documentation lists some modeling guidelines as well. More could be done for both.
- Does the tool give hints or warnings for models that may not be optimal in terms of synthesis performance? → The CIF text editor does some static checks. The CIF data-based synthesis tool performs several checks before the actual synthesis. The tool documentation lists all performance-related options and how to use them, and gives further hints. You can also use Dependency Structure Matrices (DSMs) to analyze the dependencies between plants and requirements in your model, to understand the structure and improve upon it. The multi-level synthesis tool (currently in development) uses DSMs under the hood, to structure the system into a tree and more efficiently synthesize supervisors for the different parts of the system.
- You brute-forced all the example models to find the best settings. How would that work in practice on arbitrary models? → We use the benchmark models to see what works best. For instance, we added a lot of advanced variable ordering options. We then experimented, and updated the default settings. Hence, the large out-of-the-box improvements. We keep trying to make more settings work best out of the box, if possible. For specific models, you can configure advanced settings and tune heuristics to get even better performance, but this requires some knowledge and experience.
Configure-to-order models for synthesis of waterway lock supervisors
- Slides: Configure-to-order models for synthesis of waterway lock supervisors.pdf
- Presenter: Asia van de Mortel-Fronczak, TU/e (substituting for Marzhan Baubekova)
- Summary: RWS and TU/e developed an ‘assembler’ for waterway locks. From a selected system configuration it automatically generates CIF models and visualizations. It also automatically synthesizes a supervisor and generates a simulation. It has been applied at Rijkswaterstaat, and helps to make SBE available to users with less knowledge and expertise. It also supports early validation, assists in operator training, and reduces efforts and costs.
-
Discussion/questions:
- What is monolithic synthesis? → With monolithic synthesis we synthesize a single supervisor from the plants and requirements (the synthesis inputs). With non-monolithic synthesis approaches, we synthesize multiple supervisors for parts of the system, from the same input.
- How did you do the validation, to see whether you improved the KPIs? → A practical evaluation was performed during the project. The assembler is now being used in multiple projects at RWS. The response so far is that the time needed for validation of requirements, and communication with contractors and operators is reduced. It is expected that reuse of models that have been shown to work for earlier projects will also help subsequent projects.
- Since Marzhan finished her project, PLC code generation has also been added to the assembler.
Synthesis-Based Engineering of activity diagrams
- Slides: Synthesis-Based Engineering of activity diagrams.pdf
- Presenter: Pierre van de Laar, TNO-ESI
- Summary: The behavior of control software can be modeled in state machines, for instance in CIF. But, in other communities also UML activity diagrams are used. Pierre explained that this involves similar engineering challenges. In the Poka Yoke applied-research project, ASML, VDL-ETG and TNO are developing an approach for activity diagram synthesis, reusing SBE methods and tools from state machine synthesis. First results show this to be feasible. Within the multi-year project (2023-2026), they aim to develop an SBE approach suitable of synthesizing the control software for logistic systems. As carrier case, they use the wafer handler of ASML’s TWINSCAN systems, which is jointly being developed by ASML and VDL-ETG.
-
Discussion/questions:
- Is the ambition to apply this in practice, not just research? → Yes, the goal of the applied research project is to ultimately get it landed. But, it is a multi-year project, and it is now the first year, so it may take still 2 or 3 years, assuming the research will be successful.
- Can you validate that it is faster, by doing in parallel the old and new way of working? → Probably not, as that is rather expensive, but also the people that need to be involved would be the same people. However, we want to compare new projects against past projects. Similar quantitative evaluations were performed in the past with the introduction of model-driven engineering at ASML.
- Will it take years until you know the effects of introducing the new approach? → Just modeling the behavior, and finding deadlocks, is already useful. We will use an incremental approach, rather than introducing it all at the end of the project. However, embedding a new approach into a company takes time, as people have to learn to use the new way of working, so indeed it may take years until the full effects are visible.
- What algorithms do you use to optimize the sequencing in the activity diagrams? → Petrinet synthesis comes also with guarantees. From the state space of the controlled system, as synthesized by CIF, you get a minimal Petrinet with the same behavior (same possible sequences of actions). We want to carry over the guarantees from supervisory controller synthesis. But, this needs to be worked out further.
- Do you only support feed-forward sequences, or can there be rework? → There is a loop in the example. So yes, you can have rework.
- Can we model agents, cars and humans? → You could model the interface of the car, in terms of its state and the actions it has and how the actions influence the state. You add requirements, and you can synthesize a controller. You can reuse the actions with different requirements for different purposes. Or for different cars composed of different components. This is not different for activity diagram synthesis than it is for state machine synthesis.
Connecting synthesis to UML and SysML
- Slides: Connecting synthesis to UML and SysML.pdf
- Speaker: Wytse Oortwijn, TNO-ESI
- Summary: While CIF features synthesis possibilities, modeling in CIF is not (yet) an industry practice. This begs the question whether synthesis could be connected to, or integrated with, industry-standard UML/SysML modeling environments. TNO, Cordis and Rijkswaterstaat explore such options in the context of Cordis SUITE and Enterprise Architect, to lower the threshold for SBE adoption. Wytse showed the general idea for the integration of both worlds. He also showed the feasibility of the approach using two pilots performed together with Cordis and Additive Industries, using Cordis SUITE and CIF.
-
Discussion/questions:
- Is an automatic transformation available? → No, not yet. In the two pilots, the transformations were performed manually, but in a structured and mechanical way. We believe it can be automated, and don’t foresee problems there.
- How could you prove the transformations are correct? → Formal proofs are quite involved, but could be possible. A more practical approach would be to make a test set with all the different aspects being translated, and check that for those the semantics is preserved. Also, there is already a transformation from Cordis models to verification tool mCRL2, so we could compare the state spaces in mCRL2 and CIF, to increase the confidence.
- Are there rules for the Cordis models to be able to transform them? → Cordis models and CIF are quite similar. Both use state machines. They are relatively easy to map, and it is expected that no restrictions on Cordis models are needed.
- They look the same, but are they really semantically mappable? → We don’t have a proof. We only looked at the subset we needed for these pilots, to show feasibility. We have to work out the details. That is future work.
- Would it be easier to start from scratch in Cordis SUITE, or should we transform from Enterprise Architect, or should we start in CIF? → The proposal is to stay in a world that people are familiar with (MBE), such that they don’t need a lot of training and new knowledge. So, connect SBE to what is currently used, integrating it in existing MBE tools. Similar to the previous presentation, we adapt SBE to the current way of working, whether that is activity diagrams, Cordis or UML/SySML models. As mentioned, there are however open questions, like how to map the different perspectives, such as specifying ‘what’ vs ‘how’.
Application of SBE at Rijkswaterstaat
- Slides: Application of SBE at Rijkswaterstaat.pdf
- Speaker: Harry Lammeretz, RWS
- Summary: Rijkswaterstaat has been working with TU/e on SBE for over 7 years. Through several pilot projects, suitability of SBE is now being investigated. For a first type of application, developing and validating requirements for tendering, SBE will from now on be used for all waterway lock projects. RWS has formed an SBE team and is funding some ESCET development. Harry also came back to the questions asked during the previous community meeting, which were answered in a report by CGI and TNO. Harry concluded with an outlook to the future, including the wish for a larger ecosystem.
-
Discussion/questions:
- You mentioned that CGI indicates changes are needed for CIF. What should be changed? → The PLC code generator needs to ensure that requirements are traceable from the code back to the original models. Errors in code also need to be traceable back to the models. We are working on a new PLC code generator that addresses the concerns. The documents that TNO and CGI wrote, with all the recommendations, are public.
- Is synthesis scalable enough for RWS? Can more and more requirements be added? → Tunnels are the most difficult for RWS. It was at first not possible to synthesize a monolithic supervisor. But, with smart algorithms, exploiting for instance symmetry, it was possible to synthesize a monolithic supervisor it in a short time. See the PhD thesis of Lars Moorman for more information.
Discussion: Maturing and growing the Eclipse ESCET ecosystem
- Slides: Discussion - Maturing and growing the Eclipse ESCET ecosystem.pdf
- Presenter: Dennis Hendriks, Eclipse ESCET project lead, TNO-ESI / Radboud University
- Summary: Dennis first presented a proposal for the need for ecosystem orchestration, to mature the ecosystem, by among others developing a shared vision and roadmap, and fostering collaboration and growth. The focus is not only on methods and tools, but also on knowledge and the ecosystem. Dennis proposed a 3-phase plan, and asked the community for feedback.
-
Discussion/questions:
- The community recognizes the need for SBE. With growing complexity of systems in general, a large number of infrastructural objects needing to be replaced and renovated in the coming years, and the shortage on the labor market, continuing in the current way of working is not an option.
- The community was in agreement that there is indeed the need to work more together, and be less fragmented. There was also agreement that the ecosystem needs to grow, especially with more users, but also with more developers. A stable and sustainable ecosystem is essential. Having a shared vision and roadmap, and jointly executing it, is considered a good next step. Working on knowledge sharing and design skills training were also positively received. Various parties indicated, during the meeting and afterwards, that they’d like to be involved in the next steps.
- RWS is already working on a roadmap, but welcome a shared roadmap.
- There was quite some discussion on what other parties we should seek to be next users of SBE. The two domains that were the most prominent choices are the infrastructural domain (e.g., provinces, bigger municipalities, water boards, ProRail) and high tech industry (TU/e and TNO have many contacts). The financial domain was also mentioned. Some proposed that it may be good to have some focus, and not do everything at once.
- There was some discussion on whether the benefits of SBE should be explained to market parties so that they want to use it, or that they should be forced to use SBE by mandating it in for instance contracts (carrot or stick). Both could help, so maybe ‘and’ would be better than ‘or’. RWS indicates that it can’t prescribe a specific tool, but could maybe ask for SBE, or at least ask for a certain quality that would be easier to achieve with SBE. In a contractor/contractee relation, you ideally want both parties to be on board.
- To get other parties on board, several proposals were made, including: performing stakeholder analysis; showing parties already using SBE; sharing case studies and success stories, as concrete proof points; making the benefits tangles, fact-based, and quantitative; developing a good business case for industry, showing SBE can give better quality and/or lower costs; making SBE easier adoptable, e.g., by integrating into existing MBE tools; working on awareness, spreading the word, and sharing knowledge with industry; companies that use it already can support others in adopting it as well; bringing tool vendors, customers, and industry together; considering not just evangelists, but also the critics, and not just engineers and architects, but also management.
- The proposal of Dennis is accepted. Dennis will contact the participants of the community meeting, and check who wants to be involved in the next steps, and in what role.