escet merge requestshttps://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests2022-06-18T11:20:16Zhttps://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/333#196 Add node ordering algorithms, including dependencies, for later use by DCSH2022-06-18T11:20:16ZDennis Hendriks#196 Add node ordering algorithms, including dependencies, for later use by DCSHNow that we have graphs (!332), here are the algorithms that work on graphs:
* Rooted level structure computation.
* Pseudo-peripheral node finders.
* Node orderers.
* Variable orderers, based on the node orderers.
Note that:
* I tried ...Now that we have graphs (!332), here are the algorithms that work on graphs:
* Rooted level structure computation.
* Pseudo-peripheral node finders.
* Node orderers.
* Variable orderers, based on the node orderers.
Note that:
* I tried to stay as close as possible to the original papers. We thus have two pseudo-peripheral node finders, one for each of the node ordering algorithms, such that they use what they describe in their papers.
* Weighted Cuthill-McKee has been improved wrt the original paper. It now works also for multiple unconnected partitions with >1 node.
* These algorithms form the ingredients for the DCSH algorithm itself, to be contributed in a follow-up merge request.
Addresses #196v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/332#196 Add graphs as representation to VarOrdererHelper.2022-06-15T21:29:10ZDennis Hendriks#196 Add graphs as representation to VarOrdererHelper.The DCSH algorithm is based on multiple node ordering algorithms, working on graphs. This merge request adds graphs as a representation that can be used by data-based synthesis variable ordering algorithms.
For now, graphs are created f...The DCSH algorithm is based on multiple node ordering algorithms, working on graphs. This merge request adds graphs as a representation that can be used by data-based synthesis variable ordering algorithms.
For now, graphs are created from the existing hyper-edges:
- This ensures consistency between the representations.
- We could later opt for other ways to create the hyper-edges/graphs.
Also added some extra utility methods for use by variable ordering algorithms, to work with the various representations.
Addresses #196v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/328#374 Use default maximum available memory for product as determined by JVM2022-06-16T13:43:07ZDennis Hendriks#374 Use default maximum available memory for product as determined by JVM- For product and product runtime launch configuration.
- Also updated product documentation.
Closes #374- For product and product runtime launch configuration.
- Also updated product documentation.
Closes #374v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/327#196 Add DCSH variable ordering algorithm to data-based synthesis.2022-06-27T19:51:45ZDennis Hendriks#196 Add DCSH variable ordering algorithm to data-based synthesis.Variable ordering changes/additions:
- Prepared `VarOrdererHelper` for more metrics besides total span.
- Reduced space used for printing total span metric.
- Variable amount of space based on initial metric value.
- Added WES to `Va...Variable ordering changes/additions:
- Prepared `VarOrdererHelper` for more metrics besides total span.
- Reduced space used for printing total span metric.
- Variable amount of space based on initial metric value.
- Added WES to `VarOrdererHelper` as additional variable ordering metric.
Graph related additions:
- Added `Graph` and `Node` classes to represent graphs for data-based synthesis variable ordering algorithms to use.
- Added graph representation support to `VarOrdererHelper`.
- For now, graphs are created from the existing hyper-edges, ensuring consistency between the different representations.
- We could later opt for other ways to create the hyper-edges/graphs.
- Added extra utility methods for use by variable ordering algorithms.
- Updated `CifToSynthesisConverter` for the added graph representation.
DCSH algorithm related additions:
- Added rooted level structure constructor for graphs.
- Added pseudo-peripheral node finder algorithms.
- I tried to stay as close as possible to the original papers.
- We have multiple algorithms here.
- These are the ones which are needed for the node ordering heuristic algorithms.
- We could experiment later which ones work best. (see below)
- For now, node ordering algorithms use the ones from their corresponding papers.
- Added node ordering heuristic algorithms.
- I tried to stay as close as possible to the original papers.
- These are the ones we need for the DCSH variable ordering algorithm.
- Weighted Cuthill-McKee has been improved wrt the original paper: it works also for multiple unconnected partitions with >1 node.
- Added variable ordering algorithms.
- Added Sloan and Weighted Cuthill-McKee variable ordering algorithms, using the corresponding node ordering algorithms.
- Added 'choice' and 'reverse' wrapper variable ordering algorithms.
- Add DCSH variable ordering algorithm.
- Is for now based on graphs constructed from the existing hyper-edges, ensuring consistency between the different representtations.
- Could later create DMSs/hyper-edges based on the paper's approach.
Data-based synthesis tool additions:
- Added new option for enabling DCSH variable ordering.
- Is disabled by default, as I consider it experimental for now.
- The 'experimental' label here is for end users. It does not concern the implementation.
Data-based synthesis variable ordering documentation additions/changes:
- Added DCSH algorithm/option.
- Extended explanation of heuristic algorithms in general.
- Extended explanation of when algorithms are not applied.
- Extended descriptions of the FORCE and sliding window algorithms.
Data-based synthesis tests additions/changes:
- Renamed data-based synthesis test cases relating to `force`.
- The `force` tests were about FORCE and sliding window, so were wrongly named.
- Changed `_force_` to `_algos_`.
- Also extracted the related common options in the test ToolDef script to variables.
- Added DCSH to variable ordering integration tests, to be applied with all the other algorithms for the `algos` tests.
- Resulting metrics are the same or better for all these tests with DCSH added to it.
- Added single variable ordering algorithm integration tests, besides the ones that test all the algorithms at once.
- But only for a single initial random order, not for all the other initial orders, to avoid an explosion of tests.
Regarding reviewing:
- I tried to keep the stack of commits reviewable one by one, although there are a few fixes/changes at the end that could potentially have been integrated into earlier commits.
- Still, reviewing this per commit can be useful, as otherwise the totality of changes can be overwhelming, and the commits build towards the end result nicely.
I stopped here with this branch, as it is quite extensive already. The following are left as future work. I've created issues for the most relevant ones:
- #375 It would be good to have some scripts to benchmark the benchmark models, comparing for instance FORCE + sliding window against DCSH + FORCE + sliding window, in an automated fashion. I partly put something together for this already.
- #376 I think more testing is needed. I don't yet know whether the improvements we get in terms of better variable orderings and reduced synthesis effort match what the DCSH paper indicates. That would be good to check before lifting the 'experimental' label for end users to use it.
- The choice variable orderer could support reverse orders as well, natively, to avoid applying the algorithms twice. However, that leads to more code, and a less modular design. I hope execution times are very short already anyway. But, I still want to measure those times to be sure. Otherwise we may want to look into this optimization.
- #374 I found that our `product-escet` launch configuration is configured with only 4 GB of memory. That seems very low for todays systems.
- #377 Hyper-edge creation has a bug, where for location invariants actually the component invariants are used. I'm a bit hesitant to change this now though, as it makes comparing the performance of DCSH against the paper more difficult. Also, what if it reduces synthesis performance. We have no good benchmarking set yet to test this.
- #378 Data-based synthesis now only allows DCSH to be applied before FORCE and FORCE before sliding window. The order is fixed. I think the options should be generalized, to allow for instance `choice(dcsh, sequence(force, sliding-window(9)))` etc. This gives more flexibility for testing different configurations. Actual testing would require more benchmark models though.
- #379 There are several places where choices can be made. We can choose the current hyper-edges vs the ones defined in the DCSH paper. We can enable DCSH by default or not. We can choose per algorithm (DCSH, FORCE, sliding window), to base it on the total span or WES metric. For the node ordering algorithms, we can choose which pseudo-peripheral node finding algorithm to use. All of this requires that we can determine what is better, hence we need ways to specify these choices, scripts to easily test the differences, and enough benchmarking models to conclude what is really better in general.
- Variable ordering seems to lack cancellation checking. If the code runs for longer times (see other point), then we may need to add more of it. For now, this does not seem needed.
- #380 We currently use the original hyper-edges. We could implement the hyper-edges defined in the DCSH paper, which may perform better.
Closes #196v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/326#370 Generalized CIF to Supremica precondition checks to reusable checks.2022-06-13T15:25:41ZDennis Hendriks#370 Generalized CIF to Supremica precondition checks to reusable checks.Model walkers:
- `ModelWalkerGenerator` now also generates composite model walkers.
- Removed unused `rootClassName` argument/variable from `ModelWalkerGenerator`.
EMF code generators
- Removed 'v1x0x0' from JavaDoc.
- Small improvement...Model walkers:
- `ModelWalkerGenerator` now also generates composite model walkers.
- Removed unused `rootClassName` argument/variable from `ModelWalkerGenerator`.
EMF code generators
- Removed 'v1x0x0' from JavaDoc.
- Small improvements and alignments for EMF code generators.
CIF (pre)condition checkers:
- Added generic CIF (pre)condition check(er) functionality.
- Added various preconditions checks. Based on the ones used by CIF to Supremica. Some more general checks as well.
- Change `CifToSupremicaPreChecker` to use generic checker, and updated CIF to Supremica expected test output. Other checkers to be done in separate branches.
CIF common:
- Added `CifTextUtils.hasName/getNamedSelfOrAncestor`.
I recommend reviewing per commit.
Addresses #370v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/324#371 Improved CIF data-based synthesis variable ordering API2022-05-31T13:00:13ZDennis Hendriks#371 Improved CIF data-based synthesis variable ordering APIThere are quite a few changes in this branch:
* `AutoVarOrderer` is now `VarOrderer` and is an interface instead of a base class. It now longer forces a representation of the specification and internal storage decisions upon the actual a...There are quite a few changes in this branch:
* `AutoVarOrderer` is now `VarOrderer` and is an interface instead of a base class. It now longer forces a representation of the specification and internal storage decisions upon the actual algorithms. The initialization and cleanup is also gone. There is now just a single method that algorithms should implement.
* `VarOrderHelper` contains the hyper-edges, which is currently the only representation of the specification. More representations can be easily added to it.
* `VarOrderHelper` also contains various utility methods, moved from `AutoVarOrderer`.
* Moved all non-algorithm-specific functionality out of `AutoVarOrderer` to `CifToSynthesisConverter`. It now determines whether to apply algorithms or not, and indicates in debug output why it is skipped. Or also prints the general debug output, such as number of hyper-edges.
* The variable ordering algorithms (through the `VarOrderer` interface), no longer reorder the variables in `SynthesisAutomaton`. They just compute new orders for the variables. `CifToSynthesisConverter` applies the final order to `SynthesisAutomaton`. This keeps the 'original order' stable. Also, it makes it easier to implement a `ChoiceVarOrderer` that applies multiple algorithms to the same input order and chooses the best order produced by various algorithms.
* The `VarOrderer` interface has `dbgLevel` to allow arbitrary trees of algorithms. For instance, atomic algorithms such as FORCE and sliding window can now easily be combined with sequence, choice, reverse, etc wrappers algorithms to form complex algorithm trees. Since we get arbitrary nesting levels, `dbgLevel` allows printing intuitive debug output matching the algorithm tree structure.
* As a result of the changes, with `CifToSynthesisConverter` handling the non-algorithm-specific things, with `dbgLevel` being introduced, and with the reason for skipping automatic variable ordering being printed, the debug output of the regression tests have changed. However, these are only debug output changes. The algorithms function as before, and produce the same variable orders. Functionality, it works as before for end users.
* I got confused several times while working on this about terminology. Terms like 'order' and 'indices' were used in different ways, which was confusing, and led to regression bugs. Hence, I now use 'order' for orders of the variables (including per variable in the new order its original 0-based index), and (new) indices when referring to per variable in its original order its new 0-based index. Previously 'order' and 'indices' where sometimes used to refer to both these meaning. I've improved and clarified in various places the naming, JavaDoc, comments, etc, to hopefully avoid such confusion in the future.
I did several iterations of changing the API, not just for the current algorithms, but also to support future extensions (#196). This is the design I ended up with, that seems to fit well.
If you want to review this, I propose to review it commit by commit. I completely reworked my final solution into smaller commits, for reviewing efficiency. Each commit message has further details on the changes. However, some things only become clear in a later commit, as the changes in different commits are somewhat intertwined, and I could not spend days on even better-reviewable commits.
Closes #371
Addresses #196v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/323#340 Improved v0.6-M1 release notes.2022-05-27T13:10:53ZDennis Hendriks#340 Improved v0.6-M1 release notes.Closes #340Closes #340v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/322#340 develop to master for v0.6-M12022-05-26T13:47:24ZDennis Hendriks#340 develop to master for v0.6-M1Addresses #340Addresses #340v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/321#340 Updated release notes for v0.6-M1.2022-05-27T11:18:36ZDennis Hendriks#340 Updated release notes for v0.6-M1.Addresses #340Addresses #340v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/319#364 CIF data-based synthesis performance test models: water_lock2023-03-14T12:59:59ZFerdie Reijnen#364 CIF data-based synthesis performance test models: water_lockAddresses #364Addresses #364v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/318#367 Improved website style. Cleaned up unused CSS styles.2022-05-12T07:40:40ZDennis Hendriks#367 Improved website style. Cleaned up unused CSS styles.Closes #367Closes #367v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/317#364 CIF data-based synthesis performance test models: wizard + festo2022-05-19T12:13:08ZFerdie Reijnen#364 CIF data-based synthesis performance test models: wizard + festoAddresses #364
* Added benchmark import wizard.
* Added FESTO benchmark as an example.Addresses #364
* Added benchmark import wizard.
* Added FESTO benchmark as an example.v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/315#311 Add warning for plant referencing requirements in CIF2024-01-19T09:51:36ZFerdie Reijnen#311 Add warning for plant referencing requirements in CIFCloses #311Closes #311v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/314#344 Add Design Structure Matrix (DSM) clustering (2nd branch)2022-05-09T08:26:51ZAlbert Hofkamp#344 Add Design Structure Matrix (DSM) clustering (2nd branch)Closes #344
EDIT: Also don't forget about the previous branch!Closes #344
EDIT: Also don't forget about the previous branch!v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/313#362 Upgrade to JavaBDD 3.0.02022-05-03T20:25:32ZDennis Hendriks#362 Upgrade to JavaBDD 3.0.0Closes #362Closes #362v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/312#361 Lowered maven.minimal.version to real minimal version.2022-05-01T06:41:33ZDennis Hendriks#361 Lowered maven.minimal.version to real minimal version.- Lowered maven.minimal.version to real minimal version.
- Also only update it if needed.
- And indicated more literally what other versions to update for Tycho.
Closes #361- Lowered maven.minimal.version to real minimal version.
- Also only update it if needed.
- And indicated more literally what other versions to update for Tycho.
Closes #361v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/311#111 Build railroad diagrams during the build2022-09-29T11:47:53ZDennis Hendriks#111 Build railroad diagrams during the build- Railroad diagram generator code adapted to be a library.
- No application framework app/handler/options/exceptions/paths.
- New main and library methods.
- Custom logging functionality.
- Project build updates.
- Added railroa...- Railroad diagram generator code adapted to be a library.
- No application framework app/handler/options/exceptions/paths.
- New main and library methods.
- Custom logging functionality.
- Project build updates.
- Added railroad diagram generator Maven mojo.
- Reordered common.asciidoc .classpath entries. They are displayed in the specified order.
- Added missing test launch config for common.asciidoc project.
- Generate railroad diagrams during the build.
- Read railroad diagram configurations with buffered I/O.
- License header check updates.
Closes #111v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/309#271 DEPENDENCIES.txt update.2022-04-20T14:54:56ZDennis Hendriks#271 DEPENDENCIES.txt update.Addresses #271Addresses #271v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/307#355 Ensure correct minimal Java/Maven versions for build.2022-04-20T13:59:30ZDennis Hendriks#355 Ensure correct minimal Java/Maven versions for build.Closes #355Closes #355v0.6https://gitlab.eclipse.org/eclipse/escet/escet/-/merge_requests/306#354 Linearize merge/product documentation improvements.2022-04-20T13:59:40ZDennis Hendriks#354 Linearize merge/product documentation improvements.- They now refer to each other.
- They indicate their strengths and weaknesses, compared to each other.
- I opted not to make it a warning block, as I think it fits well as a normal paragraph, especially now that it is phrased as pros an...- They now refer to each other.
- They indicate their strengths and weaknesses, compared to each other.
- I opted not to make it a warning block, as I think it fits well as a normal paragraph, especially now that it is phrased as pros and cons, rather than only as a warning.
Closes #354v0.6