PLCgen: testing plan and test set for S7-300
RWS wants to test the new PLC code generator in 2024-09. This means we should do our own tests before then, to test PLCgen, find issues, and fix them, beforehand. We want to reach a point that we have confidence in the correct functioning of the new PLCgen. I discussed with RWS (Lars, Koen) and we came to the following 'plan'.
Scoping:
- Priority is S7-300 target. The hardware setup at the TU/e lab has these PLCs. It is currently also planned for the RWS projects, although that may still change next year. Thus we will not focus on testing of other targets.
Different types of tests:
- Start with small models, and grow towards real RWS objects.
- Make an ESCET test set with small models.
- They should cover the CIF language concepts supported by PLCgen.
- They should cover the different parts of the code generator.
- This differs from the current integration tests in ESCET, as they are not only for regressions, and not only for checking the output (code). The goal is to compile them in TIA Portal and run them.
- We thus need not only the test models, but also the expected output. For instance a variable that eventually gets a certain value or so. This could then be inspected in the debugger.
Even if the PLC target changes, the test set should still be usable for testing other targets, is our expectation.
We agreed that Lars and Koen will work on (1). Within ESCET we will make the test set with test expectations for (2). Lars and Koen will execute the tests for (2) on the lab setup at TU/e.
Lars and Koen will report back with any issues found. They will also use the PLCgen documentation, and see whether that is understandable, whether anything is incorrect, etc. And they'll provide back feedback on it. That way we can improve the documentation as well, and we'll end up with 'tested' documentation.
Addresses #679