Skip to content

PLCgen: testing plan and test set for S7-300

RWS wants to test the new PLC code generator in 2024-09. This means we should do our own tests before then, to test PLCgen, find issues, and fix them, beforehand. We want to reach a point that we have confidence in the correct functioning of the new PLCgen. I discussed with RWS (Lars, Koen) and we came to the following 'plan'.

Scoping:

  • Priority is S7-300 target. The hardware setup at the TU/e lab has these PLCs. It is currently also planned for the RWS projects, although that may still change next year. Thus we will not focus on testing of other targets.

Different types of tests:

  1. Start with small models, and grow towards real RWS objects.
  2. Make an ESCET test set with small models.
    • They should cover the CIF language concepts supported by PLCgen.
    • They should cover the different parts of the code generator.
    • This differs from the current integration tests in ESCET, as they are not only for regressions, and not only for checking the output (code). The goal is to compile them in TIA Portal and run them.
    • We thus need not only the test models, but also the expected output. For instance a variable that eventually gets a certain value or so. This could then be inspected in the debugger.

Even if the PLC target changes, the test set should still be usable for testing other targets, is our expectation.

We agreed that Lars and Koen will work on (1). Within ESCET we will make the test set with test expectations for (2). Lars and Koen will execute the tests for (2) on the lab setup at TU/e.

Lars and Koen will report back with any issues found. They will also use the PLCgen documentation, and see whether that is understandable, whether anything is incorrect, etc. And they'll provide back feedback on it. That way we can improve the documentation as well, and we'll end up with 'tested' documentation.

Tasks

  • Make first tests, and see if approach works.
  • v1: Put first set of tests in a branch.
  • v2: Make second set of tests, fixing the issues of v1:
    • Leave out runtime errors (fixes #845 (closed))
    • Document alternative OK values for test output (literals become integers for S7-300) (addresses #846 (closed))
    • Document that non-listed variables can be ignored (addresses #846 (closed))
  • v3: Extend the test set with new tests.
    • Improved explanation of different values as outcome.
    • Test without simplifying values and inlining constants (if supported).
    • Test nearly empty model, enums, timers and tuples.
  • v4: Extend the test set with new tests.
    • Extend timer test.
    • Test more expressions (extended tuples test, added 'other' expressions test).
    • Test more declarations (algebraic variables, constants, state variables).
  • v5: Fix new issues found in test models.
    • #845 (closed) Still division by zero in expr_binary model.
    • #861 (closed) PLCgen test model enums does not comply with expected results.
  • v6: Fix new issues and add another model.
  • v7: Removed 'cbrt' and 'pow' as they are no longer supported by PLCgen.
  • v8: Fix 'tuples' model for too large real values, see #863 (comment 2471245)
  • v9: Extended/added test models.
    • Extended edges and tuples test models.
    • Added channels, io and sync_events test models.
  • v10: Various improvements/fixes.
    • ToolDef script: generate rename warnings.
    • Adapted all test models to prevent renames, and list the real PLC variable names for the expected output.
    • Fixed io test model.
    • Improved expected output of timers test model.
    • Extended sync_events test model.
    • Added iter_limit1, iter_limit2, max_neg_int, names and negative_zero test models.
  • v11: Fixed output of a test model.
    • #889 (closed) PLCgen test model "channels" results in incorrect value
  • v12: Various added/extended tests.
    • Test long identifiers/names.
    • names test indentation improvements.
    • Extend 'names' test with more S7 keywords.
    • Add op_bind_and_prio_bool and op_bind_and_prio_nums test models.
    • Rewrap output comments about real values.
    • Improve/extend tuples test model.
    • Add arrays test, inspired by tuples test models.
    • Add array_tuple_combi test models.
    • Don't test models with arrays for S7-300.
  • v13: Fix and extend.
    • Fix ToolDef script exclude typo (comb instead of combi).
    • Fix sync_events test model's sync10 automaton expected output.
    • Extend sync_events test model with more monitor testing.
  • v14: Fix expected output of 'iter_limit1' test model. (#920 (closed))
  • v15
    • Extra comments in 'sync_events' test model.
    • Extend 'io' model with an algebraic variable as output.
  • v16: fix some test models
    • #961 (closed) PLCgen test model "negative_zero" fails due to maximum PLC cycle time
    • #962 (closed) PLCgen test model "op_bind_and_prio_num" does not download to device
  • v17: duplicated sync_events test for code generated in main/functions (per !1046 (merged)).
  • v18: Test adherence to CIF controller properties checker execution scheme (new ctrl_chk_exec_scheme.cif test).
  • v19
    • #981 (closed) Fix expected test output of 'sync_events_*' tests.
    • Add 'iter_limit3' test.
  • v20: Split iter_limit3 test into two tests (for main/funcs), to test the working of the isProgress variable also for event functions.

To see the progress, check out the branch 798-plcgen-s7-300-self-certification-tests here.

Or download the test sets here:

Addresses #679

Edited by Dennis Hendriks

Copyright © Eclipse Foundation, Inc. All Rights Reserved.     Privacy Policy | Terms of Use | Copyright Agent