TITAN is reporting warnings and errors on publicly available 3GPP test suites on ttcn-3.org
Summary
For a research article, we tried to do measurements with TITAN on publicly available official TTCN-3 test suites from www.ttcn-3.org For this, we used the 3GPP Test suites. As they seem to be related to currently very important 5G technology they should have the best support. Also, they looked to be the largest official and standardized projects, which is beneficial for our measurements.
Steps and/or TTCN-3 code to reproduce
- Download the zip file of the project
- create a project for it in Eclipse for TITAN to handle.
- Convert the xsd files in the project into .ttcn files using TITAN's converter using the -N flag to make sure that all definitions of the XML Schemas with no target namespaces are generated into different TTCN-3 modules.
What is the current bug behavior?
TITAN reports warnings and errors for every project. For many of them, TITAN also reports unsupported elements and syntactical problems.
34123-3-g20 (SSNITZ) 11 warnings 249 errors //2 syntactic problems, 8 warnings for not supported elements 34123-3-g20 (UTRAN) 10 warnings 278 errors //3 syntactic problems, 3 warnings for not supported elements 34229-3-g20 (IMS_EUTRA) 22 warnings 70 errors //5 syntactic problems 34229-3-g20 (IMS_IRAT) 21 warnings 226 errors //4 syntactic problems, 3 warnings for not supported elements 34229-3-g20 (IMS_NR5GC) 42 warnings 52 errors //3 syntactic problems, 1 warning for not supported elements 34229-3-g20 (IMS_UTRAN) 10 warnings 266 errors //3 syntactic problems, 2 warnings for not supported elements 34229-3-g20 (IMS_WLAN) 14 warnings 62 errors //2 syntactic problems 36523-3-g90 (LTE) 25 warnings 62 errors //4 syntactic problems, 3 warnings for not supported elements 36523-3-g90 (LTE_A_IRAT)24 warnings 234 errors //4 syntactic problems, 3 warnings for not supported elements 36523-3-g90 (LTE_IRAT) 24 warnings 248 errors //4 syntactic problems, 3 warnings for not supported elements 38523-3-g20 (ENDC) 40 warnings 53 errors //8 syntactic problems (4 errors), 2 warning for not supported elements 38523-3-g20 (NR5GC) 67 warnings 54 errors //3 syntactic problems, 1 warning for not supported elements 38523-3-g20 (NR5GC_IRAT)55 warnings 62 errors //8 syntactic problems (4 errors), 2 warning for not supported elements
Please note, that for some of these projects the semantic analysis of TITAN also sometimes throws exceptions. Luckily for our measurements, this is not happening every time, but should be investigated. To reproduce you need to open + close a given project several times (this is the method we used for our measurements). The exception(s) should be found in the error log view of Eclipse after a few dozen iterations.
What is the expected correct behavior?
As these are the official publicly available and standardized test suites of 3GPP (and probably ETSI) testing the functionalities of 5G and other very important protocols ... it would be preferable if they would be supported by TITAN. Without the need to change them manually to remove all of the warnings/errors ... as having to modify official tests manually to make sure they can be used raises suspicion related to whether the equipment being tested would really pass the official tests or only hand modified ones.
Relevant logs and/or screenshots
Possible fixes
Titan version
8.0.0
Platform details (OS type and version)
Windows 10 (our observations and measurements were done a few months ago, but on a fully updated operating system at that time)
/cc @aknappqwt