7-the_run-time_configuration_file.adoc 114 KB
Newer Older
Elemer Lelik's avatar
Elemer Lelik committed
1
2
3
4
5
6
7
8
9
10
= The Run-time Configuration File
:toc:
:table-number: 12

The behavior of the executable test program is described in the run-time configuration file. This is a simple text file, which contains various sections. The usual suffix of configuration files is `.cfg`.

Each section begins with a section name within square brackets. Different sections use different syntax, thus the section name determines the possible syntax of the members.

The configuration file can contain any white space characters. There are three ways to put comments in the file: you can use the C comment delimiters (i.e. /* and */). Additionally, characters beginning with # or // are treated as comments until the end of line.

balaskoa's avatar
balaskoa committed
11
Character string values shall be given between quotation marks. The non-printable characters as well as the quotation mark character within string values must be escaped using TTCN-3 or C escape sequences or quadruple notation. All kinds of escape sequences that are available in TTCN-3 and C languages (including octal and hexadecimal notation) are recognized.
Elemer Lelik's avatar
Elemer Lelik committed
12
13
14
15
16

All sections are optional, thus an empty file is also a valid (but meaningless) configuration file. The sections are processed in the given order. In case of duplicated sections, all sections will be processed and no warnings will be issued.

In the following we present all possible sections of the configuration file. The majority of sections are applicable in both single and parallel modes with identical meaning. However, some sections or settings in a section are applicable either in single or parallel mode only.

balaskoa's avatar
balaskoa committed
17
The indication (Parallel mode) following the section title signals that the concerned section is processed in parallel operation mode only. Irrelevant sections or options are ignored in each mode and a warning message is displayed during configuration file processing. For details on running TITAN TTCN-3 test suites in either single or parallel mode using see the TITAN TTCN-3 User Guide (see <<13-references.adoc#_13, [13]>>).
Elemer Lelik's avatar
Elemer Lelik committed
18

19
The component name defined by the `create` operation (see <<4-ttcn3_language_extensions.adoc#parameters-of-create-operation, here>>) can contain any characters. This component name can be used without quoted marks but it shall be used with quoted marks (i.e as quoted string) if it contains extra characters e.g. space (" "), hyphen ("-") or dot ("."). See also the examples of this chapter in sections <<module-parameters, [MODULE_PARAMETERS]>> and <<testport-parameters, [TESTPORT_PARAMETERS]>>.
Elemer Lelik's avatar
Elemer Lelik committed
20
21
22
23

[[module-parameters]]
== [MODULE_PARAMETERS]

balaskoa's avatar
balaskoa committed
24
This section may contain the values of any parameters that are defined in your TTCN-3 modules.
Elemer Lelik's avatar
Elemer Lelik committed
25
26
27
28
29
30
31
32
33
34
35

The parameters shall be specified after each other in any order. Each parameter must be given in the following order: module name (optional) <<13-references.adoc#_24, [24]>>, parameter name, field names (in case of records, sets or unions; optional) or array indexes (in case of arrays, records of or sets of; optional), assignment or concatenation operator (that is, the characters := or &=) and parameter value. An optional terminator semicolon may be used after each parameter value. The concatenation operator can be applied to list types (record of, set of). In this case only the value list notation can be used.

[[bnf-productions-for-this-section-25]]
=== BNF Productions for this Section

In the Function Test runtime:
[source]
----
ModuleParametersSection ::= "[MODULE_PARAMETERS]" {ModuleParameter}
ModuleParameter ::= ParameterName ParamOpType ParameterValue [SemiColon]
36
ParameterName ::= [(ModuleName | "*") "."] ParameterIdentifier
Elemer Lelik's avatar
Elemer Lelik committed
37
ModuleName ::= Identifier
38
39
40
41
ParameterIdentifier ::= Identifier | ParameterIdentifier "." Identifier
| ParameterIdentifier "[" ParameterExpression "]"
ParamOpType ::= ":=" | "&="
ParameterValue ::= ParameterExpression [LengthMatch] ["ifpresent"]
Elemer Lelik's avatar
Elemer Lelik committed
42
ParameterExpression ::= SimpleParameterValue | ParameterIdentifier
43
44
45
	| "(" ParameterExpression ")" | ("+" | "-") ParameterExpression
	| ParameterExpression ("+" | "-" | "*" | "/" | "&") ParameterExpression
LengthMatch ::= "length" "(" LengthBound [".." (LengthBound|"infinity")] ")"
Elemer Lelik's avatar
Elemer Lelik committed
46
47
48
LengthBound ::= ParameterExpression
SimpleParameterValue ::= IntegerValue | FloatValue | BitstringValue | HexstringValue
	| OctetstringValue | StringValue | UniversalCharstringValue | BooleanValue
49
50
51
	| ObjIdValue | VerdictValue | EnumeratedValue | "omit" | "NULL" | "null"
	| "?" | "*" | IntegerRange | FloatRange | StringRange
	| "pattern" PatternChunk
Elemer Lelik's avatar
Elemer Lelik committed
52
	| BitStringMatch | HexStringMatch | OctetStringMatch
53
	| "mtc" | "system"
Elemer Lelik's avatar
Elemer Lelik committed
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
	| CompoundValue
IntegerValue ::= Number
FloatValue ::= FloatDotNotation | FloatENotation
StringValue ::= Cstring
BitstringValue ::= Bstring
HexstringValue ::= Hstring
OctetstringValue ::= Ostring
UniversalCharstringValue ::= Quadruple
Quadruple ::= "char" "(" ParameterExpression "," ParameterExpression ","
    ParameterExpression "," ParameterExpression ")"
ObjIdValue ::= "objid" "{" {ObjIdComponent}+ "}"
ObjIdComponent ::= NumberForm | NameAndNumberForm
NumberForm ::= Number
NameAndNumberForm ::= Identifier "(" Number ")"
EnumeratedValue ::= Identifier
PatternChunk ::= Cstring | Quadruple
70
71
72
IntegerRange ::= "(" ("-" "infinity" | IntegerValue) ".." (IntegerValue | "infinity") ")"
FloatRange ::= "(" ("-" "infinity" | FloatValue) ".." (FloatValue | "infinity") ")"
StringRange ::= "(" StringRangeBound ".." StringRangeBound ")"
Elemer Lelik's avatar
Elemer Lelik committed
73
StringRangeBound ::= Cstring | Quadruple
74
75
76
77
78
79
80
81
CompoundValue ::= "{" "}"
 	| "{" FieldValue {"," FieldValue} "}"
 	| "{" ArrayItem {"," ArrayItem} "}"
 	| "{" IndexItem {"," IndexItem} "}"
	| "(" ParameterValue "," ParameterValue {"," ParameterValue} ")"
	| ("complement" | "superset" | "subset") "("ParameterValue {","
   ParameterValue} ")"
FieldValue ::= FieldName ":=" ParameterValueOrNotUsedSymbol
Elemer Lelik's avatar
Elemer Lelik committed
82
FieldName ::= Identifier | ASN1LowerIdentifier
83
84
85
ArrayItem ::= ParameterValueOrNotUsedSymbol | ("permutation" "("ParameterValue {"," ParameterValue} ")")
IndexItem ::= "[" ParameterExpression "]" ":=" ParameterValue
ParameterValueOrNotUsedSymbol ::= ParameterValue | "-"
Elemer Lelik's avatar
Elemer Lelik committed
86
87
88
89
90
91
92
93
94
95
----

The BNF productions in the Load Test runtime are mostly the same with one difference:

`ParameterIdentifier ::= Identifier`

The parameter value can be one of the following:

* Integer value. A number in decimal notation or an expression composed of numbers using the basic arithmetic operations to allow more flexibility with macro substitution.
* Floating point value. A floating point number in decimal dot notation or exponential notation or an arithmetic expression composed of such values.
balaskoa's avatar
balaskoa committed
96
97
98
* Bitstring value (in the notation of TTCN-3). String fragments can be concatenated to allow more flexible macro substitution.
* Hexstring value (in the notation of TTCN-3). String fragments can be concatenated to allow more flexible macro substitution.
* Octetstring value (in the notation of TTCN-3). String fragments can be concatenated to allow more flexible macro substitution.
Elemer Lelik's avatar
Elemer Lelik committed
99
* Charstring value (between quotation marks; escape sequences are allowed). String fragments can be concatenated to allow more flexible macro substitution.
balaskoa's avatar
balaskoa committed
100
* Universal charstring value (sequence of concatenated fragments; each fragment can be either a printable string within quotation marks or a quadruple in TTCN-3 notation).
Elemer Lelik's avatar
Elemer Lelik committed
101
* Boolean value (`true` or `false`).
balaskoa's avatar
balaskoa committed
102
* Object identifier (`objid`) value (in the notation of TTCN-3). Only `NumberForm` or `NameAndNumberForm` notations are allowed.
Elemer Lelik's avatar
Elemer Lelik committed
103
104
105
* Verdict value (`none`, `pass`, `inconc`, `fail` or `error`).
* Enumerated value (the symbolic value, i.e. an identifier). Numeric values are not allowed.
* Omit value (i.e.` omit`). Valid for optional record or set fields only.
balaskoa's avatar
balaskoa committed
106
* `null` value for TTCN-3 component and default references.
Elemer Lelik's avatar
Elemer Lelik committed
107
108
* `NULL` value for the ASN.1 `NULL` type.
* "?" value: AnyValue for matching
109
* "*" value: AnyValueOrNone for matching
Elemer Lelik's avatar
Elemer Lelik committed
110
111
* IntegerRange/FloatRange/StringRange: matching an integer/float/charstring range
* Pattern for pattern matching in charstring and universal charstring
balaskoa's avatar
balaskoa committed
112
* Bit/Hex/Octet -string matching mechanism which are bit/hex/octet strings that contain "?" and "*" for matching
Elemer Lelik's avatar
Elemer Lelik committed
113
114
115
116
117
* "mtc" and "system" values for component references
* Compound value with assignment notation. One or more fields (separated by commas) with field names within brackets for types `record` and `set`.
* Compound value with value list notation. Comma separated list of values between brackets for types `record of`, `set of` and `array`.
* Compound value with indexed list notation. One or more fields with field index and value for types record of and set of
* Compound value containing a template list. Can be a value list, complemented value list, superset and subset list.
118
* Reference to a module parameter, or a field/element of a module parameter (only in the Function Test runtime). Its syntax is the same as the left hand side of a module parameter assignment/concatenation (except the symbol ‘*’ cannot be used to specify all modules). The reference is substituted with the current value of the specified module parameter (its value prior to the execution of this module parameter assignment/concatenation). References can also appear in the expressions specified above (integer and float references can appear in arithmetic expressions, references to string types can be concatenated with other strings of the same type). A dynamic test case error is displayed if the referenced module parameter is unbound.
Elemer Lelik's avatar
Elemer Lelik committed
119

120
Nested compound values are permitted in arbitrary depth. In compound values with assignment and value list notations the "-" symbol can be used to skip an element. In value list notation for record of and set of permutation lists can be used.
Elemer Lelik's avatar
Elemer Lelik committed
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141

Parsing conflict: An asterisk (*) after a module parameter expression could be treated as a multiplication operator or the "all modules" symbol for the next module parameter assignment/concatenation. The configuration parser always treats it as a multiplication operator. In order to use the asterisk as an "all modules" symbol, the previous statement must be closed with a semicolon (;).

Example:
[source]
----
# correct:
tsp_IntPar := tsp_IntPar + 1;
*.tsp_FloatPar := 3.0;
# incorrect, causes a parser error:
tsp_IntPar := tsp_IntPar + 1
*.tsp_FloatPar := 3.0;
----

=== Differences between the TTCN-3 and the Configuration File Module Parameters Section Syntax

Neither the ttcn-3 syntax nor the module parameter section syntax is the subset of the other. Historically some module parameter values that are not legal in ttcn-3 have been accepted, for backward compatibility reasons this behavior has been kept. In most cases the module parameter syntax is a subset of the ttcn-3 syntax but there are important exceptions:

* Field values of records and sets can be referenced multiple times, in this case all field value assignments will be executed in order of appearance. Example: mp_myRecord := { a :=1, b:=3, a:=2 } // a==2
* In an assignment notation used for a union multiple field values can appear, only the last value will be used. Example: mp_myUnion := { a:=1,b:=2,a:=3 } only a:=3 will be used, the other 2 assignments are disregarded without warnings.
* The order of fields in assignment notation for records does not have to be identical to the order of fields specified in the type definition of the record type. Example: type record MYREC { integer a, integer b } mp_myRec := { b:=2, a:=1}
142
* The "\*" matching symbol can be used for mandatory fields (in TTCN-3 this cannot be used directly but indirectly any field of a variable template can be set to "*").
Elemer Lelik's avatar
Elemer Lelik committed
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229

In the module parameters section only constant values and references to module parameters can be used. Function calls are not allowed (not even predefined functions).

Example:
[source]
----
[MODULE_PARAMETERS]
par1 := -5
MyModule1.par2 := "This is a string\n"
MyModule1.par3 := {
  ethernet_header := {
    source_address := ’000100010005’O,
    destination_address := ’0008C7993605’O,
    ether_type := 34525
  },
  ipv6 := {
    header := {
      version := 6,
      traffic_class := 0,
      flow_label := 0,
      payload_length := 0,
      next_header := 58,
      hop_limit := 255,
      source_address := ’FE80000000000000020100FFFE010005’O,
      destination_address := ’FE800000000000000208C7FFFE993605’O
    },
    extension_headers := {
      {
        hop_by_hop_options_header := {
          next_header := 1,
          header_length := 2,
          options := ’13’O
        }
      }
    },
    data := {
      f_router_advertisement := {
        icmp_type := 134,
        code := 0,
        checksum := 0,
        hop_limit := 255,
        m_bit := ’0’B,
        o_bit := ’0’B,
        reserved := ’000000’B,
        lifetime := 600,
        reachable_time := 300000,
        retrans_timer := 1000,
        options := {
          {
            lla := {
              option_type := 1,
              option_length := 1,
              lla_address := ’0008C7993605’O
            }
          }
        }
      }
    }
  }
}
MyModule2.par4 := ’10010’B
par5 := objid { itu_t(0) identified_organization(4) etsi(0) 12345 6789 }
par6 := "Character " & char(0, 0, 1, 113) & " is a Hungarian letter."
par_record_of_int &= {1,2,3}
par_record_of_int &= {4,5,6}
par_record_of_int[6] := 7
MyModule1.par3.ethernet_header.ether_type := 34526
----

[[logging]]
== [LOGGING]

The executable test program produces a log file during its run. The log file contains important test execution events with time stamps[26]. This section explains how to set the log file name and the event classes to be logged. Logging may be directed to file or displayed on console (standard error).

Various options can be set in the section [`LOGGING`]. They affect the format and appearance of the test execution logs. Any option may be omitted; that is, each has a default value that is used if the option is omitted. If the same option is present several times in a configuration file only the latest value will take effect; the previously assigned values are ignored and a warning message is issued during configuration file processing.

[[LoggerPlugins]]
=== LoggerPlugins

TITAN is equipped with an extensible logging architecture. The test execution events are directed towards the logger plugin interface. This interface can host arbitrary number of logger plugins.

The logging can be statically and dynamically loaded.

The default logging is the statically loaded built in logging.

The dynamically loaded logging can be the built in LegacyLogger, the plugins shipped with TITAN (see <<dynamically-loaded-logger-plugins-shipped-with-titan, here>>) or user created dynamically linked plugins (for advanced users, see chapter 3 in <<13-references.adoc#_16, [16]>>.

balaskoa's avatar
balaskoa committed
230
NOTE: *When using dynamically loaded logger plugins it is very important to use the dynamic runtime library of TITAN, this can be done by using the -l switch when generating the makefile.*
Elemer Lelik's avatar
Elemer Lelik committed
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251

The desired logger plugins need to be set in the `LoggerPlugins` option. The `LoggerPlugins` option takes a non-empty, comma separated list of logger plugin settings. These settings can be specified in the following ways:

* A logger plugin identifier followed by an assignment and a quoted string containing the plugin location. E.g. `LoggerPlugins := { plugin1 := "/absolute/path/to/plugin1.so" } or LoggerPlugins := { plugin1 := "/absolute/path/to/plugin1" }`. The identifier of a logger plugin is determined by its author can be learned from its documentation. The plugin location includes the file name of the plugin with an optional file system path prefix. A logger plugin is a dynamically linked shared object thus the logger plugin typically resides in a `.so` file. When the path prefix is missing or a relative path is used then TITAN attempts to locate the plugin in the path specified in the `LD_LIBRARY_PATH` environment variable. The plugin file name can be provided in two different ways: either by specifying the whole file name (ending with .so) or by specifying only the base of the name (omit the .so ending). The latter method is preferred because a logger plugin usually consists of 4 different shared library files and the proper file must be selected. The 4 different versions correspond to the single/parallel mode and the load/function test TITAN runtimes. If the file name ending is not provided the executable will determine it automatically, but if the whole file name is provided then it must correspond to the runtime which is actually used by the executable.
* A single logger plugin identifier. E.g. `LoggerPlugins := { plugin1 }`. In this case there should be a logger plugin named plugin1.so in one of the paths specified in the `LD_LIBRARY_PATH` environment variable.

NOTE: If TITAN is unable to locate any of the listed logger plugins, it will quit immediately with an error message. So, if the `LoggerPlugins` option is used, take special care to set `LD_LIBRARY_PATH` correctly or use absolute paths when specifying the plugins. There is a built-in logger plugin in TITAN, which provides the usual text-based logging format. This plugin is used when the `LoggerPlugins` option is omitted or it was listed explicitly in the list with the `LegacyLogger` case insensitive identifier. Since it’s not possible to specify the path for this special, built-in logger plugin, only the second (with no path) specification mode is applicable here.

The logger plugins are responsible for the "final appearance" of the test execution log. The current TITAN distribution comes with a single logger plugin but it also supports user written logging plugins. The built-in `LegacyLogger` plugin produces log files and console log entries with similar content to elder TITAN revisions.

In case of overlapping plugin settings in multiple `LoggerPlugins` options, all configured plugins are attached to the list of existing plugins and take part in logging (i.e. do not overwrite them).

The configured logger plugins are valid for each test component unless otherwise configured (see subsection below).

It is possible to reference all the listed plugins with * to e.g. assign the same parameters and values for multiple plugins. However, plugin configuration through * will not have an effect on plugin parameters, whose value was set previously by referencing explicitly the exact plugin using its name and optionally the component identifier it is configured for.

Logger plugins can be configured with arbitrary name-value pairs in the configuration file (see <<EmergencyLogging, here>>). E.g. `*.*.param1 := "value1"` will set the param1 parameter to a string value1 for all, not explicitly configured logger plugins on all test components. Logger plugins can ignore unknown parameters.

Each logger plugin has 4 .so files. The following table contains the names of the 4 different cases if the base file name (this is not the name of the plugin, library file names start with "lib") of the plugin is "libplugin":

NOTE: The preferred method of specifying the above logger plugin in the configuration file is: `LoggerPlugins := { MyPluginName := "libplugin"}` +
252
The name MyPluginName is the name of the plugin and can be different than the library file names. If the full plugin file name is provided ( "libplugin.so, libplugin-parallel.so, etc.) then care must be taken to always use the name that corresponds to the mode in which the executable is running.
Elemer Lelik's avatar
Elemer Lelik committed
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320

==== Component-based Logger Plugin Settings

It is possible to associate an individual set of logger plugins for each test component. The component designation can be:

* the component name as given in the command `create`,
* the component reference (though using component references as identifiers is not recommended as this is a tool dependent identifier),
* the symbol * meaning all valid test components or
* the keyword `mtc` in the case of the Main Test Component.

The component name, if present, precedes the keyword `LoggerPlugins`. They are separated by a dot (.). The absent component reference is equivalent to `*.LoggerPlugins` meaning all valid test components.

[[dynamically-loaded-logger-plugins-shipped-with-titan]]
=== Dynamically Loaded Logger Plugins Shipped with TITAN

Anyone can write a logger plugin to use with TITAN, but there are some widely used plugins that were made part of TITAN. These are available in the `$(TTCN3_DIR)/lib` sub-directory. Usually `LD_LIBRARY_PATH` contains this directory, if not then it should either be added to it or the path to the .so file has to be specified.

[[junit-logger-plugin]]
==== JUnitLogger Plugin

It outputs XML files in JUnit format. This format is needed by Hudson/Jenkins, a continuous integration test tool. The XML files written by this logger plugin are a subset of the JUnit XML output format.

At first select dynamic linking at Makefile generation according to <<LoggerPlugins, LoggerPlugins>>.

To load the plugin the section `[LOGGING]` of the runtime configuration file should contain the following line: `LoggerPlugins := { JUnitLogger := "libjunitlogger" }`

The plugin has 2 parameters:filename_stem: set the output file name, the name will start with the string specified in this parameter and end with "-<process_id>.log". The default value is "junit-xml". +
testsuite_name: the name of the test suite, this will be written into the XML file.

_Example 1:_ Simplest HelloWorld example.

Source file
[source]
----
module hello {
  type component CT {}
  testcase tc1() runs on CT {
    log("Hello Titan!");
    setverdict(pass,"Everything is ok");
  }

  testcase tc2() runs on CT {
    log("Hello Titan!");
    setverdict(fail,"Something was wrong");
  }
control {
  execute(tc1());
  execute(tc2());
}

}
}
----

Configuration file (`cfg.cfg`):

----
[LOGGING]
LogSourceInfo := Yes
SourceInfoFormat := Single
LoggerPlugins := { JUnitLogger := "libjunitlogger" }
*.JUnitLogger.filename_stem := "MyJunitLogFile"
*.JUnitLogger.testsuite_name := "myJUnitTest"

[EXECUTE]
hello.control
----

balaskoa's avatar
balaskoa committed
321
The makefile was generated by the command `makefilegen -fl hello.ttcn`.
Elemer Lelik's avatar
Elemer Lelik committed
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355

The executable was executed by the command `ttcn3_start hello cfg.cfg`.

After running the log file name was `MyJunitLogFile-6426.log`, its content was:
[source]
----
<?xml version="1.0"?>
<testsuite name='myJUnitTest'><!-- logger name="JUnitLogger" version="v1.0" -->
<!-- Testcase tc1 started -->
<!-- Testcase tc1 finished in 0.000399, verdict: pass, reason: Everything is ok -->
  <testcase classname='hello' name='tc1' time='0.000399'>
  </testcase>
<!-- Testcase tc2 started -->
<!-- Testcase tc2 finished in 0.000225, verdict: fail, reason: Something was wrong -->
  <testcase classname='hello' name='tc2' time='0.000225'>
    <failure type='fail-verdict'>Something was wrong

      hello.ttcn:14 hello control part
      hello.ttcn:10 tc2 testcase
    </failure>
  </testcase>
</testsuite>
----

*Format of the results*

The results are included in testcases (between <testcase> tags) within a testsuite (between <testsuite> tags).The testsuite has only one attribute "name" which contains the name of the testsuite.

Each testcase starts with two xml style comments which is followed by a <testcase> xml tag.

*_The comments_*

Each testcase starts with the following two xml comments:

balaskoa's avatar
balaskoa committed
356
357
* `<!- Testcase "name of the testcase" started ->`
* `<!- Testcase "name of the testcase" finished in "execution time is seconds", verdict:"verdict" ->`
Elemer Lelik's avatar
Elemer Lelik committed
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380

This is followed by a <testcase> tag.

*_The <testcase> tag_*

The <testcase> tag has the following attributes:

* `classname=’name of the module’`
* `name=’name of the testcase’`
* `time=’execution duration in seconds’`

Depending of the verdict the <testcase> may have element contents which can be the following:

_Verdict: pass_

No children

_Verdict: fail_

Has the following element content with the following attribute:

<failure type=`fail-verdict'>

balaskoa's avatar
balaskoa committed
381
The <failure> tag can have several of the following text contents - each in a separate line (see results log example above):
Elemer Lelik's avatar
Elemer Lelik committed
382
383
384
385
386
387
388
389
390
391

* control part
* testcase
* altstep
* function
* external function
* template

Each line contains `"filename:linenumber identifier definition"`, where each of the items mean the following:

balaskoa's avatar
balaskoa committed
392
393
394
* filename:linenumber - the file and the line where the test failed
* identifier - the identifier depending on the definition, e.g. the classname in case of "control part" or the name of the test in case of "testcase"
* definition - see the list in <failure> tag text content
Elemer Lelik's avatar
Elemer Lelik committed
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481

This is a simple stacktrace of the failure.

Verdict: none

[source]
<skipped>no verdict</skipped>

Verdict: `inconclusive`

No children

Verdict: `error`

Has the following element content with the following attribute:

[source]
<error type='DTE'>

The <error> tag has the text contents containing the reason of the error.

*_XSD validation_*

This is the xsd file to validate the xml generated by the plugin:
[source]
----
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

    <xs:element name="failure">
        <xs:complexType mixed="true">
            <xs:attribute name="type" type="xs:string" use="optional"/>
        </xs:complexType>
    </xs:element>

    <xs:element name="skipped" type="xs:string"/>

    <xs:element name="error">
        <xs:complexType mixed="true">
            <xs:attribute name="type" type="xs:string" use="optional"/>
        </xs:complexType>
    </xs:element>

    <xs:element name="testcase">
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="skipped" minOccurs="0" maxOccurs="1"/>
                <xs:element ref="error" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="failure" minOccurs="0" maxOccurs="unbounded"/>
            </xs:sequence>
            <xs:attribute name="classname" type="xs:string" use="required"/>
            <xs:attribute name="name" type="xs:string" use="required"/>
            <xs:attribute name="time" type="xs:string" use="required"/>
        </xs:complexType>
    </xs:element>

    <xs:element name="testsuite">
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="testcase" minOccurs="0" maxOccurs="unbounded"/>
            </xs:sequence>
            <xs:attribute name="name" type="xs:string" use="required"/>
        </xs:complexType>
    </xs:element>

</xs:schema>
----

==== JUnitLogger2 Plugin

It outputs XML files in JUnit format. This format is needed by Hudson/Jenkins, a continuous integration test tool. The XML files written by this logger plugin are a subset of the JUnit XML output format.

The JUnitLogger2 plugin works as the <<junit-logger-plugin, JUnitLogger plugin>> with the following exceptions:

* Configuration file (`cfg.cfg`):
+
In the configuration file when specifying the logger plugin libjunitlogger2 should be written.
+
[source]
----
[LOGGING]
LogSourceInfo := Yes
SourceInfoFormat := Single
LoggerPlugins := { JUnitLogger := "libjunitlogger2" }
*.JUnitLogger.filename_stem := "MyJunitLogFile"
*.JUnitLogger.testsuite_name := "myJUnitTest"
----
Kristof Szabados's avatar
Kristof Szabados committed
482
* The results are included in testcases (between <testcase> tags) within a testsuite (between <testsuite> tags) like the JUnitLogger but the testsuite has attributes other than "name" which contains the name of the testsuite.
Elemer Lelik's avatar
Elemer Lelik committed
483
484
485
+
New attributes:
+
balaskoa's avatar
balaskoa committed
486
tests - number of all testcases executed
Elemer Lelik's avatar
Elemer Lelik committed
487
+
balaskoa's avatar
balaskoa committed
488
failures - number of testcases whose final verdict is fail
Elemer Lelik's avatar
Elemer Lelik committed
489
+
balaskoa's avatar
balaskoa committed
490
errors - number of testcases that encountered an error
Elemer Lelik's avatar
Elemer Lelik committed
491
+
balaskoa's avatar
balaskoa committed
492
skipped - number of testcases with a none verdict
Elemer Lelik's avatar
Elemer Lelik committed
493
+
balaskoa's avatar
balaskoa committed
494
time - the time in seconds from the start of the tests until all the testcases executed.
Elemer Lelik's avatar
Elemer Lelik committed
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
+
* XML comments are not added to the output.
* XSD:
+
[source]
----
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

	<xs:element name="failure">
		<xs:complexType mixed="true">
			<xs:attribute name="type" type="xs:string" use="optional"/>
		</xs:complexType>
	</xs:element>

	<xs:element name="skipped" type="xs:string"/>

	<xs:element name="error">
		<xs:complexType mixed="true">
			<xs:attribute name="type" type="xs:string" use="optional"/>
		</xs:complexType>
	</xs:element>

	<xs:element name="testcase">
		<xs:complexType>
			<xs:sequence>
				<xs:element ref="skipped" minOccurs="0" maxOccurs="1"/>
				<xs:element ref="error" minOccurs="0" maxOccurs="unbounded"/>
				<xs:element ref="failure" minOccurs="0" maxOccurs="unbounded"/>
			</xs:sequence>
			<xs:attribute name="classname" type="xs:string" use="required"/>
			<xs:attribute name="name" type="xs:string" use="required"/>
			<xs:attribute name="time" type="xs:string" use="required"/>
		</xs:complexType>
	</xs:element>

	<xs:element name="testsuite">
		<xs:complexType>
			<xs:sequence>
				<xs:element ref="testcase" minOccurs="0" maxOccurs="unbounded"/>
			</xs:sequence>
			<xs:attribute name="name" type="xs:string"  use="required"/>
			<xs:attribute name="tests" type="xs:int" use="required"/>
			<xs:attribute name="failures" type="xs:int" use="required"/>
			<xs:attribute name="errors" type="xs:int" use="required"/>
			<xs:attribute name="skipped" type="xs:int" use="required"/>
			<xs:attribute name="time" type="xs:string" use="required"/>
		</xs:complexType>
	</xs:element>

</xs:schema>
----

==== TSTLogger plugin

The `TestStatistics` TITAN Logger plugin sends HTTP messages to the `TestStatistics` web based tool. `TestStatistics` has a web interface (http://eta-teststatistics.rnd.ki.sw.ericsson.se/ts/login) where the test result data can be examined. Currently the following messages are sent to the `TestStatistics` tool:

* Test suite started (tsstart)
* Test case started (tcstart)
* Test case finished (tcstop)
* Test suite finished (tsstop)
* Test case fail reason (tcfailreason)

The content of these messages is filled based on the log data and data given in the configuration file. Some data needs to be set as logger plugin parameters in the configuration file because it is needed by the `TestStatistics` tool but the TITAN log messages do not contain such information. The plugin parameters:

[cols="m,,",options="header",]
|===
|Name |Default value |Description
|tst_host_name |"eta-teststatistics.rnd.ki.sw.ericsson.se" |`TestStatistics` web service host name
|tst_service_name |"http" |`TestStatistics` web service name or port number
|tst_tcstart_url |"/ts-rip/rip/tcstart" |Path for the tcstart message
|tst_tcstop_url |"/ts-rip/rip/tcstop" |Path for the tcstop message
|tst_tsstart_url |"/ts-rip/rip/tsstart" |Path for the tsstart message
|tst_tsstop_url |"/ts-rip/rip/tsstop" |Path for the tsstop message
|tst_tcfailreason_url |"/ts-rip/rip/tcfailreason" |Path for the tcfailreason message
|dbsUrl |"esekilx0007-sql5.rnd.ki.sw.ericsson.se:3314" |database URL
|dbUser |"demo" |database user
|dbPass |"demo" |plain text password of the user
|dbName |"teststatistics_demo" |name of the database
|log_plugin_debug |"0" |The logger plugin will print some debug information to the console if this value is not "0".
|testConfigName |"DefaultConfigName" |name of this specific configuration of the test suite
|suiteName |"DefaultSuiteName" |name of test suite
|executingHost |the host name of the MTC (determined with gethostname()) |host where the test was executed
|sutId |"0.0.0.0" |IP address of SUT
|sutName |"DefaultSUTName" |name of SUT
|lsvMajor |"1" |major version number of SUT
|lsvMinor |"0" |minor version number of SUT
|runByUser |Login name of the user running the MTC (determined with getlogin()) |name of user running the tests
|projectName |"DefaultProjectname" |name of the project
|productName |"DefaultProductName" |name of the product
|productVersion |"0.0" |version of the product
|configType |"configType" |
|configVersion |"configVersion" |
|testType |"testType" |
|logLink |"default_log_location" |absolute location of log files
|logEnd |"default_web_log_dir" |log directory relative to web server root
591
|reportEmail |automatically set to < runByUser > +"@ericsson.com&quot; |who is to be notified via email
Elemer Lelik's avatar
Elemer Lelik committed
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
|reportTelnum |"0" |where to send the SMS notification
|===

The parameters starting with "tst_" should be modified only if the TestStatistics tool is moved to another location on the intranet.

There are 4 parameters for setting the database connection, the database can also be selected on the web interface, all information sent by TITAN will be stored there. The default values will use a demo database which can be used for experimentation.

All parameters have default values, thus the plugin can be used even without setting any parameters. However some parameters should be set to a meaningful value, such as the suiteName or the projectName. An example from a configuration file:
[source]
----
LoggerPlugins := { TSTLogger := "libtstlogger" }
*.TSTLogger.testConfigName = "Hiper Giga Test"
*.TSTLogger.sutId = "11.22.33.44"
*.TSTLogger.projectName = "MagicProject"
*.TSTLogger.suiteName = "Super Test Suite"
*.TSTLogger.lsvMajor = "3"
*.TSTLogger.lsvMinor = "14"
----
To load the plugin the runtime configuration file should contain the following line: +
`LoggerPlugins := { TSTLogger := "libtstlogger" }`

[[lttngustlogger-plugin]]
==== LTTngUSTLogger plugin

The LTTng-UST logger plugin emits each logging statement as an http://lttng.org/[LTTng-UST] event. LTTng is a low-overhead tracer for Linux which generates http://diamon.org/ctf[CTF] traces.

To use the LTTng-UST logger plugin:

. Make sure LTTng (2.7 or greater) is installed (the LTTng-tools and LTTng-UST components are required).
. Add the following line to your runtime configuration file:LoggerPlugins := { LTTngUSTLogger := "liblttng-ust-logger" }
. Create an LTTng tracing session:lttng create
balaskoa's avatar
balaskoa committed
623
. Enable TITAN events:lttng enable-event -userspace titan_core:'*'
Elemer Lelik's avatar
Elemer Lelik committed
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
. Start tracing:lttng start
. Run your test script.
. When you are done, stop tracing:lttng stop
. Inspect the recorded events with an LTTng trace viewer, for example http://tracecompass.org/[Trace Compass] or http://diamon.org/babeltrace[Babeltrace].

When the plugin is loaded, it dynamically loads the LTTng-UST tracepoint provider, a shared object installed in the same directory. It is important that the `LD_LIBRARY_PATH` environment variable be set to this directory, otherwise the plugin issues a warning message and does not emit LTTng events.

[[logfile]]
=== `LogFile`

Option `LogFile` stands for the name of the log file. A string value is expected, which is interpreted as a skeleton to determine the file name. To make the file name handling more flexible the string may contain special metacharacters, which are substituted dynamically during test execution.

The table below contains the list of available metacharacters in alphabetical order. Any unsupported metacharacter sequence, that is, if the `%` character is followed by any character that is not listed in the table below or a single percent character stays at the end of the string, will remain unchanged.

.Available metacharacters for setting log file names
[cols="m,",options="header",]
|===
|Meta-character |Substituted with . . .
balaskoa's avatar
balaskoa committed
642
643
|%c |the name of the TTCN-3 test case that the PTC belongs to.
|%e |the name of the TTCN-3 executable. The .exe suffix (on Windows platforms) and the directory part of the path name (if present) are truncated.
Elemer Lelik's avatar
Elemer Lelik committed
644
645
646
647
648
649
650
651
652
653
|%h |the name of the computer returned by the gethostname(2) system call. This usually does not include the domain name.
|%i |the sequence number of the log fragment.
|%l |the login name of the current user. If the login name cannot be determined (e.g. the current UNIX user ID has no associated login name) an empty string is returned.
|%n |
- the name of the test component if the PTC has been given a name with the command create in the TTCN-3 create operation; an empty string otherwise. +
- the string HC if the logfile is generated by a Host Controller +
- the string MTC if the component is the Main Test Component (both in parallel and in single mode)
| %p | the process ID (`pid`) of the UNIX process that implements the current test component. The `pid` is written in decimal notation.
| %r | the component reference or component identifier. On PTCs it is the component reference (an integer number greater or equal to 3) in decimal notation. On the Main Test Component, Host Controller or in single mode the strings `mtc`, `hc` or `single` are substituted, respectively.
| %s | the default suffix for the log files without the leading dot. Now it always results in string log, but future versions may support log file compression. In this case the suffix will depend on the chosen compression method.
balaskoa's avatar
balaskoa committed
654
| %t | the TTCN-3 component type. Note: The MTC and HC have no dedicated component type since they can implement several component types in different test cases.
Elemer Lelik's avatar
Elemer Lelik committed
655
656
657
658
659
| %% | a single % character.
|===

The outcome of substitution will result in the name of the log file. It may resolve either to a simple file name or to an absolute or relative path. The relative pathnames are always related to the current working directory of the executable tests in single mode or that of the Host Controller in parallel mode, respectively. If the pathname contains one or more nonexistent directories, those directories (and/or subdirectories) will be automatically created with permissions `0755` before the log file is opened.

Kristof Szabados's avatar
Kristof Szabados committed
660
If the given string or the result of substitution is empty, no log file will be created and only console logging will be performed regardless the setting of `FileMask`. Empty log files will not be created when logging to files is completely disabled (i.e. `FileMask` is set to `LOG_NOTHING`) even if the value of `LogFile` would yield a valid file name.
Elemer Lelik's avatar
Elemer Lelik committed
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718

In parallel mode the user must ensure that the resulting log file names are unique for every component. Otherwise, if several components try to write into the same log file, the contents of the log will be unpredictable. The uniqueness is automatically provided if the host name (`%h`) and the component reference (`%r`) or the process ID (`%p`) is included in the file name.

If testcase name (`%c`) or component name (`%t`) is included, the log file name may change during runtime. A new log file will be created/opened in this case. If a log file with the new name already exists, it is overwritten by default. Because of this, it is *highly recommended* to set `AppendFile` (see <<AppendFile, here>>) to `Yes` if LogFile contains `%c` or `%t`.

If omitted, the default value for option `LogFile` is `%e-part%i.%s` in single mode and `%e.%h-%r-part%i.%s` in parallel mode, respectively. This ensures backward compatibility with earlier versions in parallel mode and follows the most commonly used file naming convention in single mode.

[[filemask]]
=== `FileMask`

Option `FileMask` determines which events will be written to the log file and which ones will be filtered out.

==== Category-based Filtering

Table 14 enumerates the first level groups of events (that is, logging categories) along with a symbolic constant. The selected constants separated by a pipe (|) will determine what will be logged. When `FileMask` is omitted from the configuration file, its default value (`LOG_ALL`) is applied.

Some of the first level logging categories may be divided in second level subcategories to get a finer logging granularity. These categories are listed in the tables 11 to 21. First level categories and second level subcategories may be mixed in the option.

First level logging categories may be considered as notational convenience. They are maintained also for backward compatibility: legacy configuration files containing only first level categories will continue to work unmodified. However, the resulting log file may look different, as event categories have been modified; some messages, mostly `PARALLEL` and `FUNCTION`, have changed category, usually to `EXECUTOR`. When a first level logging category is included in the option `FileMask`, all second level subcategories pertaining to it will also be implicitly included, thus, it is redundant to list one ore more subcategories along with the concerned first level category.

==== Component and Plugin Based Filtering

It is possible to filter events based on the identity of the component generating them. For component designation it is recommended to use the component name given in the command `create` or the keyword `mtc`; latter one in the case of the Main Test Component. Using component numbers as identifiers is not recommended as this is a tool dependent identifier.

The component name, if present, precedes the keyword `FileMask`. They are separated by a dot (.).

It is also possible to apply the filtering on selected logger plugins of a component. The identifier of the desired logger plugin is appended after the component designation. The component and plugin identifiers are separated by a dot(.).

[[consolemask]]
=== `ConsoleMask`

Option `ConsoleMask` determines which events will be written to the console and which ones will be filtered.

[[category-based-filtering-0]]
==== Category-based Filtering

Table 14 enumerates the first level groups of events (that is, logging categories) along with a symbolic constant. The selected constants separated by a pipe (|) will determine what will be logged. When `ConsoleMask` is omitted from the configuration file, its default value (`ERROR|WARNING|ACTION |TESTCASE|STATISTICS`) is applied.

Some of the first level logging categories may be divided in second level subcategories to get a finer logging granularity. These categories are listed in the tables 11 to 21. First level categories and second level subcategories may be mixed in the option.

First level logging categories may be considered as notational convenience. They are maintained also for backward compatibility: legacy configuration files containing only first level categories will continue to work unmodified. However, the resulting log file may look different, as event categories have been modified; some messages, mostly `PARALLEL` and `FUNCTION`, have changed category, usually to `EXECUTOR`. When a first level logging category is included in the option `ConsoleMask`, all second level subcategories pertaining to it will also be implicitly included, thus, it is redundant to list one ore more subcategories along with the concerned first level category.

In single mode the console log messages are written to the standard error of the ETS. For the interpretation of console logging in parallel mode, see section 12.3.3. of the TITAN User Guide (<<13-references.adoc#_13, [13]>>).

WARNING: Please note that neither the timestamp nor the event type, nor location information is printed on the console.

[[component-and-plugin-based-filtering-0]]
==== Component and Plugin Based Filtering

It is possible to filter events based on the identity of the component generating them. For component designation it is recommended to use the component name given in the command `create` or the keyword `mtc`; latter one in the case of the Main Test Component. Using component numbers as identifiers is not recommended as this is a tool dependent identifier.

The component name, if present, precedes the keyword `ConsoleMask`. They are separated by a dot (.).

It is also possible to apply the filtering on selected logger plugins of a component. The identifier of the desired logger plugin is appended after the component designation. The component and plugin identifiers are separated by a dot (.).

.First level (coarse) log filtering
[cols=",",options="header",]
|===
Kristof Szabados's avatar
Kristof Szabados committed
719
|Logging categories |Symbolic constantfootnote:[In order to be compatible with older configuration files, the following symbolic constants are also recognized: TTCN_ERROR, TTCN_WARNING, TTCN_PORTEVENT, TTCN_TIMEROP, TTCN_VERDICTOP, TTCN_DEFAULTOP, TTCN_TESTCASE, TTCN_ACTION, TTCN_USER, TTCN_FUNCTION, TTCN_STATISTICS, TTCN_PARALLEL, TTCN_EXECUTOR, TTCN_MATCHING and TTCN_DEBUG. These constants have the same meaning as their counterparts without the prefix TTCN_.]
balaskoa's avatar
balaskoa committed
720
|TTCN-3 action(…) statements(No subcategory is implemented) |`ACTION`
Elemer Lelik's avatar
Elemer Lelik committed
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
|Debug messages in Test Ports and external functions(For subcategories see Table 15) |`DEBUG`
|Default operations (`activate`, `deactivate`, `return`)(For subcategories see Table 16) |`DEFAULTOP`
|Dynamic test case errors (e.g. snapshot matching failures)(No subcategory is implemented) |`ERROR`
|Internal TITAN events(For subcategories see Table 17) |`EXECUTOR`
|Random number function calls(For subcategories see Note: `EXECUTOR_EXTCOMMAND` is always logged, regardless of the user’s settings. |`FUNCTION`
|Bitwise OR of all the above except `MATCHING` and `DEBUG` |`LOG_ALL`
|Nothing to be logged |`LOG_NOTHING`
|Analysis of template matching failures in receiving port operations(For subcategories see Table 19) |`MATCHING`
|Parallel test execution and test configuration related operations (`create`, `done`, `connect`, `map`, etc.)(For subcategories see Table 20) |`PARALLEL`
|Port events (`send`, `receive`)(For subcategories see Table 22) |`PORTEVENT`
|Statistics of verdicts at the end of execution(For subcategories see Table 23) |`STATISTICS`
|The start, the end and the final verdict of test cases(For subcategories see Table 21) |`TESTCASE`
|Timer operations (`start`, `stop`, `timeout`, `read`)(For subcategories see Table 24) |`TIMEROP`
|User log(…) statements(No subcategory is implemented) |`USER`
|Verdict operations (`setverdict`, `getverdict`)(For subcategories see Table 25) |`VERDICTOP`
|Run-time warnings (e.g. stopping of an inactive timer)(No subcategory is implemented) |`WARNING`
|===

.Second level (fine) log filtering for *DEBUG*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|Debug information coming from generated functions of dual faced ports and built-in encoder/decoders.footnote:[Everyone writing encoder/decoder functions should implement logging in this subcategory.] |`DEBUG_ENCDEC`
| |`DEBUG_TESTPORT`
|Other, non categorized log messages of the category |`DEBUG_UNQUALIFIED`
|===

.Second level (fine) log filtering for *DEFAULTOP*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|TTCN-3 `activate` statement (activation of a default) |`DEFAULTOP_ACTIVATE`
|Deactivation of a `default` |`DEFAULTOP_DEACTIVATE`
|Leaving an invoked default at the end of a branch (causing leaving the `alt` statement in which it was invoked) or calling `repeat` in an invoked default (causing new snapshot and evaluation of the `alt` statement) |`DEFAULTOP_EXIT`
|Other, non categorized log messages of the category |`DEFAULTOP_UNQUALIFIED`
|===

.Second level (fine) log filtering for *EXECUTOR*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|Starting and stopping MTC and HCs |`EXECUTOR_COMPONENT`
|ETS runtime events (user control of execution, control connections between the processes of the ETS, ETS overloaded messages, etc.) |`EXECUTOR_RUNTIME`
|Runtime test configuration data processing |`EXECUTOR_CONFIGDATA`
|When this subcategory is present in the configuration file, logging options are printed in the second line of each log file. |`EXECUTOR_LOGOPTIONS`
|Running of external command |`EXECUTOR_EXTCOMMAND`
|Other, non categorized log messages of the category |`EXECUTOR_UNQUALIFIED`
|===
NOTE: `EXECUTOR_EXTCOMMAND` is always logged, regardless of the user’s settings.

.Second level (fine) log filtering for *FUNCTION*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|Random number functions in TTCN-3 |`FUNCTION_RND`
|Other, non categorized log messages of the category |`FUNCTION_UNQUALIFIED`
|===

.Second level (fine) log filtering for *MATCHING*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|Matching a TTCN-3 `done` operation |`MATCHING_DONE`
|Timer in timeout operation is not started or not on the list of expired timers |`MATCHING_TIMEOUT`
|Procedure-based mapped ports: successful template matching |`MATCHING_PMSUCCESS`
|Procedure-based mapped ports: unsuccessful template matching |`MATCHING_PMUNSUCC`
|Procedure-based connected ports: successful template matching |`MATCHING_PCSUCCESS`
|Procedure-based connected ports: unsuccessful template matching |`MATCHING_PCUNSUCC`
|Message-based mapped ports: successful template matching |`MATCHING_MMSUCCESS`
|Message-based mapped ports: unsuccessful template matching |`MATCHING_MMUNSUCC`
|Message-based connected ports: successful template matching |`MATCHING_MCSUCCESS`
|Message-based connected ports: unsuccessful template matching |`MATCHING_MCUNSUCC`
|Unsuccessful matching |`MATCHING_PROBLEM`
|Other, non categorized log messages of the category |`MATCHING_UNQUALIFIED`
|===

.Second level (fine) log filtering for *PARALLEL*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|PTC creation and finishing, starting and finishing a function started on a PTC |`PARALLEL_PTC`
|Port `connect` and `disconnect` operations |`PARALLEL_PORTCONN`
|Port `map` and `unmap` operations |`PARALLEL_PORTMAP`
|Other, non categorized log messages of the category |`PARALLEL_UNQUALIFIED`
|===

.Second level (fine) log filtering for *TESTCASE*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|A testcase is starting |`TESTCASE_START`
|A testcase ends; final verdict of the testcase |`TESTCASE_FINISH`
|Other, non categorized log messages of the category |`TESTCASE_UNQUALIFIED`
|===

.Second level (fine) log filtering for *PORTEVENT*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|Procedure-based ports: call, reply or exception enqueued in the queue of the port or extracted from the queue |`PORTEVENT_PQUEUE`
|Message-based ports: message enqueued in the queue of the port or extracted from the queue |`PORTEVENT_MQUEUE`
|Port state changesfootnote:[In mixed ports message and proc. ports cannot be distinguished] (`halt`, `start`, `stop`, port `clear` operation finished) |`PORTEVENT_STATE`
|Procedure-based mapped ports: incoming data received (`getcall`, `getreply`, `catch`, `check`) |`PORTEVENT_PMIN`
|Procedure-based mapped ports: outgoing data sent (`call`, `reply`, `raise`) |`PORTEVENT_PMOUT`
|Procedure-based connected ports: incoming data received (`getcall`, `getreply`, `catch`, `check`) |`PORTEVENT_PCIN`
|Procedure-based connected ports: outgoing data sent (`call`, `reply`, `raise`) |`PORTEVENT_PCOUT`
|Message-based mapped ports: incoming data received (`receive`, `trigger`, `check`) |`PORTEVENT_MMRECV`
|Message-based mapped ports: outgoing data sent (`send`) |`PORTEVENT_MMSEND`
|Message-based connected ports: incoming data received (`receive`, `trigger`, `check`) |`PORTEVENT_MCRECV`
|Message-based connected ports: outgoing data sent (`send`) |`PORTEVENT_MCSEND`
|Mappings of incoming message from the external interface of dual-faced ports to the internal interface (decoding) |`PORTEVENT_DUALRECV`
|Mappings of outgoing message from the internal interface of dual-faced ports to the external interface (encoding) |`PORTEVENT_DUALSEND`
|Other, non categorized log messages of the category |`PORTEVENT_UNQUALIFIED`
|===

.Second level (fine) log filtering for *STATISTICS*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|Verdict statistics of executed test cases (% of `none`, `pass`, `inconc`, `fail`, `error`) |`STATISTICS_VERDICT`
|Other, non categorized log messages of the category |`STATISTICS_UNQUALIFIED`
|===

.Second level (fine) log filtering for *TIMEROP*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|TTCN-3 `read timer` operation |`TIMEROP_READ`
|TTCN-3 `start timer` operation |`TIMEROP_START`
|Log events related to the guard timer used in TTCN-3 `execute` statements |`TIMEROP_GUARD`
|TTCN-3 `stop timer` operation |`TIMEROP_STOP`
|Successful TTCN-3 `timeout` operation (timer found on the list of expired timers) |`TIMEROP_TIMEOUT`
|Other, non categorized log messages of the category |`TIMEROP_UNQUALIFIED`
|===

.Second level (fine) log filtering for *VERDICTOP*
[cols=",",options="header",]
|===
|Logging subcategories |Symbolic constant
|TTCN-3 `getverdict` operation |`VERDICTOP_GETVERDICT`
|TTCN-3 `setverdict` operation |`VERDICTOP_SETVERDICT`
|Final verdict of a test component (MTC or PTC) |`VERDICTOP_FINAL`
|Other, non categorized log messages of the category |`VERDICTOP_UNQUALIFIED`
|===

[[AppendFile]]
=== `AppendFile`

Option `AppendFile` controls whether the run-time environment shall keep the contents of existing log files when starting execution. The possible values are `Yes` or `No`. The default is `No`, which means that all events from the previous test execution will be overwritten.

This option can be used in both single and parallel modes. Its usefulness in single mode is obvious. If the executable test suite is started several times, the logs of the successive test sessions will be stored in the same single file after each other.

In parallel mode the naming of log files is automatic and is based on the host name and component references. The option is applicable to all log files: all of them will be either appended or overwritten. If the test execution is repeated several times with different configuration or test case selection, the same file may contain the log of totally different test components. When appending is enabled the log files can be interpreted after using the logmerge utility (see Section 13.1. of the TITAN User Guide, <<13-references.adoc#_13, [13]>>). The option `AppendFile` guarantees only that no logged events will be lost during the entire test campaign.

[[TimeStampFormat]]
=== `TimeStampFormat`

Option `TimeStampFormat` configures the formatting of timestamps in the log file. It can have three possible values: `Time` stands for the format `hh:mm:ss.microsec`. `DateTime` results in `yyyy/Mon/dd hh:mm:ss.microsec`. This is useful for long test durations (for instance, when a stability test runs for a couple of days). Using the third alternative (`Seconds`) results relative timestamps in format `s.microsec`. The time is related to the starting of the test component or test execution (i.e. this is the zero time). The default value for `TimeStampFormat` is `Time`.

=== `ConsoleTimeStampFormat`

Option `ConsoleTimeStampFormat` configures the formatting of timestamps in console log. It can have the same three values as `TimeStampFormat` can have: `Time`, `DateTime` and `Seconds` (see <<TimeStampFormat, here>>). If it is omitted (default) timestamp will not be inserted in the console log.

=== `LogEventTypes`

Option `LogEventTypes` indicates whether to include the symbolic event type (without the TTCN prefix) in each logged event immediately after the timestamp. This option can be useful for log post-filtering scripts. The possible values for `LogEventTypes` are `Yes`, `No`, `Detailed` and `Subcategories`.

The default is `No`: no events will be logged.

The setting `Yes` results a logfile containing event categories listed in Table 14.

The setting `Subcategories` (and the equivalent `Detailed`) produces a logfile containing both event categories and subcategories. Subcategories are listed in the tables 11 to 21.

In both single and parallel modes some log events are created before processing the configuration data. At this time the logging options (name of the log file, filter settings, timestamp format, etc.) are not known by the run-time environment, thus, these events are collected in a temporary memory buffer and are flushed to the log file when the processing of configuration file is finished. This implies that if the Host Controller is stopped in parallel mode before configuring it, no log file will be created at all.

=== `SourceInfoFormat`

balaskoa's avatar
balaskoa committed
898
Option `SourceInfoFormat` controls the appearance of the location information for the test events. The location information refers to a position in the TTCN-3 source code. It consists of the name of the TTCN-3 file and the line number separated by a colon character (:). Optionally, it may contain the name of the TTCN-3 entity (function, testcase, etc.) in parentheses that the source line belongs to. See also the option <<LogEntityName, `LogEntityName`>> below.
Elemer Lelik's avatar
Elemer Lelik committed
899

balaskoa's avatar
balaskoa committed
900
The option can take one of the three possible values: `None`, `Single` and `Stack`. In case of `Single`, the location information of the TTCN-3 statement is logged that is currently being executed. When `Stack` is used the entire TTCN-3 call stack is logged. The logged information starts from the outermost control part or testcase and ends with the innermost TTCN-3 statement. An arrow (that is, the character sequence ->) is used as separator between the stack frames. The value `None` disables the printing of location information. The default value for `SourceInfoFormat` is `None`.
Elemer Lelik's avatar
Elemer Lelik committed
901
902
903

The location information is placed in each line of the log file between the event type or timestamp and the textual description of the event.

balaskoa's avatar
balaskoa committed
904
This option works only if the command line option `-L` is passed to the compiler (see <<6-compiling_ttcn3_and_asn1_modules.adoc#command-line-syntax, here>>). This feature is useful for debugging new TTCN-3 code or for understanding the traces of complex control constructs. If the location information is not generated into the {cpp} code the executable tests run faster, which can be more important when doing performance tests.
Elemer Lelik's avatar
Elemer Lelik committed
905

balaskoa's avatar
balaskoa committed
906
NOTE: The reception of messages or procedure calls can only occur while the run-time environment is taking a new snapshot. A new snapshot is taken when the testcomponent is evaluating a stand-alone receiving operation, an `alt` construct or a standalone `altstep` invocation. Thus, the location information of the incoming messages or calls points to the first line of the above statements. The log event belonging to a TTCN-3 operation can be the extraction of a message from the port queue and not the reception of an incoming message.
Elemer Lelik's avatar
Elemer Lelik committed
907

balaskoa's avatar
balaskoa committed
908
If the event has no associated line in the TTCN-3 source code (e.g. because it belongs to test environment startup or termination) and `SourceInfoFormat` is set to either `Single` or `Stack`, a single dash character `(-)` is printed into the log instead of the location information. This makes the automated processing of log files easier.
Elemer Lelik's avatar
Elemer Lelik committed
909
910
911
912
913
914

The obsolete option `LogSourceInfo` is also accepted for backward compatibility with earlier versions. Setting `LogSourceInfo` `:= Yes` is equivalent to `SourceInfoFormat` `:= Single`, and similarly `LogSourceInfo := No` means `SourceInfoFormat := None`.

[[LogEntityName]]
=== `LogEntityName`

balaskoa's avatar
balaskoa committed
915
Option `LogEntityName` controls whether the name of the corresponding TTCN-3 entity (`function`, `testcase`, `altstep, control` part, etc.) shall be included in the location information. If this option is set to `Yes`, the file name and line number is followed by the name of the TTCN-3 entity within parentheses. The default value is `No`. The option has no effect if `SourceInfoFormat` is set to `None`.
Elemer Lelik's avatar
Elemer Lelik committed
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937

=== `LogFileSize`

Option `LogFileSize` sets the upper limit for the log file size. The limitation prevents load tests or long duration tests from triggering dynamic test case error when the growing log file exceeds file system size limits or available disk space.

When the size limit is reached, the file is closed and a new log file will be created with an increased part number. For example, the first two log files when running `ExampleTestCase` in single mode will be `ExampleTestCase-part1.log` and `ExampleTestCase-part2.log`, respectively provided that the file name skeleton default values have not been modified.

This option must be set together with <<LogFileNumber,`LogFileNumber`>>.

The parameter value, a non-negative integer, is understood in kilobytes. The default value is 0, meaning that the file size is unlimited; or, to be precise, is only limited by the file system.

==== Component and Plugin Dependent File Size

It is possible to set different file sizes based on the identity of the component generating the log. For component designation it is recommended to use the component name given in the parameter of the command `create` (or the keyword `mtc` for the Main Test Component). Using component numbers as identifiers is not recommended as this is a tool dependent identifier.

The component name, if present, precedes the keyword `LogFileSize`. The name and the keyword are separated by a dot (.).

It is also possible to limit the file size on selected logger plugins of a component. The identifier of the desired logger plugin is appended after the component designation. The component and plugin identifiers are separated by a dot (.).

[[LogFileNumber]]
=== `LogFileNumber`

Kristof Szabados's avatar
Kristof Szabados committed
938
Option `LogFileNumber`, a positive integer, sets the maximum number of log files (fragments) kept. If the log file number limit is reached, the oldest log file of the component will be deleted and logging continues in the next log fragment file.
Elemer Lelik's avatar
Elemer Lelik committed
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000

The default value is 1, meaning that the number of log files equals one.

==== Component and Plugin Dependent Fragment Number

It is possible to set different fragment limits based on the identity of the component generating the log. For component designation it is recommended to use the component name given in the parameter of the command `create` (or the keyword `mtc` for the Main Test Component). Using component numbers as identifiers is not recommended as this is a tool dependent identifier.

The component name, if present, precedes the keyword `LogFileNumber`. The name and the keyword are separated by a dot (.).

It is also possible to limit the number of log fragments on selected logger plugins of a component. The identifier of the desired logger plugin is appended after the component designation. The component and plugin identifiers are separated by a dot (.).

=== `DiskFullAction`

Option `DiskFullAction` determines TITAN behavior when writing to the log file fails.

If this option set to `Stop` test case execution continues without logging when an error occurs.

The setting `Retry` causes test case execution to continue without logging and TITAN attempts to restart logging activity periodically (events in the unlogged periods are lost). The retry period is set by default to 30 seconds and can be changed by a parameter. Example: `Retry`(`60`) doubles the period.

If the parameter is set to `Delete`, TITAN deletes the oldest log file and continues logging to a new log file fragment. If log writing fails again, the procedure is repeated until one two log files (the actual one and the previous one) are left. Further log writing failure causes a dynamic test case error.

The default behavior is `Error`. With this setting, writing error causes a runtime (dynamic) test case error.

==== Component and Plugin Dependent Behavior

It is possible to set different error behavior based on the identity of the component generating the log. For component designation it is recommended to use the component name given in the parameter of the command `create` (or the keyword `mtc` for the Main Test Component). Using component numbers as identifiers is not recommended as this is a tool dependent identifier.

The component name, if present, precedes the keyword `DiskFullAction`. The name and the keyword are separated by a dot (.).

It is also possible configure different error behavior on selected logger plugins of a component. The identifier of the desired logger plugin is appended after the component designation. The component and plugin identifiers are separated by a dot (.).

=== `MatchingHints`

Option `MatchingHints` controls the amount and format of log messages describing template matching failures. These are written during port receive operations as logging category `MATCHING`, and as a response to TTCN-3 `log(match(…))` statements as logging category `USER`.

There are two possible values: `Compact` and `Detailed`.

When the `Detailed` option is in effect, a field-by-field description of the value and template is logged, followed by additional hints when matching set-of types. Example:

[source]
----
{
    {
        field_rr1 := 1,
        field_rr2 := 2
    },
    {
        field_rr1 := 3,
        field_rr2 := 4
    }
} with {
    {
        field_rr1 := 1,
        field_rr2 := 2
    },
    {
        field_rr1 := 3,
        field_rr2 := 5
    }
} unmatched Some hints to find the reason of mismatch: {
    value elements that have no pairs in the template: {
        field_rr1 := 3,
For faster browsing, not all history is shown. View entire blame