-
René Paris authored
- Updates generated code - Adds new actions and conditions - Adds new conversion stubs - Updates all copyrights of changed files
René Paris authored- Updates generated code - Adds new actions and conditions - Adds new conversion stubs - Updates all copyrights of changed files
OpenScenarioEngine
- User Guide
- Architecture Documentation
- Generator Documentation (this document)
Generator Documentation
The behavior tree of the OpenScenarioEngine is created for the most part through the utilization of a Python code generator. In the following the background, basic usage, and technical details, such as integration with manually written code or schema processing will be described in detail.
For convenience, the generated files are already checked into the repository (
engine/gen
), so calling the generator is optional.
Background
Parallel to the development of OpenSCENARIO support for parsing and importing of OpenSCENARIO XML conform descriptions into C/C++ objects have been developed, and are available through the OpenSCENARIO API and OpenScenarioLib. Both are based on the official ASAM OpenSCENARIO XML specification, ensuring full compliance with the standard, and generated from a well defined UML model, also available in the sources.
When an OpenSCENARIO description is being read by the parser component of the OpenScenarioLib, it maps each individual tag onto an object that implements the OpenSCENARIO API, such as the Storyboard
or SpeedAction
class.
While this provides access to validated scenario file information, it does not provide the logical relationship between objects.
The objective of the OpenScenarioEngine generator is to extract maximum logical relationship data from the XML model and augment it with metadata to create a functional behavior tree.
In order to do so, the UML model for both the OpenSCENARIO API and OpenScenarioLib was further used to generate the basis for the OpenScenarioEngine.
This means that the generator uses the emitted json
representation of the OpenSCENARIO API, also stored here: generator/model/v1_3.json
.
Usage
cd generator
python3 generate.py
Command Line Arguments
usage: generate.py [-h] [--clang-format CLANG_FORMAT] [-l LOG_FILE] [-d] [-c CLASSES [CLASSES ...]] [-f]
OpenScenarioEngine CodeGenerator
optional arguments:
-h, --help show this help message and exit
--clang-format CLANG_FORMAT
Specifies executable for clang-format
-l LOG_FILE, --log_file LOG_FILE
Name of log file
-d, --debug Additionally print debug information
-c CLASSES [CLASSES ...], --classes CLASSES [CLASSES ...]
Restricts generation scope to given class (e.g. TeleportAction)
-f, --force Forces generation even for detected user specializations
The generator applies clang-format on every C++ source file.
The path can be specified (mandatory under Windows), e.g. by callingpython3 generate.py --clang-format="c:\clang-format.exe"
.
Be aware that generation under Windows might mix up line endings w.r.t the files already checked-in.
Dependencies
- Python 3.10
- jinja2
- case-converter
Technical Details
If you are not familiar with the underlying behavior tree, it's strongly recommended to read the Architecture Documentation first.
Before generation begins, the UML model generator/model/v1_3.json
is converted into a tree:
Storyboard (1)
├── Init
│ └── InitActions
│ ├── GlobalAction
│ │ ├── EnvironmentAction (2)
│ │ └── ...
│ ├── UserDefinedAction
│ │ └── ...
│ └── Private
│ └── ...
├── Stories
│ └── ...
└── StopTrigger
└── ConditionGroup
└── Condition
├── ByEntityCondition
│ ├── TriggeringEntities
│ └── EntityCondition
│ └── EndOfRoadCondition (2)
└── ByValueCondition
Legend
- Hardcoded root of the tree
- Leaves of the tree = Actual condition or action, which need to be implemented manually.
In general, the generator distinguishes between leaves and "everything else" (aka nodes).
In order to do so, the generator need a selector
, which defines a certain branch of the tree (see metainfo.py
).
'selector':
[
# node, pattern, ose_type, dependencies
('InitActions', '\w+Action$', 'Storyboard/GenericAction', ['Environment']),
('Action', '\w+Action$', 'Storyboard/GenericAction', ['Environment']),
]
For each selector
, the generator searches for matching (regex) pattern
s within the types of the UML model.
If a match is found and the root
of the selector is part of the branch, the whole branch is being processed.
Note:
dependencies
are discussed further down
For the first selector
of the example, a matching branch could be:
Storyboard > Init > InitActions > GlobalAction > EnvironmentAction
^ start ^ required node ^ matches \w+Action$
-
Storyboard
,Init
,InitActions
, andGlobalAction
are processed as Node. -
EnvironmentAction
will e processed as Leaf. -
EnvironmentAction
also has children, such asEnvironment
, which will become Parameters of the action.
Processing of Nodes, Leaves, and Parameters are discussed further down in detail. For sake of clarity, Integration of Generated Code into the file structure of the engine and Definition of Dependencies will be described first:
Integration of Generated Code
This section describes how manually written code and generated code play together. From the perspective of the code generator, the following directories are most important:
engine
├── gen . . . . . . . . . . . . (1)
│ ├── Conversion
│ │ └── OscToNode . . . . . (2)
│ └── Storyboard . . . . . . (3)
│ ├── ByEntityCondition
│ ├── ByValueCondition
│ ├── GenericAction
│ └── MotionControlAction
└── src . . . . . . . . . . . . (4)
├── Conversion
│ ├── OscToMantle . . . . (5)
│ └── OscToNode
├── Node . . . . . . . . . (6)
├── Storyboard
│ ├── ByEntityCondition
│ ├── ByValueCondition
│ ├── GenericAction
│ └── MotionControlAction
(7)
Legend
- Generated files are located within this subdirectory.
- All Node parsers are located here (= everything except leaves).
- Leaf code is located here.
- Manually written files are located within this subdirectory.
-
Parameter converters are located here.
Currently not generated, but there is commented out code for generating stubs (withingen
). - Customized yase nodes, used by certain
OscToNode
parsers. - The actual engine has several other source files, but they are not generated.
Interaction between generated and manually written files is done on a file basis, meaning that a file either exists in the file structure of gen
or src
.
Therefore, the generator always checks if a corresponding file already exists in src
and if so, it will be skipped.
Use
--force
to suppress this check. After forced file generation, a diff betweengen
andsrc
can be useful when adding new features.
The generator generates the complete storyboard parsers and leaves.
For unimplemented conditions or actions (= no corresponding files within src
), stubs will be generated.
The stubs are intended as entry point for implementers. Files which need modification, such as an unimplemented condition, can be simply moved from
gen
tosrc
(example provided here). From this moment on, the generator will simply skip generation for this file. The correspondingcmake file
is updated as last step, when executing the generator.
Defining Dependencies
Dependencies are needed by conditions and actions (via the aforementioned selector
), and by some parameter converters (see Processing Parameter further below).
Right now, new dependencies need to be (manually) provided by blackboard of the behavior trees root (see yase documentation and src/Node/RootNode.h
).
Once available, leaves and attached parameter converters can parameterized to get access to their dependencies over the nodes blackboard.
Available dependencies are defined in the metainfo.py
:
'dependencies':
{
// reference used in selector and converters
'Environment':
{
// variable name, if applicable
'name': 'environment',
// type, if applicable
'type': 'std::shared_ptr<mantle_api::IEnvironment>',
// include file
'include': '<MantleAPI/Execution/i_environment.h>'
}
}
Processing Nodes
Given a branch of the model tree, the generator starts to generate C++ files
for the nodes parser by calling generate_parser()
.
The files will be generated in the folder engine/gen/Conversion/OscToNodes
(see also Architecture Documentation).
Children (= properties) stand in a certain relation to their parent or to each other. The generator distinguishes the following:
-
xor
The processes OpenScenario API object offer getters for each xor-ed child, but in a valid scenario description one and only one getter will provide an instantiated object. So one and only one paths is valid.
Applies To
Properties of a node with attributeisXorElement
set toTrue
.Example
TheCondition
object (c.f. line 2361 inv1_3.json
) can hold either aByEntityCondition
or aByValueCondition
, both properties in an xor-relationship.The generator will create the following code:
yase::BehaviorNode::Ptr parse(std::shared_ptr<NET_ASAM_OPENSCENARIO::v1_3::ICondition> condition) { if (auto element = condition->GetByEntityCondition()) { return parse(element); } if (auto element = condition->GetByValueCondition()) { return parse(element); } throw std::runtime_error("Corrupted openSCENARIO file: No choice made within std::shared_ptr<NET_ASAM_OPENSCENARIO::v1_3::ICondition>"); }
Used Templates
-
list
The processed OpenScenario API object offers a getter which returns a list of sub-objects.
Applies To
Properties of a node with attributeisList
set toTrue
.Example
TheStory
object (c.f. line 11302 inv1_3.json
) holds a list of acts. Processing needs to consider, thatparse(story->GetActs())
is a vector ofAct
objects.Before continuing with the next sub-type
Act
, the generator will create a list-seam forActs
:yase::BehaviorNode::Ptr parse(std::vector<std::shared_ptr<NET_ASAM_OPENSCENARIO::v1_3::IAct>> acts) { auto node = std::make_shared<yase::ParallelNode>("Acts"); for (const auto& act : acts) { node->addChild(parse(act)); } return node; }
Used Templates
-
Transient
The processed OpenScenario API only contains a single child.
Applies To
Checked by logic.Example
TheTrigger
object (c.f. line 13681 inv1_3.json
) holds only single elementconditionGroups
.The generator will create the following code:
yase::BehaviorNode::Ptr parse(std::shared_ptr<NET_ASAM_OPENSCENARIO::v1_3::ITrigger> trigger) { return trigger ? parse(trigger->GetConditionGroups()) : nullptr; }
The code assumes that all types are optional. This is a valid simplification as it is expected that scenario descriptions are validated before parsing.
Used Templates
-
Glue Logic
The processed OpenScenario API object is a leaf.
Applies To
Checked by logic.Example
TheAddEntityAction
object.The generator will create the following code:
yase::BehaviorNode::Ptr parse(std::shared_ptr<NET_ASAM_OPENSCENARIO::v1_3::IAddEntityAction> addEntityAction) { return std::make_shared<AddEntityAction>(addEntityAction); }
The code makes use of a specialized
AddEntityAction
(yase) node, which will be generated during processing of the leaves (see next chapter).Used Templates
Processing Leaves
For leaves, which are either conditions or actions, the corresponding stubs are generated. As described here), these stubs are always 4 files:
- A yase node
- A base class header holding parameters
- A derived implementation header
- The actual implementation
Recalling the selector described here, a leaf matching pattern
is always accompanied by an ose_type
.
This ose_type
selects the corresponding leaf templates.
Example
For the leaf SimulationTimeCondition
, the ose_type
will resolve to Storyboard/ByValueCondition
, picking the following templates:
- Storyboard/ByValueCondition.h.j2
- Storyboard/ByValueCondition_base.h.j2
- Storyboard/ByValueCondition_impl.h.j2
- Storyboard/ByValueCondition_impl.cpp.j2
Note that a SimulationTimeCondition
does not rely on an entity, while a TraveledDistanceCondition
certainly does.
So template selection is used to separates conditions and actions with an entity relation from those without.
Entities with entity relation get access to a service named EntityBroker
, provided by the blackboard of the yase
tree (see https://gitlab.eclipse.org/eclipse/openpass/yase/-/tree/main/agnostic_behavior_tree).
Processing Parameter
Augmenting model data with metadata is also necessary for decoupling of objects implementing the OpenSCENARIO API from the actual implementations of conditions and actions, formulated by the design principle separation of values described in the Architecture Documentation. In essence, the generator possesses knowledge regarding the mappings between properties from the OpenSCENARIO API and internally used values. The most used mapping is for the complex datatype Position offering diverse specializations, such as WorldPosition, RelativeLanePosition, or GeoPosition. By applying the mapping, the specification (which specifies the source for the position) is transformed into its result - a Cartesian point and an orientation.
Example
OpenScenarioAPI OpenScenarioEngine
┌────────────────────────┐ ┌─────────────────────────┐
│ ReachPositionCondition │ │ ReachPositionCondition │
├────────────────────────┤ ├─────────────────────────┤
│ + double tolerance │ │ + double tolerance │
│ + Position position │ - mapping -> │ + mantle_api::Pose pose │
└────────────────────────┘ └─────────────────────────┘
Definition
The corresponding conversion is defined in the metainfo.py
:
'converter':
{
// **OpenScenario API** type, which shall be mapped
'Position':
{
// Function name of the converter
'converter': 'OPENSCENARIO::ConvertScenarioPosition',
// Return type of the converter
'return_type': 'std::optional<mantle_api::Pose>',
// Necessary include file
'include': '"Conversion/OscToMantle/ConvertScenarioPosition.h"',
// type == "static": Call converter on initialization
// type == "dynamic: Call converter on demand (wrap in lambda)
'type': 'static',
// Optional: dependencies of the converter
'dependencies':
[
"Environment"
]
// Other properties of the leaf, necessary for conversion
'consumes': ['Foo', 'Bar']
}, ...
}
Note that parameter belong to leaves and the corresponding parsers are used by the generating templates.
Output
// from "dependencies"
#include <MantleAPI/Execution/i_environment.h>
// from "include"
#include "Conversion/OscToMantle/ConvertScenarioPosition.h"
...
// When approaching parameter "Position":
[=](){ // <- dynamic: will be mapped to "GetPosition()"
// "converter"
return OPENSCENARIO::ConvertScenarioPosition(
// from "dependencies"
environment,
// from "Position"
condition->GetPosition(),
// Consumes Foo
condition->GetFoo()
// Does not consumes Bar, because not found <- no error
);
}
Note that consume really means that the property (Foo) is removed from the list of properties, which shall be directly mapped.
For generic usage, it is allowed to specify multiple parameters, but only those available will be consumed.
Processing Rules
Rules are a particular type of parameter that allow for logical comparisons between values. An example of such a comparison could be determining if the current value of a condition is greater than or equal to another parameter within the node.
Therefore the rule parser is configured to consume the corresponding value of the node and instead provide a class which already contains the correct logic for checking if the rule is satisfied (see engine/src/Conversion/OscToMantle/ConvertScenarioRule.h
)