AsciiDoc TCK
Specification projects under the Eclipse Foundation Specification Process MUST produce a Technology Compatibility Kit (TCK) that delivers on the promise of enabling multiple compatible implementations.
It’s the role of the TCK to ensure both compatibility of implementations and portability of applications. Implementations that want to be certified must follow the AsciiDoc TCK Process.
Purpose
The purpose of the AsciiDoc Language TCK is to verify that an AsciiDoc implementation (i.e., AsciiDoc processor) correctly comprehends the AsciiDoc format according to the AsciiDoc Language Specification. This mostly entails validating that an AsciiDoc document is correctly parsed, though some additional processor behaviors are also required and verified.
The TCK does not currently validate how the parsed model is converted to other formats such as HTML and DocBook. The goal, at present, is to ensure that there’s a consistent interpretation of the AsciiDoc Language and, therefore, to avoid the emergence of variants.
Architecture
A key goal of the AsciiDoc Language specification is to foster the development of compatible AsciiDoc implementations across different programming languages and runtimes. By contrast, we don’t want to have to rewrite the test suite for each of those languages. That means the tests must be language agnostic. A practical way to satisfy this requirement is to design the test suite to be data-oriented. The TCK exchanges input and output data with the implementation to determine whether the implementation properly parses and comprehends AsciiDoc source.
The TCK is centered around the concept of an Abstract Semantic Graph (ASG). Here’s how an ASG is defined on Wikipedia:
An abstract semantic graph (ASG) is a form of abstract syntax in which an expression of a formal or programming language is represented by a graph whose vertices are the expression’s subterms. An ASG is at a higher level of abstraction than an abstract syntax tree (AST), which is used to express the syntactic structure of an expression or program.
AsciiDoc is not a regular language like XML, meaning the guidelines for how to parse it is not inherited by another specification. Therefore, it’s necessary to have some other way to express its inherent structure. That structure is captured by an ASG. We can say that the AsciiDoc ASG is a hierarchical model of a parsed and interpreted AsciiDoc document. The ASG will also capture and log messages that the implementation is expected to produce.
The AsciiDoc ASG captures the semantic information writers understand to be encoded in an AsciiDoc document. It’s at a higher level than an Abstract Syntax Tree (AST) since it deals with structural elements and their meaning rather than just tokens. The distinction from an AST is important because the TCK must not dictate how the implementation works internally. The implementation is only required to present the ASG as proof that the implementation has comprehended the AsciiDoc document correctly.
If an AsciiDoc implementation can produce the expected ASG, we can assume it’s parsing and interpreting the AsciiDoc syntax in a compliant way. Simply put, the ASG is how we validate that an AsciiDoc implementation is compliant with the specification. We believe that the ASG comparison is a sufficient way of communicating that the implementation is following the specification.
In the TCK, the ASG is encoded in JSON backed by a JSON schema. JSON was chosen since it’s a portable and ubiquitous exchange format. By using JSON as the assertion payload, it’s possible for the TCK to be run on an AsciiDoc processor written in any language and running on any platform.
The TCK runs the implementation on a suite of tests that cover all the assertions made by the specification to verify the implementation is compliant. The operation of the TCK is described in the next section.
Operation
The TCK interfaces with the AsciiDoc implementation through a TCK adapter. The adapter must pass each test to the implementation to run and must return the outcome of that test from the implementation back to the TCK.
At a high level, the TCK presents the implementation with a test by way of the TCK adapter. A test is an AsciiDoc document and, if necessary, supplemental AsciiDoc documents (e.g., includes) or input parameters (e.g., attributes). (We’re considering wrapping the input in an delivery envelop). The implementation is expected to parse the document and produce an ASG that represents the information it extracted from that document. The implementation then returns that ASG in JSON format to the TCK by way of the adapter. The TCK then verifies whether the ASG produced by the implementation matches the expected ASG. If it does, the test is considered passed. If the two ASGs differ, the test is considered failed.
In order for an implementation to use the TCK, it must provide an adapter. An adapter accepts the test input (an AsciiDoc document, etc) and returns an ASG in JSON format.
Currently, the adapter must be a command that the TCK calls. The input is passed through stdin and is expected to produce the ASG in JSON format as stdout. We’re looking into supporting a client-server interface as an alternative.
The path to the adapter must be specified using the ASCIIDOC_TCK_ADAPTER environment variable.
With that environment variable is set, the asciidoc-tck command can be invoked.
The TCK will take over from there and run all the tests by hooking in the specified TCK adapter.
Requirements
In order to pass the AsciiDoc TCK tests, you must create a CLI interface that reads AsciiDoc content from stdin and return an Abstract Semantic Graph (encoded in JSON) to stdout. The CLI must return an exit code of 0 on success and non-zero in case of failure.