Initial project assessment

Purpose

The purpose of this issue is to begin to deconstruct the goals of the 1.0.0 release, identify areas of initial focus (enough to get started), and set a course towards that objective. It's a bootstrap issue. Once we've identified a task or decision to be made, it should be spawned as a dedicated issue. Each issue should only address one primary decision or task and have an "Acceptance criteria" section to reinforce that scope. If the issue is trying to do more than that, it should be broken down further. (We want to steer clear of issues that become too epic).

A portion of the issues spawned from this one will be sorted into milestones (specifically pre-1.0.0 releases). That scheduling process will continue throughout the 1.0.0 release cycle. Once the first milestone has been identified, and all substantial topics raised in this issue have been captured as issues or shelved, this issue should be closed. In other words, it's fate is to dissolve into other issues.

Methods

There are two ways to go about breaking down the goals of this project into actionable items. One is based on the artifacts that will need to be delivered and the other is based on the language anatomy.

Artifact-based

The deliverables in a stable release include the specification document, the TCK, and the API. We should think about what we need to do or what decisions need to be made in order to get these deliverables off the ground, then create issues for them.

For example:

• What will be the outline of the specification document?
• How is the TCK going to function (test harness, ASG, etc)?
• What format are we going to use for the ASG?
• What will be the structure of the ASG?
• How will APIs (including the DOM) be defined in a language independent way (e.g., IDL from XML DOM Core, UML, etc; see https://en.wikipedia.org/wiki/Language-independent_specification)?

Anatomy-based

Since the purpose of the specification is to define and extract the structure of the AsciiDoc language from text, we know we need a grammar. Granted, we're working from the position of retaining compatibility with existing content, but we need to move towards a formal grammar that can describe and validate that compatibility, especially across implementations. Here's a high-level (albeit crude and incomplete) sketch of the AsciiDoc grammar at the block level that we can use to think about the kinds of issues we'll need.

Document        ::= (Header?,Preamble?,Section*)
Header          ::= (Title,(AuthorInfo,RevisionInfo?)?,AttributeEntry*)
AuthorInfo      ::= (FirstName,(MiddleName?,LastName)?,Autolink?)
RevisionInfo    ::= (RevisionNumber?,RevisionDate,RevisionRemark?)
Preamble        ::= (Block*)
Section         ::= (Title,Block*,Section*)
Block           ::= (BlockTitle?,BlockAttributeList*,(Paragraph|DelimitedBlock|List|Table|BlockMacro))
DelimitedBlock  ::= (Admonition|Sidebar|Example|Blockquote|Listing|Literal|Open|Comment|Pass)
List            ::= ...

When we look at the structure as a whole, or any individual element, there are four aspects we need to consider: the terminology, the syntax/grammar, the ASG, and the DOM. Thus, we could end up with 4 issues for each element in the structure. We should think about what we need to do or what decisions need to be made in order to define the anatomy, then create issues for them.

For example:

• What notation system are we going to use to describe the grammar?
• How is the preprocessor going to impact our ability to make a grammar?
• How will syntax extensions be described in the grammar?
• What terms are we going to use to refer to the list of attributes in raw vs parsed form?
• Which space characters are permitted in the language and how are they going to be preserved?
• What are the rules for nesting blocks?
• Is it possible to define an inline grammar in a backwards-compatible way, or will we need to introduce it as an optional, forward-looking mode?

Ask, don't answer

The point of this issue is not to answer any of these questions. Any such discussions will be disregarded. The purpose of this issue is to identify and deliberate which topics need to be addressed and file issues to address them. So you can ask additional questions, but you can't give answers or try to solve the questions raised ;) We're also looking for any challenges we know we will face, either right up front or further down the line leading up to 1.0.0. Let's identify them and get them filed as issues.

Related ideas

We may want to think about capturing each major decision as an ADR (Architectural Decision Record). An ADR formalizes a decision by recording the premise, the outcome, the reason why the outcome was chosen, and any solutions that were discussed and/or discarded. I'm considering using the Requirements Management feature of GitLab for this purpose as it will make managing ADRs simple and lightweight. If we decide to move forward with this idea, it should be discussed in a separate issue.

Prospective issues

Here are a few topics that have come up already for which we know we need issues:

• How is the TCK going to function (test harness, ASG, etc)?
• What format should we use for the ASG?
• What will be the structure of the ASG?
• What grammar language / framework should we use to describe the AsciiDoc syntax?
• How will we define the APIs in a language-independent way?
• How is the relationship between the preprocessor and parser going to be defined? How will we decouple them? (or can we?)
• How are space characters in the source going to be handled? What space characters are permitted?
• Are line and block comments preserved in the parse tree and DOM?
• What are the rules for nesting blocks?
• Propose the nestable open block with the standard delimiter length
• Should we define an inline grammar for the first release, or stick with the substitution-based approach?

Feel free to propose additional candidates.

Editing note from Sarah: I changed some of the paragraphs that posed multiple questions into lists so each question/issue was easier to see.