joetjen.net
EN DE
Grammar language

The Aether grammar language

Aether describes a language completely in one file: its tokens, its syntax, how whitespace is treated and which parts of a match are kept. Upper-case names are tokens, matched by a lexer; lower-case names are rules, matched by a parser over those tokens.

Ichor 0.7

A first look

This grammar reads a small settings file — lines of name = value, with comments:

settings.aether
@grammar "settings"
@root document
@skip TRIVIA

COMMENT := "#" (!"\n" .)*
TRIVIA  := (SPACE | COMMENT)*

TRUE   := "true"
FALSE  := "false"
NAME   := [a-z_] [a-z0-9_]*
NUMBER := DIGIT+
STRING := "\"" (!"\"" .)* "\""

document := entry*
entry    := key:NAME "=" value:value
value    := TRUE | FALSE | NUMBER | STRING
  • @grammar names the grammar and @root the rule matching starts from. Both are required, in that order.
  • @skip TRIVIA lets whitespace and comments stand between any two parts of a rule, and before and after the whole input.
  • COMMENT to STRING are tokens. TRUE comes before NAME on purpose: both match true, and the earlier one wins.
  • document, entry and value are rules. The quoted "=" becomes a token of its own without being declared.
  • key: and value: name the parts of an entry that matter to whatever processes the match.
# service
port = 8080
debug = true
name = "api"
ok
true = 1
1:1: unexpected "true" -- expected NAME
1 | true = 1
  | ^

Design

  • Lexer and parser in one file. Spelling decides which is which, so there is no separate token file to keep in step.
  • Two stages, always. Tokens take the longest match, rules take the first alternative that fits — each stage keeps its own simple rule.
  • Whitespace is a setting, not a chore. One pragma makes every rule tolerate it; one prefix forbids it where two parts must touch.
  • Syntax only. A grammar says what is valid and names the parts; what they mean is decided elsewhere.
  • Escape hatches, clearly marked. Where a language changes its own syntax mid-file, @native hands one token or rule to hand-written code.

File facts

PropertyValue
Extension.aether
EncodingUTF-8; names are ASCII, strings and classes may hold any character
Header@grammar "name", then @root rule — mandatory, in that order
Comments; to the end of the line
ResultA lexer and a parser — PEG by default, LR or GLR on request
ImplementationsIchor in Elixir, the reference, and its port to Praxis — see below

Implementations

ImplementationLanguageVersionReads .aether with
Ichor, the referenceElixir0.7.3 on hex.pm, with ichor_runtime 0.6.3Aether.Lexer, Aether.Reader
the Praxis port, libs_prx/ichor in the Praxis repositoryPraxis0.7.0, not released on its own: Ichor 0.6.0 plus one step, the move to Praxis; it moves with each Elixir releaseichor.aether.lexer, ichor.aether.reader

Ichor’s README opens with a warning: it is under heavy development and not for production use before 1.0.0. Until then the library and the language it reads can change between any two releases, which is why Aether is marked as in development here.

The port is a reimplementation, not a fork: the same three engines, the same five importers and the same VM, written in Praxis. Both readers are written by hand, and each is held by its own test to the same Aether-in-Aether grammar. The Ichor Conformance Suite, kept in the Praxis workspace for now, gives every implementation the same grammars and inputs and compares the trees and errors that come out, interpreted and generated alike.

Written in Aether

GrammarDefinesUsed by
casc.aetherthe CASC configuration format — grammarcooper and its Praxis port; node-cooper and php-cooper follow it by hand
dxn.aetherthe DXN text format — grammardextrin and its Praxis port
praxis.aetherthe surface of the Praxis language, the reference for its readerPraxis
logos.aetherthe reader of Logos, a Clojure-flavoured Lisplogos
grammar.aetherProlog clauses, with op/3 directives that change the operator table mid-filealetheia
quire.aetherwell-formed markup with {{ … }} holesquire, a Praxis library
xeger.aetherregular expressionsxeger
abnf.aether …ABNF, BNF, ISO and W3C EBNF, and PEGthe importers of both Ichor implementations

On these pages

  • File structure — header, pragmas, comments, names, definitions and predefined tokens.
  • Expressions — operators, strings, character classes, repetition, lookahead and regex literals.
  • Lexing and parsing — longest match, ordered choice, whitespace, captures, layout, engines and error reports.
  • Escape hatches — @keywords, @refine and @native.
  • Grammar — Aether written in Aether, the grammar both implementations test their readers against.