joetjen.net
EN DE
Data format

Values

DXN has 31 data types in four groups: scalars, collections, temporal values and extended types. Each one has exactly one text spelling and one binary encoding.

Format 1.0

At a glance

TypeText
nil, booleannil true false
integer, float-17 3.14 5e-3 NaN -Infinity
decimal, rational19.99M 22/7
string, char"text\n" ?a ?s
symbol, keywordPoint ns/name :ok name:
list, tuple, array[1 2] {:ok 200} @array[1 2]
map, ordered map%{a: 1} %{"k" => 1} @ordered %{…}
set, sorted set@{1 2} @sorted-set @{3 1 2}
struct%Point{x: 1 y: 2} %Point[1 2] %Point(2){x: 1 y: 2}
date, time@date "2024-01-01" @time "12:30:00.5"
datetime, local datetime@datetime "2024-01-01T12:30:00Z" @local-datetime "2024-01-01T12:30:00"
duration@duration "P1Y2M10D"
uuid, objectid, ip@uuid "…" @objectid "507f1f77bcf86cd799439011" @ip "192.0.2.1"
uri, bytes, bits, regex@uri "…" @bytes "SGVsbG8=" @bits {"oA==" 3} @regex(i) "^a+$"
custom tag@my-app/money 500

Numbers

Four kinds, each exact about what it is:

  • Integer — an optional - and digits, with arbitrary precision. There is no + sign, no hex and no digit separator.
  • Float — an IEEE 754 double, with a fraction, an exponent or both. NaN, Infinity and -Infinity are literals; NaN has no sign.
  • Decimal — digits with a trailing M, exact fixed-point. 19.99M stays 19.99, in text and in binary, and 1.50M keeps its second decimal place.
  • Rational — numerator/denominator, stored as written: 22/7 and 44/14 are different values. A zero denominator is an error.
[42 -17 12345678901234567890123 3.14 1e3 NaN -Infinity 19.99M -1/3]

Strings and characters

Strings are UTF-8 in double quotes, and may span lines. The escapes are \", \\, \n, \t, \r, \0, \a, \b, \f, \v, and \x{…} with any number of hex digits for a code point. There is no single-quoted form and no interpolation.

A character is ? and one code point — a value of its own, not a one-letter string. ?s is the space, and escapes work after ? too.

["tab\there, smile \x{1F600}" ?a ?s ?\n ?é]

Symbols and keywords

A bare identifier is a symbol: a name that refers to something — a type, a handler, a schema — and is never evaluated. A keyword is a name used as a value, like a status or a mode: :ok, or :"with spaces" when it is not a plain identifier. In a map, name: before a value is a keyword key.

%{
  handler: billing/invoice-created   # a symbol
  status:  :ok                       # a keyword
  label:   :"needs review"
}

:nil, :true and :false are keywords like any other, not nil and the booleans. Their names are reserved, so they are written back quoted:

[:nil :true :false]
[:"nil" :"true" :"false"]

Collections

FormTypeMeaning
[…]listan ordered sequence that may grow
{…}tuplea fixed group — a pair, a result, a record
@array[…]arraya fixed-size, indexed sequence
%{…}mapkeys to values, in no particular order
@ordered %{…}ordered mapa map whose order is part of its identity
@{…}setdistinct values, unordered
@sorted-set @{…}sorted seta set kept in sorted order

A map entry is either name: value, for keyword keys, or key => value, for the other key types. The two styles mix freely. A plain map promises no order even though its entries are written one after another; if order matters, say so with @ordered.

%{
  name: "api"
  "Content-Type" => "application/json"
  404 => :not-found
  @date "2024-12-24" => "closed"
}
@sorted-set @{3 1 2}
@sorted-set @{1 2 3}

Map keys

A map key must be a string, keyword, integer, uuid, objectid, date, bytes, symbol or char. Any other key is an error, key/not-allowed, whether it is read from text or binary or about to be written. Two keys are the same key exactly when their canonical forms are equal: same type, same value. So 1 and "1" are different keys, and so are :x and "x". A key that occurs twice keeps its last value.

%{a: 1, a: 2}
%{a:2}
%{1.5 => :x}
key/not-allowed: 1.5 cannot be a map key: a key is a string, keyword, integer, uuid, objectid, date, bytes, symbol or char

The nine are the types whose identity is exact and obvious in every language. Floats, decimals and rationals are out because 1 and 1.0, 0.0 and -0.0, 1.0M and 1.00M, 2/4 and 1/2 are distinct DXN values that some languages hold as one. Times, datetimes, local date-times, durations and URIs are out for precision, offsets, P1D against PT24H, and URI normalisation. nil and booleans collide with other keys in some languages — in PHP, nil with "" and true with 1. IP addresses, bits, regexes, custom tags and collections are out too.

Structs

A struct is a named record, written keyed — field names inline, in any order — or positional, with the values in the order the schema defines. Given its schema, both forms read as the same value. A number in parentheses after the name is the version of the schema the struct was written with; see schema versions.

%Point{y: 2, x: 1}
%Point[1, 2]
%Point(2){x: 1, y: 2}

A struct may carry more fields than its schema declares. In the keyed form they are just more entries. In the positional form they follow the items as entries: an item is a value not followed by =>, all items come first, and an item after an entry is a syntax error. In a struct, name:, :name => and "name" => all name the same field; an entry keyed by anything else, like 3 => :x, is always an extra field.

%User(3)["a" "a@x" nickname: "b", 3 => :x]
%User(3)["a","a@x",nickname:"b",3=>:x]

A struct parses without a schema: a reader that does not know Point returns an opaque value carrying the name, the version and the fields as written. With a schema, it can check the fields, convert between the two forms and give the fields their types.

Dates and times

FormTypeMeaning
@date "2024-01-01"datea calendar date
@time "12:30:00.5"timea time of day, up to nanoseconds
@datetime "2024-01-01T12:30:00Z"datetimean instant with its offset: Z for UTC, or ±HH:MM
@local-datetime "2024-01-01T12:30:00"local-datetimea wall-clock date and time, no offset — not an instant
@duration "P1Y2M10DT2H30M"durationan ISO 8601 duration

Every date and time is an ISO 8601 string under its tag, and a string that is not the form its tag names is a syntax error. A time takes up to nine fraction digits. A datetime keeps the offset it was written with; Z is +00:00, and -00:00, “offset unknown”, is refused — a value without an offset is a local-datetime.

@datetime "2024-01-01T12:30:00+00:00"
@datetime "2024-01-01T12:30:00Z"

A duration keeps years, months, weeks, days, hours, minutes and seconds as separate fields, because a month or a year has no fixed length and cannot be folded into one number of seconds.

Identifiers, addresses, bytes and regexes

  • @uuid "…" takes 8-4-4-4-12 hex digits, in either case; it is written in lowercase.
  • @objectid "…" is BSON’s 12-byte ObjectId, written as 24 hex digits.
  • @ip "…" is an IPv4 or IPv6 address, or a prefix with its length. A prefix whose host bits are not zero, 10.0.0.1/8, is a syntax error.
  • @uri "…" is an RFC 3986 URI, kept exactly as written.
  • @bytes "…" is raw binary data, base64 in text and plain bytes in .dxnb.
  • @bits {"…" n} is a run of n bits that is not whole bytes: base64 data and the count, the bits running from the high bit of the first byte.
  • @regex "pattern" is a regular expression, PCRE syntax, with the flags i m s u x f r — caseless, multiline, dotall, unicode, extended, firstline, ungreedy — in a header, @regex(im) "…".
%{
  id:     @uuid "9b74c989-1e2a-4b17-9b1f-2a6a4a2b6f10"
  doc:    @objectid "507f1f77bcf86cd799439011"
  host:   @ip "192.0.2.1"
  net:    @ip "2001:db8::/32"
  home:   @uri "https://example.com/path"
  avatar: @bytes "SGVsbG8="
  flags:  @bits {"oA==" 3}
  handle: @regex(i) "^[a-z0-9_]{3,20}$"
}

Bits and bytes are one value space: a run of bits whose count is a multiple of eight is bytes, and is written so.

@bits {"AQID" 24}
@bytes "AQID"

A regex pattern is an ordinary string, so a backslash in it is doubled. PCRE’s inline modifiers belong to the pattern and need no header: the two regexes below match the same text and are different values, since their patterns differ. A pattern PCRE does not compile is a syntax error. node-dextrin cannot tell every pattern apart, since JavaScript’s engine is not PCRE, and reads any pattern.

[@regex(im) "^\\d+$" @regex "(?im)^\\d+$"]
@regex "a("
syntax: invalid regex "a(": {~c"missing closing parenthesis", 2}

Custom tags

Any tag name that is not built in is a custom tag: one value wrapped in an application-defined name. A reader with a decoder for the name turns it into its own type; a reader without one keeps it as an opaque tagged value and passes it on. Namespace the name to keep it from colliding with someone else’s.

@my-app/money {1999 :eur}

Custom tags suit a single wrapped value. Anything with real fields is better written as a struct, which a schema can check.