joetjen.net
EN DE
Data format

The binary format

.dxnb is DXN encoded as CBOR (RFC 8949): a three-byte envelope, then one self-describing CBOR item. Types CBOR already has use its major types or registered tags; the rest use a small block of private tags. Every value takes the smallest representation that keeps it exactly.

Format 1.0

The envelope

Every .dxnb file starts with the bytes 44 58 — the letters DX — and a version byte, currently 01. The envelope is mandatory, unlike the text header: it costs three bytes and lets tools recognise the file by its first bytes. No end marker follows; a CBOR item knows where it ends.

[1 2 3]
44 58 01        "DX", version 1
83 01 02 03     array(3): 1, 2, 3

Comments and discarded values have no binary form: they are gone before there is a value to encode.

Type mapping

TypeCBORPayload
nil, booleansimple values 22, 20, 21
integermajor 0/1; tag 2/3 beyond 64 bitsbignum bytes
floatmajor 7the shortest of half, single and double precision that holds the value exactly; NaN and the infinities in half
decimaltag 4[exponent, mantissa]
rationaltag 30[numerator, denominator]
stringmajor 3UTF-8
bytesmajor 2raw bytes, no base64
listmajor 4items
mapmajor 5pairs
uritag 32text
uuidtag 3716 raw bytes
iptag 52 (IPv4), tag 54 (IPv6), RFC 9164an address: its raw bytes; a prefix: [length, bytes], trailing zero bytes left off
everything elseprivate tags 200–221see below

Smallest means: raw bytes where a type is bytes at heart — a UUID, an ObjectId, an IP address — integers where it is a count — dates, times, durations — and CBOR’s shortest integer and length forms everywhere, never the text form a value has in .dxn. Integers past 64 bits use the bignum tags, never a float, and times and datetimes are integer nanoseconds, never a float, which would lose them.

Private tags

TagTypeWraps
200charthe code point, as an integer
201symboltext
202keywordtext
203tuplearray
204arrayarray
205ordered mapmap
206setarray
207sorted setarray, in canonical order
208struct[name, field, …, extras?]
209datedays since 1970-01-01
210–212time, datetime, duration in microsecondsDXN 0.2’s forms: read, never written
213regexthe pattern; with flags, [pattern, flags]
214custom tag[name, value]
215extra fieldsa map — only ever a struct’s last item
216bits[bytes, bit count]
217local-datetimenanoseconds since 1970-01-01T00:00:00 on the wall clock
218timenanoseconds since midnight
219datetimeepoch nanoseconds in UTC; with another offset, [epoch nanoseconds, offset minutes]
220durationa field bitmask, then the fields present
221objectid12 raw bytes

The block sits in CBOR’s one-extra-byte range, clear of the busy low numbers. It is not registered with IANA, so it is collision-free only among DXN documents. A tag never changes its meaning: when times went from microseconds to nanoseconds, the new forms took new tags, and a reader still reads the old ones — DXN 0.2’s tags 210 to 212, and its UTC timestamps under CBOR tag 1 in microseconds — as what they meant.

Structs are always positional in binary — [name, field-1, field-2, …] — because field names only help human readers. The keyed text form exists for them alone. The name carries the schema version, "Point(2)", when the struct has one. Extra fields follow the declared ones as a single map under tag 215, keyed by plain strings where the key is a name; tag 215 appears nowhere else.

Bit layouts

A duration’s bitmask marks which fields follow, lowest bit first; each present field follows as a signed integer, in bit order. A regex carries its flags as one byte in the same way. Bit 7 is reserved in both and must be 0.

BitDuration fieldRegex flag
0yearsi
1monthsm
2weekss
3daysu
4hoursx
5minutesf
6nanoseconds (seconds included)r

Byte by byte

Each dump below is what dextrin writes for the text above it.

%{x: 1}
44 58 01        envelope
A1              map(1)
D8 CA 61 78     tag 202 (keyword) "x"
01              1
19.99M
44 58 01        envelope
C4 82           tag 4 (decimal), array(2)
21              exponent -2
19 07 CF        mantissa 1999
[1.5 0.1 NaN]
44 58 01                        envelope
83                              array(3)
F9 3E 00                        1.5, half precision
FB 3F B9 99 99 99 99 99 9A      0.1, double precision
F9 7E 00                        NaN, half precision
%Point(2)[1 2 label: "a"]
44 58 01                     envelope
D8 D0 84                     tag 208 (struct), array(4)
68 50 6F 69 6E 74 28 32 29   "Point(2)"
01 02                        1, 2
D8 D7 A1                     tag 215 (extra fields), map(1)
65 6C 61 62 65 6C            "label"
61 61                        "a"
With %{Point: %schema{x: :integer, y: :integer}} loaded
%Point{y: 2, x: 1}
44 58 01                     envelope
D8 D0 83                     tag 208 (struct), array(3)
68 50 6F 69 6E 74 28 31 29   "Point(1)" — the latest version loaded
01 02                        x = 1, y = 2, in schema order
@datetime "2024-01-01T12:30:00Z"
44 58 01                        envelope
D8 DB                           tag 219 (datetime)
1B 17 A6 39 04 62 68 D0 00      1704112200000000000 ns, UTC
@datetime "2024-01-01T14:30:00+02:00"
44 58 01                        envelope
D8 DB 82                        tag 219 (datetime), array(2)
1B 17 A6 39 04 62 68 D0 00      the same instant in ns
18 78                           offset 120 minutes
@duration "P1Y2M10D"
44 58 01        envelope
D8 DC 84        tag 220 (duration), array(4)
0B              bitmask 0000 1011: years, months, days
01 02 0A        1, 2, 10
@regex(i) "^a+$"
44 58 01        envelope
D8 D5 82        tag 213 (regex), array(2)
64 5E 61 2B 24  "^a+$"
01              flags 0000 0001: i
@regex "^a+$"
44 58 01        envelope
D8 D5           tag 213 (regex), no flags
64 5E 61 2B 24  "^a+$"
@objectid "507f1f77bcf86cd799439011"
44 58 01                                    envelope
D8 DD                                       tag 221 (objectid)
4C 50 7F 1F 77 BC F8 6C D7 99 43 90 11      bytes(12)
@ip "10.0.0.0/8"
44 58 01        envelope
D8 34 82        tag 52 (IPv4), array(2)
08              prefix length 8
41 0A           bytes(1): 10, zero bytes left off
@bits {"oA==" 3}
44 58 01        envelope
D8 D8 82        tag 216 (bits), array(2)
41 A0           bytes(1): 1010 0000
03              3 bits: 101

Value sharing

Two optional CBOR extensions shrink documents that repeat themselves. Encoders may use them; decoders must accept them.

  • String references — tag 256 opens a scope, tag 25 refers to the n-th string already seen in it. Meant for repeated symbols, keywords and struct names.
  • Shared values — tag 28 marks an item as shareable, tag 29 refers to a marked item by index. Works for any value: a map, a list, a struct.

Sharing only saves bytes; it never creates identity. A decoder hands back an independent copy for every reference, so the result is equal to a document that wrote everything out. A reference to an item not yet seen, or one that would form a cycle, makes the document invalid: .dxnb values are always trees.

One address used three times — 129 bytes, 75 with sharing
%{
  "hq"       => %{"street" => "1 Main St", "city" => "Springfield"}
  "billing"  => %{"street" => "1 Main St", "city" => "Springfield"}
  "shipping" => %{"street" => "1 Main St", "city" => "Springfield"}
}
44 58 01                    envelope
A3                          map(3)
67 "billing"
D8 1C A2 …                  tag 28: the address map, marked (its strings too)
62 "hq"
D8 1D 04                    tag 29: marked item 4 — the map
68 "shipping"
D8 1D 04                    tag 29: marked item 4