The DXN data format
DXN — Data eXchange Notation — is one data model written two ways: .dxn, a text form you can read and edit, and .dxnb, a binary form built on CBOR. Both carry the same 31 types, from exact decimals to sets, structs and timestamps, and convert into each other without loss.
A first look
A DXN document is a single value. This one is a map that uses most of the format at once:
@dxn "1.0"
%{
id: @uuid "550e8400-e29b-41d4-a716-446655440000"
name: "Ada Lovelace"
active: true
score: 19.99M
tags: @{:admin :staff}
meta: @ordered %{created: @datetime "1990-01-01T00:00:00Z"}
address: %Point[51.05, 13.74]
result: {:ok, 200}
handle: @regex(i) "^[a-z0-9_]{3,20}$"
}@dxn "1.0"is an optional header naming the format version.%{…}is a map;name:is a keyword key. Commas are optional everywhere.19.99Mis an exact decimal, not a float — the type for money.@{…}is a set,{…}a tuple,@ordered %{…}a map whose order is part of its value.%Point[…]is a struct, written by position; a schema gives its fields names and types.@uuid "…",@datetime "…"and@regex(i) "…"are a UUID, an instant in UTC and a regular expression — values of their own types, not strings. Every such type is a tag in front of its value.
The same document as .dxnb takes 216 bytes instead of 320, and decodes back to exactly the same value.
Design
- One model, two encodings. Every value has a text form and a binary form, and the round trip between them is lossless.
- Exact numbers. Integers have arbitrary precision; decimals and rationals are exact and stay that way in binary. Nothing is normalised:
1.50Mstays1.50M,2/4stays2/4. - Intent in the syntax. A tuple is not a list, an ordered map is not a map, a set is not a list without duplicates — the difference is written down, not agreed on the side.
- One way to write a value. Beyond numbers, strings, characters, names and collections, every type is a tag:
@date "2024-01-01",@uuid "…",@regex "…". Dates and times are ISO 8601 strings under their tag. - Graceful degradation. A struct or tag a reader knows nothing about still parses, into an opaque value it can pass on unchanged — and so do fields a newer writer added to a struct the reader does know.
- Schemas in the same notation. A
.dxnsschema is a DXN document; there is no second language to learn. Schemas carry versions, so data written against an old one can still be read. - Comparable across languages. Every value has one canonical text form and every error one category, so implementations can be checked against each other.
- Familiar to Elixir eyes. Maps, tuples, keywords and
?ccharacters are written the way Elixir writes them.
File facts
| Property | Value |
|---|---|
| Extensions | .dxn text, .dxnb binary, .dxns schema |
| Media types | application/vnd.dxn, application/vnd.dxnb, application/vnd.dxns; any type with a +dxn suffix is text |
| Text encoding | UTF-8 |
| Text header | @dxn "1.0" — optional, before the value; a reader accepts any 1.x |
| Binary | DX, a version byte, then one CBOR item (RFC 8949) — see Binary format |
| Comments | # to the end of the line; text only |
| Grammar | PEG, written in Aether — see Grammar |
A program that receives a body without a type can test cheaply whether it is worth decoding. Binary is recognised by DX and a version byte. Text and schemas are recognised when, after whitespace, comments and an optional header, the first character is % — a map, a struct or a schema document. Lists and scalars are left unrecognised on purpose: [, ", digits and true begin JSON just as well. A yes means the body is worth decoding, never that it decodes.
What changed with dextrin 0.3
The format version is still 1.0, and it stays 1.0 until the libraries that implement it reach 1.0.0. Until then the format changes between releases without a new version number, and 0.3 is such a release. The libraries say so themselves: they are under heavy development and not for production use before 1.0.0.
- Sigils are gone.
~D[2024-01-01]is now@date "2024-01-01",~T[…]is@time "…",~U[… 12:30:00Z]is@datetime "…T12:30:00Z", and~r/^a/iis@regex(i) "^a". A sigil is a syntax error. - Timestamp and datetime are one type,
datetime: an instant with its offset,Zfor UTC. The newlocal-datetimeis a wall-clock time without one. Times, datetimes and durations carry nanoseconds. - Three new types:
@objectid,@ipand@bits. An ObjectId may be a map key, so there are now nine key types — see map keys. - A name may have dotted segments:
acme.billing/invoice,%Foo.Bar/Baz{…}. .dxnbhas new private tags, 216 to 221, and writes every value in its smallest exact form. A 0.2 reader refuses the new tags, so readers are deployed first; a 0.3 reader still reads everything 0.2 wrote. See Binary format.- The schema language, DXNS, has its own reference and many new type forms.
{:one-of …}now means exactly one variant matches —{:any-of …}is the old meaning — and an unknown refine key or a refine argument of the wrong kind no longer compiles. See Schemas. - The specification now names media types and how to identify a body, and lists four error categories for edits to a document — see Conformance.
Implementations
These pages describe the format. The libraries below read and write it; each documents its own API.
| Name | Language | Role |
|---|---|---|
| dextrin | Elixir | The reference implementation, on Hex: text, binary and schemas, and a document type that keeps comments when a file is edited or formatted |
| node-dextrin | JavaScript | The same documents for Node.js, on npm as @joetjen/dextrin |
| php-dextrin | PHP | The same documents for PHP, on Packagist as joetjen/dextrin |
| dextrin for Praxis | Praxis | The port to Praxis, part of the Praxis repository; not released on its own |
| ichor | Elixir | The grammar compiler that generates dextrin’s text parser |
dextrin is at 0.3.3, with a faster parser regenerated by Ichor 0.7.3; node-dextrin and php-dextrin are at 0.3.2, with the same behaviour. The four implementations follow the same specification, and the Dextrin Conformance Suite checks that they agree — see Conformance.
dextrin also brings command-line tasks. For documents: mix dextrin.validate, encode, decode and format, which keeps comments. For schemas: mix dextrin.check checks a tree of .dxns files — syntax, compiling, file layout and names — and reports each problem as file:line; mix dextrin.docs writes HTML documentation for it, every reference a link; and mix dextrin.compat checks schema versions in CI.
On these pages
- Text format — documents, separators, comments, identifiers, tags and discard.
- Values — all 31 types, how to write them, and which may be map keys.
- Binary format — the envelope, the CBOR mapping, private tags and value sharing.
- Schemas —
.dxnsdocuments: type forms, constraints, struct schemas, shapes, extra fields and versions. - Conformance — the canonical form, error categories and the suite that holds the implementations to them.
- Grammar — the formal grammar and where the reference implementation still differs from it.