Under heavy development: not for production use before 1.0.0.
Things can and will break, and the specification can and will change,
between any two releases until 1.0.0. Do not use this library in a
production system yet.
A Node.js port of dextrin, an
Elixir implementation of DXN (Data eXchange Notation): a
human-writable text format (.dxn), a compact binary format
(.dxnb) built on CBOR, and a schema format (.dxns) — all three
sharing one in-memory value representation and one extension
mechanism. A sibling of php-dextrin,
the PHP port of the same library.
Status: feature-complete port, published to npm. The value model,
the .dxn text codec, the .dxnb binary codec, the .dxns schema
system, the dextrin CLI, and the guides are all implemented. See
CHANGELOG.md for exactly what's landed.
Most serialization formats pick one point on a spectrum. JSON is human-writable but has no dates, no bytes, no exact decimals and no boundary between an integer and a float. Protobuf and Avro are compact and typed but need a schema compiled ahead of time, and aren't meant for a person to read or edit. CBOR and MessagePack are compact and self-describing but binary only. YAML and TOML are made for people, and have no binary form for machines.
DXN's premise is that a .dxn text document and a .dxnb binary
document are the same value space: 31 scalar, collection, temporal and
extended types (reference §1.3). It is precise
enough for money (decimals), exact ratios (rationals) and integers of
any size, has struct and custom-tag as first-class, schema-describable
extension points, and its schemas (.dxns) are DXN too. You choose text
or binary by what matters where the data is going -- a person reading
it, or bytes on the wire -- and the value is the same either way.
| DXN | JSON | YAML | TOML | BSON | MessagePack | CBOR | Protobuf | Ion | |
|---|---|---|---|---|---|---|---|---|---|
| Text a person writes | ✓ | ✓ | ✓ | ✓ | – | – | –¹ | –² | ✓ |
| Binary, same values | ✓ | – | – | – | ✓ | ✓ | ✓ | ✓ | ✓ |
| Comments | ✓ | – | ✓ | ✓ | – | – | – | – | ✓ |
| Schema, part of the format | ✓ | ~³ | – | – | – | – | – | –² | ~³ |
| Readable without a schema | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – | ✓ |
| Integer and float kept apart | ✓ | – | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Integers beyond 64 bits | ✓ | ~⁴ | ~⁴ | – | – | – | ✓ | – | ✓ |
| Exact decimals | ✓ | – | – | – | ✓ | – | ✓ | – | ✓ |
| Exact ratios | ✓ | – | – | – | – | – | ✓ | – | – |
| Timestamps, dates, times, durations | ✓ | – | ~⁵ | ~⁶ | ~⁷ | ~⁷ | ~¹³ | ~⁸ | ~¹⁴ |
| Bytes | ✓ | – | ~⁵ | – | ✓ | ✓ | ✓ | ✓ | ✓ |
| UUIDs, URIs | ✓ | – | – | – | ~⁹ | – | ✓ | – | – |
| Sets, tuples | ✓ | – | ~⁵ | – | – | – | ~¹⁰ | – | – |
| Map keys that are not strings | ✓ | – | ✓ | – | – | ✓ | ✓ | ~¹¹ | – |
| Typed records (structs) | ✓ | – | – | – | – | – | – | ✓ | – |
| Types an application adds | ✓ | – | ✓ | – | – | ✓ | ✓ | – | ✓ |
| Versioned schemas, checked in CI | ✓ | – | – | – | – | – | – | ~¹² | – |
¹ CBOR has a diagnostic notation for reading, not for authoring.
² Protobuf's text format is for debugging; its schema is a separate
.proto file compiled ahead of time.
³ JSON Schema and Ion Schema are written in JSON and Ion, as separate
specifications.
⁴ The grammar allows any size; JavaScript's and PHP's parsers lose the
digits past 53 and 64 bits.
⁵ YAML 1.1 tags (!!timestamp, !!binary, !!set), dropped from 1.2's
core schema.
⁶ Dates, times and date-times; no durations.
⁷ An instant only: no local dates, times or durations.
⁸ The well-known Timestamp and Duration messages.
⁹ A UUID as binary subtype 4.
¹⁰ Sets by registered tag 258; no tuples.
¹¹ Integer and string keys only.
¹² Field numbers keep old readers working; nothing checks a change.
¹³ Instants (tags 0 and 1) and dates (RFC 8943); no times of day or
durations.
¹⁴ Timestamps of any precision, dates included; no times of day or
durations.
The same four payloads in each format, in bytes, raw and gzipped.
Measured with dextrin's
bench/format_sizes.exs (mix run bench/format_sizes.exs); the script says how
each format writes the values it has no type for (a decimal or a date
as a string in JSON, for instance).
| config file | API response, 100 records | time series, 1,000 points | record with a 4 KiB blob | |
|---|---|---|---|---|
| DXN text | 279 / 199 | 19,402 / 4,218 | 63,159 / 13,895 | 5,516 / 4,222 |
| DXN text, pretty | 314 / 216 | 23,504 / 4,249 | 67,161 / 13,961 | 5,527 / 4,234 |
.dxnb |
264 / 210 | 14,185 / 3,802 | 34,006 / 14,381 | 4,145 / 4,168 |
.dxnb, shared |
264 / 210 | 9,217 / 3,752 | 27,017 / 14,019 | 4,145 / 4,168 |
| JSON | 306 / 207 | 19,202 / 4,201 | 56,159 / 13,692 | 5,514 / 4,216 |
| JSON, pretty | 403 / 229 | 26,253 / 4,287 | 74,160 / 14,182 | 5,527 / 4,231 |
| BSON | 312 / 231 | 17,621 / 4,448 | 36,903 / 15,700 | 4,161 / 4,184 |
| MessagePack | 237 / 199 | 12,271 / 3,840 | 25,003 / 11,165 | 4,136 / 4,159 |
| CBOR | 237 / 188 | 12,282 / 3,514 | 25,003 / 11,164 | 4,136 / 4,159 |
| Protocol Buffers | 141 / 111 | 8,552 / 3,702 | 19,000 / 10,832 | 4,122 / 4,145 |
@datetime "...") where JSON writes a bare string..dxnb sits between JSON and the compact binaries. Value sharing
writes a repeated key or string once; on the API response that brings
it within 8% of Protocol Buffers, with no schema needed to read it.
This package reads shared .dxnb; the Elixir and Praxis encoders
write it..dxnb datetime keeps
nanoseconds, its offset and its type, where CBOR's and MessagePack's
carry whole seconds; a keyword key is tagged as a keyword rather than written as a
plain string. Protocol Buffers is smallest because its schema, not the
data, holds the field names..dxnb, and JSON within about a quarter of CBOR; past that,
the choice is about what each format can say.Decoding a document from text and checking it against a schema, timed in
each implementation on the same data sets: one Universe each, about
4 KB of DXN text, against bench/schemas/universe.dxns, a 225-line
schema set more complex than any one met in practice (recursion,
generics, cases, refinements, formats, documents inside documents).
About 30% of the data sets carry exactly one error, of 44 kinds. Each
implementation ran for 90 minutes on one core of an Apple M2 Max
(Elixir 1.19.5 on OTP 28 with ichor 0.7.3's parser, 2026-10-26; Node
24.18.1 and PHP 8.4.24 without opcache, 2026-10-22), and every verdict matched what the data set was meant to be.
Times are per data set, in milliseconds.
| data sets | decode, mean | decode, p99 | check, mean | check, p99 | together, mean | together, p99 | |
|---|---|---|---|---|---|---|---|
| Elixir | 713,050 | 3.45 | 7.74 | 0.16 | 0.93 | 3.60 | 7.97 |
| Node | 4,698,138 | 0.53 | 0.85 | 0.54 | 1.02 | 1.07 | 1.80 |
| PHP | 1,316,837 | 1.95 | 2.89 | 0.83 | 1.57 | 2.78 | 4.37 |
Reproduce it with bench/schema-validation.cjs, which reads its data sets from
dextrin's bench/generate_universes.exs
(the script says how); the Elixir numbers come from that repository's
bench/schema_validation.exs, on the very same data sets.
What a value is after a round trip through each language's own JSON parser, and through Dextrin:
| Written | Elixir: JSON / Dextrin | Node.js: JSON / Dextrin | PHP: JSON / Dextrin | Praxis: JSON / Dextrin |
|---|---|---|---|---|
9223372036854775808 |
integer / integer | 9223372036854776000 / BigInt |
float / DXNInteger |
integer / integer |
19.99 as an exact decimal |
float / Decimal |
float / DXNDecimal |
float / DXNDecimal |
float / dextrin.decimal |
1.0 |
float / float | 1, an integer / float | float, written back as 1 / float |
float / float |
| a datetime | string / DateTime |
string / DXNDateTime |
string / DXNDateTime |
string / a praxis.time instant |
| bytes | base64 string / Dextrin.Bytes |
base64 string / DXNBytes |
base64 string / DXNBytes |
base64 string / dextrin.bytes |
| a map keyed by integers | string keys / integer keys | string keys / integer keys | string keys / integer keys | string keys / integer keys |
| a set | list / MapSet |
array / Set |
array / DXNSet |
vector / set |
A bold entry is a value that did not come back as what was written. With Dextrin, the same document means the same thing in all four, and every implementation is held to that by the Dextrin Conformance Suite.
A .dxns schema is DXN data. It is meant to say everything the schema
languages of the formats above can say, except what has no value or no
meaning in DXN (design/schema-superset.md
goes through them construct by construct). YAML, TOML and MessagePack
have no schema language of their own; people use JSON Schema for them.
| DXNS | JSON Schema | MongoDB $jsonSchema |
CDDL | Protobuf | Ion Schema | |
|---|---|---|---|---|---|---|
| Written in the format it describes | ✓ | ✓ | ✓ | – | – | ✓ |
| Typed records, with versions | ✓ | – | – | – | ~¹ | – |
| Version changes checked in CI | ✓ | – | – | – | ~¹ | – |
| Recursive types | ✓ | ✓ | – | ✓ | ✓ | ✓ |
| Generic types | ✓ | ~² | – | ✓ | – | – |
| A list's items in order | ✓ | ✓ | ✓ | ✓ | – | ✓ |
| A type chosen by a sibling field | ✓ | ~³ | – | ~⁴ | ~⁵ | – |
| String formats, checked the same everywhere | ✓ | ~⁶ | – | ~⁷ | ~⁸ | ~⁷ |
| Decimals, ratios, dates, times, durations as types | ✓ | – | ~⁹ | ~¹⁰ | – | ~¹¹ |
| Bytes holding an encoded document | ✓ | ✓ | – | ✓ | – | – |
| Annotations and free metadata | ✓ | ✓ | ~¹² | – | ✓ | ✓ |
| Closing a type to unknown fields | –¹³ | ✓ | ✓ | ✓ | – | ✓ |
| One test suite every implementation passes | ✓ | ~⁶ | – | – | – | – |
¹ Field numbers keep old readers working; checking a change takes a
tool outside the language, such as buf breaking.
² $dynamicRef, since draft 2020-12.
³ if/then/else; OpenAPI adds a discriminator on top.
⁴ A group choice, //, over maps with a literal member.
⁵ oneof picks one field of several, not the type of one.
⁶ The official test suite exists, but format only annotates unless a
validator opts in, and each implements the formats its own way.
⁷ Regular expressions (CDDL's .regexp, Ion Schema's regex); CDDL's
.abnf takes a grammar.
⁸ Through the separate protovalidate rules.
⁹ decimal and date BSON types; no times of day or durations.
¹⁰ Through tags and the prelude (tdate, decfrac); no durations.
¹¹ decimal and timestamp, with precision and offsets; no durations.
¹² title and description only.
¹³ On purpose: see the next section.
JSON Schema's additionalProperties: false, Ion Schema's closed and
CDDL's maps refuse a field the schema does not list. DXNS has no such
switch, for two reasons:
Order
schema serves both, so whether unknown fields are refused belongs to
the decode: decode(text, { registry, strict: true }) (or
decodeBinary, or a registry marked with registry.putStrict(true))
refuses them (schema/undeclared-field), and without strict they
are kept -- in .dxnb as one trailing item (DXNS.md §7) -- and
written back out.A check that needs more -- say, refusing exactly the fields another schema allows -- runs after the decode, on the value it returns, where the application knows what it means.
Several services reading and writing the same schema never all deploy at once. DXNS makes that safe with versions, not with compatibility modes (DXNS.md §8):
%schema(3){...}, and every version
stays loadable beside the others. A writer always writes the latest it
has loaded, and says which: %Order(3){...}..dxnb, and keeps the rest
as extra fields. This is why a field is only ever appended: in
.dxnb, an older reader takes positions as they were.registry.putStructMaterializer(name, (name, version, fields) => ...)).
Here it is called for every decoded struct of that name, whatever its
version, with the version the fields were read with, so it builds the
latest value from any of them. Without one, reading an older version is
an error, not a guess.npx dextrin compat OLD NEW (or
Schema.compat(oldRegistry, newRegistry)) compares the last release's
schemas with the current ones, each a .dxns file or a directory of
them. A version whose content changed is an error -- two documents
claiming to be Order v3 must mean the same thing -- as is a version
that went down, and it lists, for each schema, what changed since each
older version: the materializer's checklist.So a change deploys like this:
OrderV2) beside the old
one, and the services move over one at a time.DXN's premise (see the Elixir project's own
README for the full
case) is that a .dxn text document and a .dxnb binary document
are the same value space — scalar, collection, temporal, and
extended types precise enough for money, exact ratios, and
arbitrary-precision integers, with a schema-describable extension
story. DXN.md (ported here once the text/binary codecs land) is the
implementation-independent, normative spec; this library only needs
to satisfy it.
Where this library differs from the Elixir original, it's by design, documented at the point of difference:
null, boolean, string, number for float, a
native Map/Set/Array, ...) and to a small DXN* wrapper class
only where nothing native fits without losing information —
mirroring the same design principle dextrin's own Dextrin.Value
follows, not necessarily the same type-by-type choice (see
src/values/ for each class's own rationale).integer is always BigInt, never number — JS number can't
hold arbitrary precision the way Elixir's native integer does, so
this is the one place fidelity requires a deviation from "closest
native type."float needs no wrapper at all, unlike Elixir (which resorts to
atoms for NaN/Infinity/-Infinity because the BEAM can't
construct a non-finite float term) — JS number natively holds all
of IEEE-754, so this is a case where the native JS type is a
better fit than Elixir's own.toX() / DXN*.fromX()) rather
than forcing you through the wrapper alone — e.g. DXNRational. fromNumber() (exact — every finite JS number is a binary fraction),
DXNOrderedMap.toMap()/.fromMap(), DXNRegex.toRegExp() (guarded
— only when no DXN-only flag is set).@regex "a(" is syntax, DXN.md
§1.2), and the Elixir, PHP and Praxis implementations check it with
PCRE. JavaScript has no PCRE, and its own engine refuses valid PCRE
-- inline modifiers such as (?im), atomic groups, \A -- so it
cannot stand in: node-dextrin reads any pattern, and DXNRegex. toRegExp() is where a pattern JavaScript cannot run fails.Every DXN type maps to either a native JS value or a small DXN*
wrapper class under src/values/. Full mapping table, including the
rationale for each wrapper-vs-native choice, lives in this project's
development plan for now and will move here as the port progresses.
const { DXNRational, DXNUUID } = require('@joetjen/dextrin');
const r = new DXNRational(22n, 7n);
r.toNumber(); // => 3.142857142857143
DXNRational.fromNumber(0.5); // => DXNRational { numerator: 1n, denominator: 2n }
const id = DXNUUID.fromString('550e8400-e29b-41d4-a716-446655440000');
id.toString(); // => '550e8400-e29b-41d4-a716-446655440000'
.dxn textconst { decode, encode } = require('@joetjen/dextrin');
const value = decode('%{x: 1, y: 2}', { trusted: true });
// => DXNMap(2) { Symbol(x) => 1n, Symbol(y) => 2n } -- a Map whose keys compare by value
encode(value);
// => '%{x:1,y:2}'
encode(value, { pretty: true });
// => '%{\n x: 1\n y: 2\n}'
Decoding is safe by default: keyword (:name) decodes as
DXNKeyword, the same way Dextrin.Keyword exists on the Elixir side,
so no input can fill the global symbol registry, which is never garbage
collected. Pass trusted: true (or registry.putTrusted(true)) for a
source you control, as above, and it decodes as a real, global
Symbol.for(name) — the closest native analog to a BEAM atom. decode/encode throw DXNError on failure —
see src/error.cjs for why that's a deliberate divergence from
Dextrin's own {:ok, _} | {:error, _} tuples (an API-shape choice,
not a value-representation one).
With no registry: option, every struct decodes as an opaque
DXNStruct, and every tag name other than the built-ins (@uuid,
@duration, @ordered, ...) decodes as DXNCustomTag — see
.dxns schemas below for schema-backed decoding.
.dxnb binaryconst { decode, decodeBinary, encodeBinary } = require('@joetjen/dextrin');
const value = decode('%{x: 1, y: 2}');
const bin = encodeBinary(value); // => Buffer, "DX" + version + CBOR
decodeBinary(bin); // => the same value back
Hand-rolled CBOR over Buffer (not built on a generic CBOR library —
bignum precision, datetime integer-only encoding, and the private
tag block aren't things a generic library gets right by default; see
src/binary/encoder.cjs's own doc). decodeBinary accepts documents
using the spec's value-sharing extensions (DXN.md §2.4/§2.5) even
though encodeBinary doesn't produce them yet — accepting is
spec-mandatory, producing is optional, and that asymmetry is
deliberate for now.
.dxns schemasconst { decode, encode, Schema, Registry } = require('@joetjen/dextrin');
const schemaDoc = decode(`
%{
Point: %schema{ x: :float, y: :float }
}
`);
const registry = Schema.compile(schemaDoc, new Registry());
decode('%Point{x: 1.0, y: 2.0}', { registry });
// => DXNStruct { name: 'Point', version: 1, fields: { type: 'keyed', pairs: [['x', 1], ['y', 2]] } }
// -- schema-validated, decode-time enforcement; a materializer can build your own class instead
encode({ x: 1.0, y: 2.0 }, { registry, schema: 'Point' });
// => '%Point(1){x:1.0,y:2.0}'
A struct/tag with no matching registry entry still falls back to an
opaque DXNStruct/DXNCustomTag — a registry only ever adds
capability, never turns an otherwise-valid document into an error.
encode/encodeBinary automatically validate every registered
struct anywhere in a value by default (validate: false to skip),
and a schema: option additionally validates — and, for a nameless
plain value, names — the top-level value itself against one named
schema.
Nothing is converted to make a value match (DXN.md §4.7): a host
value always means the same DXN type, with or without a schema. A plain
number is a float, so a :rational field refuses it rather than
turning it into a DXNRational; say which type a value is with the
DXN* classes and their fromX() helpers. A native Date is an
instant, so it always means a datetime in UTC. A field typed Point accepts
only a Point struct — a DXNStruct named Point, or the class
registered for it — never a plain object or Map.
encode({ x: 1, y: 2 }, { registry, schema: 'Point' });
// x/y are plain numbers, which is what :float means here
Also included, each mirroring its Dextrin.Schema.* counterpart:
Schema.registerProvider (let a class's own library ship its schema
without depending on dextrin, see src/schema/provider.cjs),
Std.registry()'s standard library of common named types
(PositiveInteger, NonEmptyString, ...), and
FileResolver.forPaths([...])'s resolver from names to .dxns files,
by DXNS.md §6's convention, for Registry#putResolver:
const { decode, Schema, Registry, Std, FileResolver } = require('@joetjen/dextrin');
const withStd = Std.registry(); // PositiveInteger, NonEmptyString, ...
const registry = Schema.compile(decode('%{ Age: PositiveInteger }'), withStd);
const fileBackedRegistry = new Registry().putResolver(FileResolver.forPaths(['./schemas']));
npx dextrin decode data.dxnb # -> pretty-printed .dxn text
npx dextrin encode data.dxn --out data.dxnb # -> .dxnb binary
npx dextrin format data.dxn --mode pretty|condense [--in-place]
npx dextrin validate data.dxn [--schema schema.dxns --as Name]
npx dextrin compat old-schemas/ schemas/ # changed schemas need new versions
npm install @joetjen/dextrin
DXN's format spec (dxn/DXN.md, ported from the
Elixir project's own guides/dxn/DXN.md) is implementation-independent
— this library, dextrin (Elixir), and php-dextrin all satisfy the
same document, and stay close to dextrin's own in-memory
shape-per-type choices wherever their host language allows.
| Language | Package | Source |
|---|---|---|
| Elixir | dextrin on Hex.pm |
joetjen/dextrin |
| Node.js (this project) | @joetjen/dextrin on npm |
joetjen/node-dextrin |
| PHP | joetjen/dextrin on Packagist |
joetjen/php-dextrin |
npm install
npm run precommit
npm run precommit runs tsc --checkJs --noEmit (type-checking the
JSDoc annotations) followed by the full Mocha test suite — the check
this project expects to pass before every commit.
See CONTRIBUTING.md for how to propose changes, and CHANGELOG.md for release history.
Apache-2.0 — see LICENSE.