node-dextrin - v0.4.0
    Preparing search index...

    node-dextrin - v0.4.0

    node-dextrin

    Warning

    Under heavy development: not for production use before 1.0.0. Things can and will break, and the specification can and will change, between any two releases until 1.0.0. Do not use this library in a production system yet.

    CI npm Docs

    A Node.js port of dextrin, an Elixir implementation of DXN (Data eXchange Notation): a human-writable text format (.dxn), a compact binary format (.dxnb) built on CBOR, and a schema format (.dxns) — all three sharing one in-memory value representation and one extension mechanism. A sibling of php-dextrin, the PHP port of the same library.

    Status: feature-complete port, published to npm. The value model, the .dxn text codec, the .dxnb binary codec, the .dxns schema system, the dextrin CLI, and the guides are all implemented. See CHANGELOG.md for exactly what's landed.

    Most serialization formats pick one point on a spectrum. JSON is human-writable but has no dates, no bytes, no exact decimals and no boundary between an integer and a float. Protobuf and Avro are compact and typed but need a schema compiled ahead of time, and aren't meant for a person to read or edit. CBOR and MessagePack are compact and self-describing but binary only. YAML and TOML are made for people, and have no binary form for machines.

    DXN's premise is that a .dxn text document and a .dxnb binary document are the same value space: 31 scalar, collection, temporal and extended types (reference §1.3). It is precise enough for money (decimals), exact ratios (rationals) and integers of any size, has struct and custom-tag as first-class, schema-describable extension points, and its schemas (.dxns) are DXN too. You choose text or binary by what matters where the data is going -- a person reading it, or bytes on the wire -- and the value is the same either way.

    DXN JSON YAML TOML BSON MessagePack CBOR Protobuf Ion
    Text a person writes ✓ ✓ ✓ ✓ – – –¹ –² ✓
    Binary, same values ✓ – – – ✓ ✓ ✓ ✓ ✓
    Comments ✓ – ✓ ✓ – – – – ✓
    Schema, part of the format ✓ ~³ – – – – – –² ~³
    Readable without a schema ✓ ✓ ✓ ✓ ✓ ✓ ✓ – ✓
    Integer and float kept apart ✓ – ✓ ✓ ✓ ✓ ✓ ✓ ✓
    Integers beyond 64 bits ✓ ~⁴ ~⁴ – – – ✓ – ✓
    Exact decimals ✓ – – – ✓ – ✓ – ✓
    Exact ratios ✓ – – – – – ✓ – –
    Timestamps, dates, times, durations ✓ – ~⁵ ~⁶ ~⁷ ~⁷ ~¹³ ~⁸ ~¹⁴
    Bytes ✓ – ~⁵ – ✓ ✓ ✓ ✓ ✓
    UUIDs, URIs ✓ – – – ~⁹ – ✓ – –
    Sets, tuples ✓ – ~⁵ – – – ~¹⁰ – –
    Map keys that are not strings ✓ – ✓ – – ✓ ✓ ~¹¹ –
    Typed records (structs) ✓ – – – – – – ✓ –
    Types an application adds ✓ – ✓ – – ✓ ✓ – ✓
    Versioned schemas, checked in CI ✓ – – – – – – ~¹² –

    ¹ CBOR has a diagnostic notation for reading, not for authoring. ² Protobuf's text format is for debugging; its schema is a separate .proto file compiled ahead of time. ³ JSON Schema and Ion Schema are written in JSON and Ion, as separate specifications. ⁴ The grammar allows any size; JavaScript's and PHP's parsers lose the digits past 53 and 64 bits. ⁵ YAML 1.1 tags (!!timestamp, !!binary, !!set), dropped from 1.2's core schema. ⁶ Dates, times and date-times; no durations. ⁷ An instant only: no local dates, times or durations. ⁸ The well-known Timestamp and Duration messages. ⁹ A UUID as binary subtype 4. ¹⁰ Sets by registered tag 258; no tuples. ¹¹ Integer and string keys only. ¹² Field numbers keep old readers working; nothing checks a change. ¹³ Instants (tags 0 and 1) and dates (RFC 8943); no times of day or durations. ¹⁴ Timestamps of any precision, dates included; no times of day or durations.

    The same four payloads in each format, in bytes, raw and gzipped. Measured with dextrin's bench/format_sizes.exs (mix run bench/format_sizes.exs); the script says how each format writes the values it has no type for (a decimal or a date as a string in JSON, for instance).

    config file API response, 100 records time series, 1,000 points record with a 4 KiB blob
    DXN text 279 / 199 19,402 / 4,218 63,159 / 13,895 5,516 / 4,222
    DXN text, pretty 314 / 216 23,504 / 4,249 67,161 / 13,961 5,527 / 4,234
    .dxnb 264 / 210 14,185 / 3,802 34,006 / 14,381 4,145 / 4,168
    .dxnb, shared 264 / 210 9,217 / 3,752 27,017 / 14,019 4,145 / 4,168
    JSON 306 / 207 19,202 / 4,201 56,159 / 13,692 5,514 / 4,216
    JSON, pretty 403 / 229 26,253 / 4,287 74,160 / 14,182 5,527 / 4,231
    BSON 312 / 231 17,621 / 4,448 36,903 / 15,700 4,161 / 4,184
    MessagePack 237 / 199 12,271 / 3,840 25,003 / 11,165 4,136 / 4,159
    CBOR 237 / 188 12,282 / 3,514 25,003 / 11,164 4,136 / 4,159
    Protocol Buffers 141 / 111 8,552 / 3,702 19,000 / 10,832 4,122 / 4,145
    • DXN text is about JSON's size -- keys need no quotes -- while carrying the types JSON drops. Data full of dates and times is the exception, about an eighth larger: each datetime names its type (@datetime "...") where JSON writes a bare string.
    • .dxnb sits between JSON and the compact binaries. Value sharing writes a repeated key or string once; on the API response that brings it within 8% of Protocol Buffers, with no schema needed to read it. This package reads shared .dxnb; the Elixir and Praxis encoders write it.
    • Where it is larger, it is carrying more. A .dxnb datetime keeps nanoseconds, its offset and its type, where CBOR's and MessagePack's carry whole seconds; a keyword key is tagged as a keyword rather than written as a plain string. Protocol Buffers is smallest because its schema, not the data, holds the field names.
    • Compressed, the gap narrows. Gzipped, DXN text stays within about 12% of .dxnb, and JSON within about a quarter of CBOR; past that, the choice is about what each format can say.

    Decoding a document from text and checking it against a schema, timed in each implementation on the same data sets: one Universe each, about 4 KB of DXN text, against bench/schemas/universe.dxns, a 225-line schema set more complex than any one met in practice (recursion, generics, cases, refinements, formats, documents inside documents). About 30% of the data sets carry exactly one error, of 44 kinds. Each implementation ran for 90 minutes on one core of an Apple M2 Max (Elixir 1.19.5 on OTP 28 with ichor 0.7.3's parser, 2026-10-26; Node 24.18.1 and PHP 8.4.24 without opcache, 2026-10-22), and every verdict matched what the data set was meant to be. Times are per data set, in milliseconds.

    data sets decode, mean decode, p99 check, mean check, p99 together, mean together, p99
    Elixir 713,050 3.45 7.74 0.16 0.93 3.60 7.97
    Node 4,698,138 0.53 0.85 0.54 1.02 1.07 1.80
    PHP 1,316,837 1.95 2.89 0.83 1.57 2.78 4.37
    • Node decodes fastest, Elixir checks fastest: Node decodes about six times as fast as Elixir, and Elixir checks about three times as fast as Node. Where a decode spends its time differs: Elixir evenly across lexing, parsing and building the values (1.07, 1.28 and 1.10 ms of 3.45), Node in its parser (0.45 of 0.53), PHP in its lexer (1.31 of 1.95).
    • A refused data set is checked about twice as fast as one that is accepted: the check stops at the first error (Elixir 0.09 against 0.18 ms, Node 0.31 against 0.63, PHP 0.42 against 1.00).
    • Node and PHP keep to their means: a p99 under 1.7 times the mean, where Elixir's is about 2.2 times.

    Reproduce it with bench/schema-validation.cjs, which reads its data sets from dextrin's bench/generate_universes.exs (the script says how); the Elixir numbers come from that repository's bench/schema_validation.exs, on the very same data sets.

    What a value is after a round trip through each language's own JSON parser, and through Dextrin:

    Written Elixir: JSON / Dextrin Node.js: JSON / Dextrin PHP: JSON / Dextrin Praxis: JSON / Dextrin
    9223372036854775808 integer / integer 9223372036854776000 / BigInt float / DXNInteger integer / integer
    19.99 as an exact decimal float / Decimal float / DXNDecimal float / DXNDecimal float / dextrin.decimal
    1.0 float / float 1, an integer / float float, written back as 1 / float float / float
    a datetime string / DateTime string / DXNDateTime string / DXNDateTime string / a praxis.time instant
    bytes base64 string / Dextrin.Bytes base64 string / DXNBytes base64 string / DXNBytes base64 string / dextrin.bytes
    a map keyed by integers string keys / integer keys string keys / integer keys string keys / integer keys string keys / integer keys
    a set list / MapSet array / Set array / DXNSet vector / set

    A bold entry is a value that did not come back as what was written. With Dextrin, the same document means the same thing in all four, and every implementation is held to that by the Dextrin Conformance Suite.

    A .dxns schema is DXN data. It is meant to say everything the schema languages of the formats above can say, except what has no value or no meaning in DXN (design/schema-superset.md goes through them construct by construct). YAML, TOML and MessagePack have no schema language of their own; people use JSON Schema for them.

    DXNS JSON Schema MongoDB $jsonSchema CDDL Protobuf Ion Schema
    Written in the format it describes ✓ ✓ ✓ – – ✓
    Typed records, with versions ✓ – – – ~¹ –
    Version changes checked in CI ✓ – – – ~¹ –
    Recursive types ✓ ✓ – ✓ ✓ ✓
    Generic types ✓ ~² – ✓ – –
    A list's items in order ✓ ✓ ✓ ✓ – ✓
    A type chosen by a sibling field ✓ ~³ – ~⁴ ~⁵ –
    String formats, checked the same everywhere ✓ ~⁶ – ~⁷ ~⁸ ~⁷
    Decimals, ratios, dates, times, durations as types ✓ – ~⁹ ~¹⁰ – ~¹¹
    Bytes holding an encoded document ✓ ✓ – ✓ – –
    Annotations and free metadata ✓ ✓ ~¹² – ✓ ✓
    Closing a type to unknown fields –¹³ ✓ ✓ ✓ – ✓
    One test suite every implementation passes ✓ ~⁶ – – – –

    ¹ Field numbers keep old readers working; checking a change takes a tool outside the language, such as buf breaking. ² $dynamicRef, since draft 2020-12. ³ if/then/else; OpenAPI adds a discriminator on top. ⁴ A group choice, //, over maps with a literal member. ⁵ oneof picks one field of several, not the type of one. ⁶ The official test suite exists, but format only annotates unless a validator opts in, and each implements the formats its own way. ⁷ Regular expressions (CDDL's .regexp, Ion Schema's regex); CDDL's .abnf takes a grammar. ⁸ Through the separate protovalidate rules. ⁹ decimal and date BSON types; no times of day or durations. ¹⁰ Through tags and the prelude (tdate, decfrac); no durations. ¹¹ decimal and timestamp, with precision and offsets; no durations. ¹² title and description only. ¹³ On purpose: see the next section.

    JSON Schema's additionalProperties: false, Ion Schema's closed and CDDL's maps refuse a field the schema does not list. DXNS has no such switch, for two reasons:

    • Closing is the reader's choice, not the type's. A config loader wants a typo in a hand-written file caught; a message consumer wants a newer producer's fields to pass through untouched. The same Order schema serves both, so whether unknown fields are refused belongs to the decode: decode(text, { registry, strict: true }) (or decodeBinary, or a registry marked with registry.putStrict(true)) refuses them (schema/undeclared-field), and without strict they are kept -- in .dxnb as one trailing item (DXNS.md §7) -- and written back out.
    • A closed type breaks the next version. Adding a field to a closed type makes every older reader refuse every newer message, which turns each compatible change into a breaking one. DXN's versions (below) rest on readers that let what they do not know pass.

    A check that needs more -- say, refusing exactly the fields another schema allows -- runs after the decode, on the value it returns, where the application knows what it means.

    Several services reading and writing the same schema never all deploy at once. DXNS makes that safe with versions, not with compatibility modes (DXNS.md §8):

    • Every schema has a version, %schema(3){...}, and every version stays loadable beside the others. A writer always writes the latest it has loaded, and says which: %Order(3){...}.
    • A newer message reaches an older reader: it reads the fields it knows, by name in text and by position in .dxnb, and keeps the rest as extra fields. This is why a field is only ever appended: in .dxnb, an older reader takes positions as they were.
    • An older message reaches a newer reader through the schema's materializer, one function per schema that upgrades the old fields to the latest version (registry.putStructMaterializer(name, (name, version, fields) => ...)). Here it is called for every decoded struct of that name, whatever its version, with the version the fields were read with, so it builds the latest value from any of them. Without one, reading an older version is an error, not a guess.
    • CI checks the discipline: npx dextrin compat OLD NEW (or Schema.compat(oldRegistry, newRegistry)) compares the last release's schemas with the current ones, each a .dxns file or a directory of them. A version whose content changed is an error -- two documents claiming to be Order v3 must mean the same thing -- as is a version that went down, and it lists, for each schema, what changed since each older version: the materializer's checklist.

    So a change deploys like this:

    1. A compatible change -- fields appended, nothing else -- deploys in either order.
    2. A breaking change -- a field removed, moved or retyped -- deploys readers first, each with a materializer for the old version, then the writers.
    3. A schema that services both read and write changes only compatibly, or is replaced by a new name (OrderV2) beside the old one, and the services move over one at a time.

    DXN's premise (see the Elixir project's own README for the full case) is that a .dxn text document and a .dxnb binary document are the same value space — scalar, collection, temporal, and extended types precise enough for money, exact ratios, and arbitrary-precision integers, with a schema-describable extension story. DXN.md (ported here once the text/binary codecs land) is the implementation-independent, normative spec; this library only needs to satisfy it.

    Where this library differs from the Elixir original, it's by design, documented at the point of difference:

    • Every DXN type decodes to a native JS value where one exists losslessly (null, boolean, string, number for float, a native Map/Set/Array, ...) and to a small DXN* wrapper class only where nothing native fits without losing information — mirroring the same design principle dextrin's own Dextrin.Value follows, not necessarily the same type-by-type choice (see src/values/ for each class's own rationale).
    • integer is always BigInt, never number — JS number can't hold arbitrary precision the way Elixir's native integer does, so this is the one place fidelity requires a deviation from "closest native type."
    • float needs no wrapper at all, unlike Elixir (which resorts to atoms for NaN/Infinity/-Infinity because the BEAM can't construct a non-finite float term) — JS number natively holds all of IEEE-754, so this is a case where the native JS type is a better fit than Elixir's own.
    • Any wrapper class with a close-but-imperfect native JS counterpart ships two-way conversion helpers (toX() / DXN*.fromX()) rather than forcing you through the wrapper alone — e.g. DXNRational. fromNumber() (exact — every finite JS number is a binary fraction), DXNOrderedMap.toMap()/.fromMap(), DXNRegex.toRegExp() (guarded — only when no DXN-only flag is set).
    • A documented limit: a regex pattern is not checked. DXN refuses a pattern PCRE does not compile (@regex "a(" is syntax, DXN.md §1.2), and the Elixir, PHP and Praxis implementations check it with PCRE. JavaScript has no PCRE, and its own engine refuses valid PCRE -- inline modifiers such as (?im), atomic groups, \A -- so it cannot stand in: node-dextrin reads any pattern, and DXNRegex. toRegExp() is where a pattern JavaScript cannot run fails.

    Every DXN type maps to either a native JS value or a small DXN* wrapper class under src/values/. Full mapping table, including the rationale for each wrapper-vs-native choice, lives in this project's development plan for now and will move here as the port progresses.

    const { DXNRational, DXNUUID } = require('@joetjen/dextrin');

    const r = new DXNRational(22n, 7n);
    r.toNumber(); // => 3.142857142857143

    DXNRational.fromNumber(0.5); // => DXNRational { numerator: 1n, denominator: 2n }

    const id = DXNUUID.fromString('550e8400-e29b-41d4-a716-446655440000');
    id.toString(); // => '550e8400-e29b-41d4-a716-446655440000'
    const { decode, encode } = require('@joetjen/dextrin');

    const value = decode('%{x: 1, y: 2}', { trusted: true });
    // => DXNMap(2) { Symbol(x) => 1n, Symbol(y) => 2n } -- a Map whose keys compare by value

    encode(value);
    // => '%{x:1,y:2}'

    encode(value, { pretty: true });
    // => '%{\n x: 1\n y: 2\n}'

    Decoding is safe by default: keyword (:name) decodes as DXNKeyword, the same way Dextrin.Keyword exists on the Elixir side, so no input can fill the global symbol registry, which is never garbage collected. Pass trusted: true (or registry.putTrusted(true)) for a source you control, as above, and it decodes as a real, global Symbol.for(name) — the closest native analog to a BEAM atom. decode/encode throw DXNError on failure — see src/error.cjs for why that's a deliberate divergence from Dextrin's own {:ok, _} | {:error, _} tuples (an API-shape choice, not a value-representation one).

    With no registry: option, every struct decodes as an opaque DXNStruct, and every tag name other than the built-ins (@uuid, @duration, @ordered, ...) decodes as DXNCustomTag — see .dxns schemas below for schema-backed decoding.

    const { decode, decodeBinary, encodeBinary } = require('@joetjen/dextrin');

    const value = decode('%{x: 1, y: 2}');
    const bin = encodeBinary(value); // => Buffer, "DX" + version + CBOR
    decodeBinary(bin); // => the same value back

    Hand-rolled CBOR over Buffer (not built on a generic CBOR library — bignum precision, datetime integer-only encoding, and the private tag block aren't things a generic library gets right by default; see src/binary/encoder.cjs's own doc). decodeBinary accepts documents using the spec's value-sharing extensions (DXN.md §2.4/§2.5) even though encodeBinary doesn't produce them yet — accepting is spec-mandatory, producing is optional, and that asymmetry is deliberate for now.

    const { decode, encode, Schema, Registry } = require('@joetjen/dextrin');

    const schemaDoc = decode(`
    %{
    Point: %schema{ x: :float, y: :float }
    }
    `);
    const registry = Schema.compile(schemaDoc, new Registry());

    decode('%Point{x: 1.0, y: 2.0}', { registry });
    // => DXNStruct { name: 'Point', version: 1, fields: { type: 'keyed', pairs: [['x', 1], ['y', 2]] } }
    // -- schema-validated, decode-time enforcement; a materializer can build your own class instead

    encode({ x: 1.0, y: 2.0 }, { registry, schema: 'Point' });
    // => '%Point(1){x:1.0,y:2.0}'

    A struct/tag with no matching registry entry still falls back to an opaque DXNStruct/DXNCustomTag — a registry only ever adds capability, never turns an otherwise-valid document into an error.

    encode/encodeBinary automatically validate every registered struct anywhere in a value by default (validate: false to skip), and a schema: option additionally validates — and, for a nameless plain value, names — the top-level value itself against one named schema.

    Nothing is converted to make a value match (DXN.md §4.7): a host value always means the same DXN type, with or without a schema. A plain number is a float, so a :rational field refuses it rather than turning it into a DXNRational; say which type a value is with the DXN* classes and their fromX() helpers. A native Date is an instant, so it always means a datetime in UTC. A field typed Point accepts only a Point struct — a DXNStruct named Point, or the class registered for it — never a plain object or Map.

    encode({ x: 1, y: 2 }, { registry, schema: 'Point' });
    // x/y are plain numbers, which is what :float means here

    Also included, each mirroring its Dextrin.Schema.* counterpart: Schema.registerProvider (let a class's own library ship its schema without depending on dextrin, see src/schema/provider.cjs), Std.registry()'s standard library of common named types (PositiveInteger, NonEmptyString, ...), and FileResolver.forPaths([...])'s resolver from names to .dxns files, by DXNS.md §6's convention, for Registry#putResolver:

    const { decode, Schema, Registry, Std, FileResolver } = require('@joetjen/dextrin');

    const withStd = Std.registry(); // PositiveInteger, NonEmptyString, ...
    const registry = Schema.compile(decode('%{ Age: PositiveInteger }'), withStd);

    const fileBackedRegistry = new Registry().putResolver(FileResolver.forPaths(['./schemas']));
    npx dextrin decode data.dxnb                                       # -> pretty-printed .dxn text
    npx dextrin encode data.dxn --out data.dxnb # -> .dxnb binary
    npx dextrin format data.dxn --mode pretty|condense [--in-place]
    npx dextrin validate data.dxn [--schema schema.dxns --as Name]
    npx dextrin compat old-schemas/ schemas/ # changed schemas need new versions
    • Tutorial — a walkthrough of the JS API, every snippet verified to actually run.
    • Examples — worked scenarios (config files, schema-backed API payloads, cross-file schemas, third-party schema providers, ...).
    • Cheatsheet — quick API/type/CLI lookups.
    • DXN reference — the full, implementation -independent format specification, plus its own tutorial, examples, and cheatsheet for the format itself.
    npm install @joetjen/dextrin
    

    DXN's format spec (dxn/DXN.md, ported from the Elixir project's own guides/dxn/DXN.md) is implementation-independent — this library, dextrin (Elixir), and php-dextrin all satisfy the same document, and stay close to dextrin's own in-memory shape-per-type choices wherever their host language allows.

    Language Package Source
    Elixir dextrin on Hex.pm joetjen/dextrin
    Node.js (this project) @joetjen/dextrin on npm joetjen/node-dextrin
    PHP joetjen/dextrin on Packagist joetjen/php-dextrin
    npm install
    npm run precommit

    npm run precommit runs tsc --checkJs --noEmit (type-checking the JSDoc annotations) followed by the full Mocha test suite — the check this project expects to pass before every commit.

    See CONTRIBUTING.md for how to propose changes, and CHANGELOG.md for release history.

    Apache-2.0 — see LICENSE.