TOML

Stable

TOML decoding, encoding and in-place editing for Go

Used by ViF, Stapes, LogWisp, Config and Audio

GoTOML
GitHub

A deterministic encoder, decoder and formatter for a subset of TOML, in Go on the standard library alone. It edits a document in place with its comments kept, and expands the macros a document defines.

  • Decodes into structs, maps or any, with checked conversions and errors that name the key
  • Edits a document in place, keeping its comments, order and spelling
  • Formats a document with its comments kept
  • Macros a document defines, expanded after parsing
  • Bounds on depth and table count, so hostile input fails cleanly

How a document is read

Document

TOML bytesUTF-8; no datetimes, literal or multiline strings

Read

  1. Lexer tokens with their byte offsets; comments are tokens too
  2. Parser recursive descent to map[string]any, at most 65536 tables
  3. macrooptional the templates a document defines, expanded on the tree

Use

  • Decode into structs, maps or any, all or nothing; errors name the key path
  • Unmapped the keys no field takes, for a caller that refuses them
  • Document Set and Delete rewrite only the statements they name
  • Format canonical spacing, each comment kept with what it belongs to
Marshal runs the other way, Go values to bytes, written as Format writes them. Document and Format work from the tokens’ byte offsets, so whatever they do not touch keeps its bytes.
Decode
Integers are int64 on every architecture. A conversion that overflows, goes negative into an unsigned, drops a fraction or loses float precision fails, and a string is never read as a number. All or nothing: an error leaves the destination as it was and names the key as the document spells it, rooms[1].width: cannot convert string to int.
Encode
Keys sorted, scalars before tables, a table inline once its header path would pass 256 bytes. Runes that can hide text, such as bidi overrides, are escaped, as Format escapes them.
Edit
Document.Set and Delete rewrite only the statements they name, with the comments that are theirs, then parse the result again and refuse an edit it would not take. A group of comments belongs to the side fewer blank lines part it from.
Bounds
65536 tables a document, 1000 levels of recursion, invalid UTF-8 and stray control characters refused, and errors that quote keys but never repeat a value.
Subset
No datetimes, literal or multiline strings, digit separators, inf or nan, and decimal integer keys are refused. Signed and uppercase radix integers are accepted, and a leading zero is still decimal.

Macros

A document can define templates under [macros] and invoke them anywhere a value goes; the macro package expands them on the parsed tree, so decoding only ever sees plain tables. Parameters are typed and checked where they are defined, each expands once per element of zipped arrays, and a set of macros can serve several documents. The audio engine’s patterns write a drum lane or a line of notes once this way.

[macros.Note]
params = { pos = "int", deg = "int", dur = "int" }
defaults = { dur = 1 }
body = { pos = "${pos}", deg = "${deg}", dur = "${dur}" }

[[track]]
event = [{ macro = "Note", each = { pos = [0, 4, 8], deg = [0, 2, 4] } }]