DEV Community

Daniil Baturin
Daniil Baturin

Posted on

TOML: the annoying parts

If your project needs a configuration file, TOML is usually the best format.
It's definitely easier to read and write than JSON, and much less fragile and ambiguous than YAML. There are comments, real booleans, and multi-line string literals.
It also offers better flexibility than classic INI-like formats, and there's a spec where users can learn the syntax.

I went for TOML when I wrote the Soupault static site generator/HTML processor, and in these 1.5 years that I've been using it and working on it, I never thought "I wish I'd chosen something else". None of the users told me they wish it used something else either.
However, there are still annoying points that I hope will be improved some day.

Paste-ability

Don't we all like to paste configuration snippets from tutorials and other people's configs?

When nested tables are involved, TOML doesn't make it especially easy to paste, or even discuss configuration options.

Consider this config:

[settings]
  verbose  = true
  debug    = false
Enter fullscreen mode Exit fullscreen mode

How are we going to talk about those options in the docs?
Let's try this: "When settings.verbose = true, build progress output will be printed to the console".
Users cannot paste settings.verbose = true into that config because TOML doesn't allow re-defining options.

This is an invalid config because the verbose option is re-defined:

settings.verbose = true

[settings]
  verbose  = false
  debug = false
Enter fullscreen mode Exit fullscreen mode

It gets funnier. This is technically valid TOML:

[settings]
  verbose  = false
  debug = false

settings.verbose = true
Enter fullscreen mode Exit fullscreen mode

It just doesn't mean what one may think it means: the last option is interpeted as settings.settings.verbose.

So, best you can do when talking about options is to use lengthy descriptions like "Option verbose in the [settings] table".
Well, or avoid using tables, but it defeats the purpose.

If I were designing a format from scratch, I would likely avoid having both [table]\n key and table.key notations because they clearly don't compose.

Unordered tables

Sometimes I wish the standard mandated that tables were ordered dicts. If you need order, you can only use lists of tables (and their syntax and behaviour are rather clumsy and not obvious), or provide a different way to specify the order.

When I wanted to make it possible to use output of one "widget" as an input for another, I chose to add an explicit after option.

# Highlights the link to the current section in the navigation menu
[widgets.section-link-highlight]
  after = "insert-nav-menu"
  widget = "section-link-highlight"

  active_link_class = "nav-active"
  selector = "div#nav"
Enter fullscreen mode Exit fullscreen mode

Maybe the fact that users can specify multiple dependencies with after = ["widget-1", "widget-2"] is a good thing, but it may also make complex configs hard to follow.

I suppose one reason TOML designers made tables unordered is that it's how hash tables/dicts work in most languages. In Python you have OrderedDict, and in OCaml or Haskell you can use a (string × α) list. There's no equally common and straightforward way to do that in JS or many other languages.

Automatic manipulation

Unordered tables and "more than one way to do it" mean parsing and formatting TOML objects isn't a lossless operation. None of existing libraries can parse a TOML file and print it back as it came.

It's hard to blame them. Preserving comments, line numbers, and whether a table was normal or inline for example—these are all pretty hard tasks. They are also unnecessary if you only want to de/serialize TOML.

If you are writing a validator or convertor, then things are different.

When I had to make some breaking changes to the config syntax, I made a convertor from the old format to new to simplify migration for users.

However, what it outputs is quite far from the config that comes in. You'll lose all your comments at the very least.
I tried to alleviate it by making the convertor output a detailed log of the changes it's making so that users can replicate them by hand.

Conclusion

I don't regret going with TOML for the configuration format. There's still room for improvement in its tooling and usage conventions though, and I hope that I or someone else will be able to make those improvements.

Top comments (0)