Text editor plugins for Salt states and YAML/Jinja

If you have ever written a couple of Salt states, these problems may sound familiar:

• A missing colon after the YAML key or a missing space after the colon
• Data failed to compile: State ‘foo’ in SLS ‘bar’ is not formed as a list
• A ParserError or ScannerError while running the state.highstate
• A nested dictionary, indented with two spaces instead of four
• State requirements at the wrong indentation level
• Indentation with mixed tabs and spaces, which took a surprisingly long time to debug

And even if your states are valid YAML, they still can have no sense for Salt because your wathc_in requisite has a typo. Sprinkle a few Jinja tags here and there, and the errors can get even fancier…

Is there anything that could be done?

These issues definitely have come up before (#802). Smart developers probably have this figured out, right?

Indeed, it is possible to validate your states, in a semi- or completely automated way, even before they touch your infrastructure. For Salt, you have the following options:

1. Use the salt-call --local --retcode-passthrough state.show_sls YOUR_STATE either manually or using a commit hook. This triggers the state rendering process and can catch tons of possible errors.

2. Test your states using Test Kitchen together with kitchen-salt and some verifier. A helpful guide on this topic is available here

However, all of this feels a bit heavy. You need to commit and push to run your tests, spin up VMs or containers, dive into Ruby code, and write tests for all possible pillar/grain permutations. Such a long feedback cycle could easily lead to git commit -m "Fuck YAML" in your commit history and a subsequent amend/rebase/squash dance (and you can’t squash the obscene commit notifications sent to the rest of your team!).

Is there something more lightweight?

Well, your development environment can help you. The first layer of defense can work right in your text editor as you type, providing immediate feedback. It will catch your typos before they get committed and deployed (and let the CI system deal with more complicated stuff). An ideal editor plugin can:

• Help you deal with YAML indentation levels and data structures
• Know and highlight the most common Salt state keywords
• Highlight Jinja brackets and tags
• Autocomplete the keywords

Below I’ll cover plugins for 9 different editors, so you do not have to switch away from your favorite one. Also, I will give a few installation/configuration tips to save you some googling. These plugins aren’t ideal, but they are open source, and you can contribute to any of them if you want (I did).

• Atom
• Eclipse
• Emacs
• Kate
• Midnight Commander
• Sublime Text
• PyCharm
• Vim
• Visual Studio Code
• Final tips

If your favorite editor is not covered here, or you want to share a nifty hack that helps you reduce the number of YAML mistakes, ping me on Twitter.

Atom

Out of the box, Atom understands *.sls files as YAML. Plus, you could install atom-jinja2 and manually switch a file grammar to YAML (Jinja Templates) by pressing Ctrl+Shift+L.

For better results, you can install both atom-salt and atom-jinja2. Just click Edit -> Preferences -> Install, search for atom-salt or atom-jinja and install them.

Features

• The first atom-salt language scope is source.yaml.salt, which is automatically enabled for *.sls files and provides syntax highlighting (YAML+Jinja combo)
• The second scope source.python.salt is not associated with any file extension and instead is enabled for any files which start with #!py. This means you get syntax highlighting for py, pydsl, and pyobjects Salt renderers.
• Also, it should open Salt docs via the salt-doc command, but it didn’t work for me for some reason
• Atom-jinja2 provides the source.yaml.jinja scope, which is enabled for *.jinja, *.j2 and a bunch of other extensions
• Supports syntax highlighting and folding
• Has a couple of Jinja-related snippets

You can assign specific language scope to any extension with something like this in your ~/.atom/config.cson:

"*":
  core:
    customFileTypes:
      "source.python.salt": [
        "py.sls"
        "pysls"
      ]

Other things to try

• Language-salt - an alternative Atom plugin for Salt (enabled for *.sls files, provides the source.salt scope and a bunch of snippets)
• Aligner-salt (GitHub) to visually align YAML attributes
• The indentation guides should be enabled by default by Salt plugin, but you can enforce it by clicking Edit -> Preferences -> Editor -> Show Indent Guide

Eclipse

I was unable to find a good enough solution for Eclipse, but below are two directions you can try to explore:

1. Go to Help -> Eclipse Marketplace, search for “yedit” and install it. Then go to Window -> Preferences -> General -> Editors -> File Associations and associate the *.sls extension with YEdit. This enables syntax highlighting for plain YAML files (no luck with Jinja though).

2. Another plugin is called LiClipseText. It has support for both YAML and Jinja syntaxes, but can’t enable them simultaneously for a single file. It should be possible though, and there are some hints if you want to contribute.

Indentation guides are available via a separate indent-guide plugin.

Emacs

Salt-mode is your best bet. It is based on yaml-mode and mmm-mode and is automatically enabled for *.sls files. The simplest way to install is through MELPA.

Features

• Syntax highlighting for *.sls files (also possible to enable *.jinja syntax support)
• Indentation and alignment of expressions and statements
• Spell checking of comments with flyspell
• Use salt-mode-browse-doc to browse the documentation of the state module at point
• Quickly move between Salt state functions

Things to be aware of

• No auto-completion yet
• It uses yaml-mode under the hood, which needs a new maintainer. If you are willing to step in, there are a number of issues for you to fix.

Other things to try

• Indent-tools together with hydra looks like a good combo for whitespace-sensitive languages like YAML and Python.
• Highlight-indent-guides is worth looking at (although it adds more visual clutter). Also, there is a crosshair highlighting mode.
• Install dash-at-point if you use Dash API Documentation Browser
• To highlight other Jinja-templated configuration files, establish the convention of naming them using the *.jinja extension and add the following lines to your init.el (see the README for more details):

(mmm-add-mode-ext-class nil "\\.jinja\\'" 'jinja2)

Kate

Kate-jinja2-highlighting has a bunch of syntax highlighters for different file types with Jinja on top.

Features

• Highlights *.jinja, *.jinja2 and *.j2 using Jinja2 syntax
• Highlights *.yaml, *.yml and *.sls files using YAML+Jinja2 syntax (if you apply the patch below)
• If you want to enable indentation guide, toggle the Settings -> Configure Kate -> Editor Component -> Appearance -> Show indentation lines

To install it, copy the *.xml files into the ~/.local/share/org.kde.syntax-highlighting/syntax/ folder. Then go to Settings -> Configure Kate -> Editor Component -> Open/Save -> Modes & Filetypes, select Markup/Jinja2/YAML, open the Variables drop-down, and then set both indent-width tab-width to 2 and Indent using to Spaces.

You also need to associate *.sls files with the syntax highlighter:

sed -i'' -e 's/\(\*\.yml\)"/\1;*.sls"/' yaml-jinja2.xml

The sed command above is equivalent to the patch below:

--- yaml-jinja2.xml.orig    2018-10-25 02:39:31.000000000 +0700
+++ yaml-jinja2.xml 2018-10-25 02:39:40.000000000 +0700
@@ -3,7 +3,7 @@
 <!-- Author: Dr Orlovsky MA <maxim@orlovsky.info> //-->
 <!-- Autogenerated from yaml.xml -->
 <language name="Jinja2/YAML" version="1.2" kateversion="2.3" section="Markup"
- extensions="*.yaml;*.yml" mimetype="text/yaml"
+ extensions="*.yaml;*.yml;*.sls" mimetype="text/yaml"
           author="Dr Orlovsky MA (dr.orlovsky@gmail.com)" license="LGPL">
   <highlighting>
     <contexts>

Midnight Commander

Midnight Commander is an oldie but goodie for those of us who used orthodox file managers back in the 90’s and can’t live without one. Personally, I use it to quickly navigate the filesystem tree and view the files without opening them in my primary editor. According to this ticket, out of the box, mcedit highlights *.yml and *.yaml files and understands simple Jinja variables (no support for tags though). Also, it does not understand *.jinja files.

To enable *.sls highlighting you need to make a copy of the default Syntax file and tweak it a bit:

mkdir -p ~/.config/mc/mcedit
sed -e 's/\(|YML)\)/|sls\1/' /usr/share/mc/syntax/Syntax > ~/.config/mc/mcedit/Syntax

The sed command above is equivalent to the patch below:

--- /home/user/.config/mc/mcedit/Syntax.orig 2017-08-05 04:03:29.000000000 +0700
+++ /home/user/.config/mc/mcedit/Syntax 2018-10-10 19:50:11.390542735 +0700
@@ -271,7 +271,7 @@
 file ..\*\\.cl$ OpenCL\sProgram
 include opencl.syntax

-file ..\*\\.(ya?ml|YML)$ YAML\sFile
+file ..\*\\.(ya?ml|sls|YML)$ YAML\sFile
 include yaml.syntax

 file .\*\\.osl$ OSL\sProgram

PyCharm

Well, it looks there are no Salt-specific plugins for PyCharm. Ansible users are luckier here - PyCharm has support for JSON Schemas, and someone from RedHat has created one. This means code auto-completion for Ansible-specific YAML data structures right out of the box! Plus, there is a separate YAML/Ansible plugin (GitHub).

However (aside from using Ansible plugins), you can tweak PyCharm to make it work with Salt states. Click File -> Settings -> Languages & Frameworks -> Python Template Languages, select Jinja2 in the Template language drop-down, click the “+” button and select YAML. This should enable syntax highlighting for YAML+Jinja combo. To associate it with the *.sls files, go to File -> Settings -> Editor -> File Types, find YAML and add the *.sls extension.

Other features

• You can reformat YAML code with the Reformat Code action or via a shortcut: Cmd + Alt + L on MacOS / Ctrl + Alt + L on Windows and Linux
• To customize YAML code style settings, visit File -> Settings -> Editor -> Code Style -> YAML
• Indentation guides are enabled by default, but you can disable them via File -> Settings -> Editor -> General -> Appearance -> Show indent guides

Sublime Text

The extension is available on GitHub. To install it click Preferences -> Package Control -> Install Package, search for saltstack and click to install. It is also recommended to install the separate extension for Jinja (go to Package Control and search for jinja2).

Features

• Syntax highlighting for *.sls files
• Syntax highlighting for *.j2 files if you install the Jinja extension
• Several snippets for Jinja (forked from sublime-jinja2)
• Several snippets for often used Salt state modules like file and pkg

Indentation guides are enabled by default, but you may also want to set "indent_guide_options": ["draw_normal", "draw_active"] in the settings to highlight currently active guide.

Vim

You need both salt-vim and vim-jinja2-syntax.

Features

• Syntax highlighting for *.sls files and Saltfile
• Syntax highlighting for *.jinja2, *.j2 and *.jinja files
• Proper indentation settings
• Space key will try to fold/unfold an area
• A visually selected block might be indented and unindented with < and >

If you are using Vim 8 or higher, you may skip the Pathogen/Vundle route and install the plugins directly:

mkdir -p ~/.vim/pack/plugins/start
cd !$
git clone https://github.com/saltstack/salt-vim.git salt.vim
git clone https://github.com/Glench/Vim-Jinja2-Syntax.git jinja2.vim

Then add the following lines to your .vimrc:

syntax on
set nocompatible
set modeline
filetype plugin indent on
let g:sls_use_jinja_syntax = 1

You might also consider vim-indentguides, vim-indent-guides or indentLine. Some folks use crosshair highlighting.

Visual Studio Code

The VSCode extension for SaltStack is available on the Marketplace. To install it click File -> Preferences -> Extensions, then search for saltstack and click install. The source code is available on GitHub

Features

• Has syntax highlighting for *.sls files (understands YAML+Jinja combo)
• Autocompletion of Jinja brackets
• Folding of indented lines in YAML
• Also supports *.j2 and *.jinja extensions

If you get annoyed by extra closing bracket added by autocompletion {%%}}, add the following snippet to your settings.json file (File -> Preferences -> Settings, then press the {} icon to open User Settings):

"[sls]": { "editor.autoClosingBrackets": "never" }

Final tips

For teams

If you have a team of engineers who regularly contribute to a common Salt state tree, it is essential to establish some YAML/Jinja style conventions.

You could add the following boilerplate to each state file, but it is easy to forget and requires some discipline:

# -*- coding: utf-8; mode: salt; -*-
# vim: ft=sls

Alternatively, you could ask engineers to set up their editors properly. Or you can adopt http://editorconfig.org/ - just create a .editorconfig in your project root, and any compatible editor will automatically pick up your coding convention.

Also, it makes sense to add .jinja or .j2 suffix to all config templates managed by Salt, to help most editors recognize the syntax.

YAML docs

• YAML Cheatsheet (also could be used with Dash)
• The YAML Format (from the Symfony docs)
• YAML Multiline (a nice cheatsheet for YAML multiline strings)
• Understanding YAML (from the Salt docs)
• YAML Idiosyncrasies (from the Salt docs)
• YAML Specification (authoritative and comprehensive, but not so easy to read)

Jinja docs

• Jinja Template Designer Documentation (please choose the appropriate Jinja version to match the one installed in your Salt environment)
• Understanding Jinja (from the Salt docs)

Online YAML parsers/linters

These could be helpful if you don’t want to or can’t use any editor plugins:

• https://github.com/Inveracity/jinjabread (made specifically for Salt)
• http://yaml-online-parser.appspot.com/
• https://www.json2yaml.com/
• http://www.yamllint.com/

Also, YAML is a superset of JSON, so you could just use square brackets for lists and curly ones for dictionaries:

/etc/http/conf/http.conf:
  file.managed:
    - source: salt://apache/http.conf
    - mode: 644
    - template: jinja
    - context: { custom_var: "override" }
    - defaults: { custom_var: "default value", other_var: 123 }

If you want to be notified about tips like this one, you can subscribe to the mailing list.

Also, you can follow me on Twitter where I periodically post things like this: