Learn › Salt Configuration Management › Production and Capstone

Formulas

Build a reusable Salt formula with the standard directory structure and map.jinja.

An empty folder, and one word that will do all the work

There is a directory on this master called /srv/formulas. It is empty. There is another called /srv/salt, and it holds one state file somebody else wrote before you arrived.

By the end of this lesson you will type one word on a command line. A machine you never log into will end up with a package installed and a configuration file written, both described by files you wrote yourself. The same word and the same files would do the right thing on a Red Hat machine, unchanged.

Getting there costs four things. A directory tree with a particular shape. A file that knows what each family of Linux calls things. An entry point that uses it. And one line of master configuration, without which the whole arrangement is invisible.

The black boxes below are a practice terminal: a safe sandbox that checks the one command each step teaches. Every screen in this lesson was captured by booting this exact lab and doing exactly what you are about to do. Your own timestamps and durations will be your own. The lab user is saltops, the master is named saltmaster, and there is one minion called minion1. Its key is already accepted for you. Your progress in the lab is tracked automatically, so type commands naturally.

Look at the two folders this lesson lives between

/srv/salt is where every state file you have written so far has lived. /srv/formulas is new, and it is where today's work goes.

Print both at once. Naming two directories makes ls label each one before its contents. Watch the owner column.

ls -l /srv/formulas/ /srv/salt/

prompt: saltops@saltmaster:~$ answer: ls -l /srv/formulas/ /srv/salt/ ||| ls -l /srv/formulas /srv/salt ||| ls -l /srv/salt/ /srv/formulas/ output: /srv/formulas/: total 0

/srv/salt/: total 4 -rw-r--r-- 1 saltops saltops 82 Aug 4 22:55 base-packages.sls hint: The long listing of two directories at once: ls -l /srv/formulas/ /srv/salt/

/srv/formulas/ says total 0. That is not an error. It is an empty directory, and filling it is your job today.

/srv/salt/ has one file, base-packages.sls, 82 bytes. It is an ordinary state file, and it matters again on the last screen of this lesson. Remember that it exists.

Read the owner column: saltops saltops, your own account. Every file you write here is a plain redirect with no sudo in front of it.

What a formula actually is

A formula is a directory of Salt files arranged in a particular way. That is the whole of it.

There is no formula keyword, no formula command and no formula loader. A formula is a shape people agreed on so that state files could move between machines and teams without being rewritten each time.

The shape is two directories deep:

nginx-formula/          <- outer directory. The path you list in file_roots
  nginx/                <- inner directory. This name IS the state name
    init.sls            <- the entry point, loaded when you apply that name
    map.jinja           <- the per-platform values init.sls uses

Three consequences follow, and each one costs people time when they meet it the hard way.

The inner directory name is what you type on the command line. The -formula suffix on the outer one is a label for humans and appears in no command.
init.sls is what Salt loads when you name a directory instead of a file, the same way a web server asks a folder for its index page.
Nothing is a formula because of where it sits on the disk. It becomes reachable only when file_roots names the directory above it.

That third one is the whole difficulty of this lesson, and you are going to walk into it on purpose.

Where the shape came from

As Salt spread, everybody wrote the same states. Install nginx. Drop a config. Make sure it survives a reboot. The obvious fix was to publish the good ones, and a problem appeared the moment anyone tried: a state tree that works on your master hardcodes your package names and your paths, so handing it over hands somebody an afternoon of editing.

So the community settled on a shape. An outer directory named after the software, an inner directory that is the state name, an init.sls front door, and a map.jinja holding everything that differs between one kind of Linux and another. Hundreds of formulas in that shape sit on GitHub today, and every one is installed the same way. Put the directory somewhere, add it to file_roots, apply the inner name.

Carry this sentence: a formula is a state tree with a front door and a translation table. The front door is init.sls, the translation table is map.jinja, and everything else in the convention exists so other people can find those two.

Commit: which name do you type?

You are about to build the tree above. Before you do, decide what you will eventually type to apply it.

[ ] nginx-formula, because that is the directory you tell file_roots about.
[x] nginx, the inner directory, because that is the state name and it holds init.sls.
[ ] init, because init.sls is the file that actually runs.

>>> nginx, the inner directory. Once /srv/formulas/nginx-formula is a file root, the folder nginx inside it is a state name and nginx/init.sls is what it loads. If you picked the first answer, you named the thing you configure rather than the thing you apply: the outer directory is a location, and locations go in file_roots, not on command lines. If you picked the third, init is never typed. It is the filename Salt falls back to when you hand it a directory.

Build the tree

Two directories, one command. mkdir -p creates every level of a path that does not exist yet, and stays quiet about the levels that do.

mkdir -p /srv/formulas/nginx-formula/nginx

No sudo. /srv/formulas is yours, as the owner column proved a moment ago.

mkdir prints nothing when it works, so go and look. find walks a tree, and -type d narrows what it prints to directories only.

find /srv/formulas -type d

prompt: saltops@saltmaster:~$ answer: find /srv/formulas -type d ||| find /srv/formulas/ -type d ||| find /srv/formulas -type d -print output: /srv/formulas /srv/formulas/nginx-formula /srv/formulas/nginx-formula/nginx hint: Walk the tree and show directories only: find /srv/formulas -type d

Three lines. find counts the directory you started from, which is why /srv/formulas itself is first. Underneath it, the two levels you asked for: nginx-formula is the outer directory, the path you will hand to file_roots later, and nginx inside it is the name you will type on the command line.

Building this structure with an init.sls inside it is the first of the five things your lab grades.

Write the translation table

Here is the real problem map.jinja solves, and it has nothing to do with YAML.

The same software is called different things on different systems, and keeps its files in different places. A state file that hardcodes nginx and /etc/nginx/conf.d only works on the machines you happened to test on.

A grain is a fact the minion reports about itself, which you met when you targeted machines by their properties. grains.filter_by takes a dictionary keyed by grain values and hands back the entry matching the machine it is rendering for.

Write it into the inner directory:

cat > /srv/formulas/nginx-formula/nginx/map.jinja << 'EOF'
{% set nginx = salt['grains.filter_by']({
    'Debian': {'pkg': 'nginx', 'conf': '/etc/nginx/conf.d/lab.conf'},
    'RedHat': {'pkg': 'nginx', 'conf': '/etc/nginx/conf.d/lab.conf'},
}, default='Debian') %}
EOF

That is a heredoc: everything between << 'EOF' and the closing EOF becomes the file. Quoting 'EOF' stops the shell touching anything inside, which matters here because the file is full of braces and quotes it would happily mangle.

Read it back. Never trust a heredoc you have not printed.

prompt: saltops@saltmaster:~$ answer: cat /srv/formulas/nginx-formula/nginx/map.jinja ||| cat /srv/formulas/nginx-formula/nginx/map.jinja | cat output: {% set nginx = salt'grains.filter_by' %} hint: Print the file you just wrote, full path: cat /srv/formulas/nginx-formula/nginx/map.jinja

Four lines, and every piece of them is doing a job.

{% set nginx = ... %} is Jinja creating a variable called nginx. Anything that imports this file gets that variable and nothing else.
salt['grains.filter_by'] is how Jinja calls a Salt execution function, the same functions you type after a target on the command line.
The two keys, Debian and RedHat, are values of the os_family grain, which is what filter_by looks at unless told otherwise. Each holds a small dictionary of the values that differ.
default='Debian' is the fallback for a machine whose family matches no key. Without it, an unknown machine gets nothing and fails further down wearing a much less helpful error.

Read the two rows honestly: for this pair of families the package name and the drop-in path happen to agree. The mechanism is the lesson, not the coincidence. Add a family whose package is called something else, and every state that says {{ nginx.pkg }} follows it without an edit.

A map.jinja inside the formula is the third of the five things your lab grades.

Write the front door

init.sls is the file Salt loads when you apply the inner directory name. It imports the map, then describes the work in terms of that variable rather than any one distribution.

cat > /srv/formulas/nginx-formula/nginx/init.sls << 'EOF'
{% from 'nginx/map.jinja' import nginx with context %}

nginx_package:
  pkg.installed:
    - name: {{ nginx.pkg }}

nginx_lab_conf:
  file.managed:
    - name: {{ nginx.conf }}
    - contents: |
        # written by the nginx formula
        server_tokens off;
    - require:
      - pkg: nginx_package
EOF

Print it back and read it through.

prompt: saltops@saltmaster:~$ answer: cat /srv/formulas/nginx-formula/nginx/init.sls ||| cat /srv/formulas/nginx-formula/nginx/init.sls | cat output: {% from 'nginx/map.jinja' import nginx with context %}

nginx_package: pkg.installed:

name: {{ nginx.pkg }}

nginx_lab_conf: file.managed:

name: {{ nginx.conf }}
contents: |

# written by the nginx formula server_tokens off;

require:
pkg: nginx_package

hint: Print the file you just wrote, full path: cat /srv/formulas/nginx-formula/nginx/init.sls

The first line is the one worth slowing down for.

{% from 'nginx/map.jinja' import nginx with context %} names the map file the way Salt sees it, not the way the disk does. There is no /srv in that path. Salt addresses files relative to the file roots, so the map is nginx/map.jinja. That is another reason the inner directory name matters: it is part of every import in the formula.

with context passes the current variables through, which is what lets grains.filter_by see the machine it is rendering for. Leave it off and the map is rendered blind.

Then two ordinary states, each with an ID you chose. pkg.installed takes - name: {{ nginx.pkg }}, and file.managed takes - name: {{ nginx.conf }}. Those {{ }} slots are filled in on the master, so the minion receives a plain state file with real names in it.

- contents: | writes the two indented lines below it into the file, and the | is YAML for take this block as literal text. Last, - require: - pkg: nginx_package says do not attempt the config until the package state has succeeded.

This formula installs a package and writes a config, and it deliberately stops there. Your minion is a container running the Salt minion as its first process, with no init system behind it, so a service.running state answers Running in OFFLINE mode. Nothing to do and nothing starts. A formula that claimed to start nginx here would be one you could not check. Everything this one claims, you can go and look at.

Commit: can Salt find it yet?

Both files are on the master's own disk, in a tree you built, spelled correctly. Take a position before you press Enter.

[ ] It applies. The files are on the master's own disk under /srv, which is where Salt keeps states.
[x] It fails. Nothing in the master's configuration points at the formula directory, so the name cannot be resolved.
[ ] It applies, with a warning that the formula sits outside the usual state tree.

>>> It fails. Salt never goes hunting through the filesystem for state files. It builds a list of directories from file_roots and searches that list and nothing else, so a perfect formula in a directory nobody listed is invisible. If you picked the first answer, /srv carries no special power: /srv/salt is reachable because a config file says so. If you picked the third, Salt has no notion of a usual tree to be outside of. A name resolves or it does not.

Apply it and watch it fail

Ask minion1 to apply the state name you decided on earlier. Same state.apply you have used since module 2, and the name after it is the inner directory.

sudo salt 'minion1' state.apply nginx

Read every line of what comes back, including the first one.

prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' state.apply nginx ||| sudo salt "minion1" state.apply nginx ||| sudo salt minion1 state.apply nginx output: ERROR: Minions returned with non-zero exit code minion1: Data failed to compile: ---------- No matching sls found for 'nginx' in env 'base' hint: The apply you already know, with the inner directory name after it: sudo salt 'minion1' state.apply nginx

There is the sentence, and it comes apart in three pieces.

No matching sls found means Salt looked for a state and did not find one. for 'nginx' is the name you typed. in env 'base' is where it looked, and that piece is the useful one. base is the default environment, and the master's directory list for base currently holds /srv/salt and nothing else.

Data failed to compile: names the stage that failed. Before a minion changes anything it fetches the states it was asked for and renders them into a plan, and that rendering is compiling. A state it cannot find is a compile failure, so the run stops and nothing is attempted on the machine.

ERROR: Minions returned with non-zero exit code on the first line is the headline, not the diagnosis. It sits above many different faults, and the useful line is always further down.

Nothing is wrong with your files. The master has simply never been told the directory exists.

Tell the master where formulas live

The master reads every .conf file in /etc/salt/master.d/. That is a drop-in directory: instead of one enormous config file, you add small files and the master merges them at startup. An empty one called formulas.conf is waiting there, owned by your account, so this write is a plain heredoc like the others.

Read both paths carefully.

cat > /etc/salt/master.d/formulas.conf << 'EOF'
file_roots:
  base:
    - /srv/salt
    - /srv/formulas/nginx-formula
EOF

prompt: saltops@saltmaster:~$ answer: cat /etc/salt/master.d/formulas.conf ||| sudo cat /etc/salt/master.d/formulas.conf output: file_roots: base:

/srv/salt
/srv/formulas/nginx-formula

hint: Print the config file you just wrote: cat /etc/salt/master.d/formulas.conf

Four lines, and two of them are decisions.

The path you listed is /srv/formulas/nginx-formula, the outer directory. A file root is a directory whose contents Salt can address by name, so pointing at the outer one makes nginx a name inside it. Point at the inner one by mistake and the state name becomes init.

/srv/salt is listed first because it has to be. Each environment name under file_roots holds a list of directories, searched in order. That list is one setting with one value, so writing it again here does not add to what lab.conf said. It replaces it. Leave /srv/salt out and states that worked an hour ago stop resolving.

Adding the formula path to file_roots is the second of the five things your lab grades.

The master reads its configuration once, at startup, and holds the result in memory. Editing a drop-in changes the file, not the running process. Restart it, and remember that a drop-in with broken YAML is a master that does not come back.

Restart it now. Like most systemctl verbs, it prints nothing when it works.

sudo systemctl restart salt-master

Give it a few seconds before asking it to do anything. A minion whose master goes away does not fail: it keeps trying and reconnects on its own.

Apply the same command that just failed

Nothing about your formula has changed. Not one byte of init.sls, not one byte of map.jinja. The only thing that moved is a list in the master's head.

Run the identical command again, character for character.

sudo salt 'minion1' state.apply nginx

Before you press Enter, commit to two things. How many state blocks should come back from a file holding two states? And what should Changes: show for a package that is not on the machine yet?

prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' state.apply nginx ||| sudo salt "minion1" state.apply nginx ||| sudo salt minion1 state.apply nginx output: minion1: ---------- ID: nginx_package Function: pkg.installed Name: nginx Result: True Comment: The following packages were installed/updated: nginx Started: 22:58:06.169842 Duration: 9907.14 ms Changes: ---------- nginx: ---------- new: 1.22.1-9+deb12u9 old: nginx-common: ---------- new: 1.22.1-9+deb12u9 old: ---------- ID: nginx_lab_conf Function: file.managed Name: /etc/nginx/conf.d/lab.conf Result: True Comment: File /etc/nginx/conf.d/lab.conf updated Started: 22:58:16.079166 Duration: 4.823 ms Changes: ---------- diff: New file

Summary for minion1 ------------ Succeeded: 2 (changed=2) Failed: 0 ------------ Total states run: 2 hint: The same apply that failed a moment ago, unchanged: sudo salt 'minion1' state.apply nginx

Two blocks, one per state in your file, and this is the screen the whole lesson was walking toward.

The first block is pkg.installed. Read the line Name: nginx and notice where it came from: you never wrote that word in init.sls. You wrote {{ nginx.pkg }}, and the map turned it into nginx. Changes: lists nginx and nginx-common, each with a new: version and an empty old:, Salt's way of saying these were not here before. Duration: 9907.14 ms is the package install, not Salt being slow.

The second block is file.managed, and Name: /etc/nginx/conf.d/lab.conf came out of the same map, from `{{ nginx.conf

Practice Formulas in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.