Learn › Salt Configuration Management › Pillar and Data Management
Reference pillar data inside state files and Jinja templates.
By the end of this lesson there will be a configuration file on minion1 with a port number in it. You will never log in to that machine to put it there.
Here is the part worth sitting with. The state file that creates that config file will not contain the port number anywhere. Neither will the command you type. Read the state file top to bottom and the number is simply not in it.
The number lives somewhere else, in a second file, on this master. Salt joins the two together at the last possible moment. That join is the most useful thing Salt does with pillar data. It turns one state file into something you can point at a hundred machines that each need a different answer.
The black boxes below are a practice terminal: a safe sandbox that checks the one command each step teaches. Every screen in this lesson was captured by booting this exact lab and doing exactly what you are about to do, so what you type here is what you will type there. Your own timestamps and durations will be your own. The lab user is saltops, the master is named saltmaster, and there is exactly one minion, named minion1. Your progress in the lab is tracked automatically, so type commands naturally.
Salt keeps two separate directories on the master, and everything in this lesson is about the gap between them.
/srv/salt is the file root: the state files, the ones ending in .sls, that describe what a machine should look like. Every minion is allowed to fetch from here.
/srv/pillar is the pillar root: the data. Values that differ from machine to machine live here, and the master hands each minion only its own share.
Look at both in one command. ls -l gives one entry per line with permissions, owner, size and time. Hand it two paths and it prints a labelled section for each.
Before you run it, commit to a number: how many entries come back in total?
ls -l /srv/salt /srv/pillar
prompt: saltops@saltmaster:~$ answer: ls -l /srv/salt /srv/pillar ||| ls -l /srv/pillar /srv/salt output: /srv/pillar: total 8 -rw-r--r-- 1 saltops saltops 41 Jul 31 16:31 app.sls -rw-r--r-- 1 saltops saltops 23 Jul 31 16:31 top.sls
/srv/salt: total 12 drwxr-xr-x 2 saltops saltops 4096 Jul 31 16:31 files -rw-r--r-- 1 saltops saltops 26 Jul 31 16:31 top.sls -rw-r--r-- 1 saltops saltops 99 Jul 31 16:31 webapp.sls hint: One long listing, two paths on the same line: ls -l /srv/salt /srv/pillar
Five entries across two labelled sections. ls printed /srv/pillar first because it sorts the paths you gave it, not because pillar comes first in any Salt sense.
Read the owner column on every line: saltops saltops, which is your account. The setup handed both trees to you before you logged in, and that is what makes the writing you are about to do a plain heredoc with no sudo in front of it.
Two files are worth naming. /srv/pillar/app.sls already exists, all 41 bytes of it, and you are going to replace it in a moment with the keys this lesson needs. /srv/salt/top.sls exists too, which means this master already has an assignment sheet. Leave both top.sls files and webapp.sls alone today. You will add one new state file of your own and apply that one by name.
You met pillar in the last lesson as a place to keep per-machine data on the master. On its own it does nothing. It is a filing cabinet with no hands.
A state file gets the hands. Instead of writing port=8080 into a state, you leave a labelled blank where the number should go:
port={{ pillar['app_port'] }}
That doubled curly brace is Jinja, a templating language. Think of it as a mail merge. You write the letter once with blanks in it, and the system fills each blank in with the right value for each recipient. Here the recipients are machines and the values come from pillar.
Three Jinja forms exist and you only need the first one today:
{{ value }} outputs a value into the text.{% logic %} is control flow, if and else and loops. Later module.{# comment #} is a note that disappears from the output.The payoff is reuse. One state file that reads {{ pillar['app_port'] }} serves a machine that needs 8080 and a machine that needs 9090, without a single edit. The answer was never in the state file to begin with.
Jinja was written by Armin Ronacher in 2008 for Python web applications, years before Salt existed. It had nothing to do with servers. It filled in HTML pages.
Salt adopted it wholesale rather than inventing a template language of its own, and made it the default renderer for every .sls file. That is why the syntax in a Salt state looks exactly like the syntax in a Python web template. It is the same tool.
Renderer is a real term with a precise meaning here. Every .sls file goes through two passes by default: Jinja first, which fills in the blanks and produces plain text, then YAML, which reads that text as structure. This matters when you are debugging. If Jinja cannot fill a blank, YAML never runs and nothing is applied at all, which is a different kind of failure from a state that ran and did not like what it found.
The state file with the blank in it sits on this master. The config file with the number in it ends up on the minion. Somewhere between those two facts, something replaced {{ pillar['app_port'] }} with a number. Take a position on where and by whom.
>>> The minion did it. The master compiles a pillar for each minion and hands that minion its own share, and the minion keeps that copy. When a state runs, the minion fetches the .sls from the master, renders the Jinja locally against its own pillar copy, then acts. If you picked the first answer, that is the intuitive one and it is worth correcting early, because the whole failure mode in the next step depends on the minion holding a copy. If you picked the third, note that /srv/pillar exists only on the master and minions never read it, ever. That directory is not shared, not mounted, and not fetchable. A minion can only ever see what the master decided to compile for it.
Your state is going to ask pillar for two things: a name and a port. So put them there first.
A heredoc is the shell's way of typing several lines into a file in one go. Everything between the << 'EOF' marker and the closing EOF becomes the content of the file. /srv/pillar belongs to your account, so there is no sudo here:
cat > /srv/pillar/app.sls << 'EOF'
app_name: linuxcamp
app_port: 8080
EOFTwo lines, and both are ordinary YAML: a key, a colon, a space, a value. app_name and app_port are names you chose. Nothing in Salt knows or cares what they mean. They are labels on a drawer, and the only rule is that the state file has to ask for the same labels you wrote here.
That > overwrites the file that was already there. That is deliberate. You are replacing the setup's version with the keys this lesson needs.
A missing space after the colon is the single most common pillar mistake. app_port: 8080 is a key and a value. app_port:8080 is one long piece of text with a colon in the middle, and YAML will not read it as a key at all. The state that asks for app_port then finds nothing.
The pillar file on the master now has app_name in it. minion1 has been running this whole time and has not been told anything.
>>> It fails to render. The minion still holds the pillar it was given earlier, and Jinja cannot fill a blank from a key that copy does not contain. Rendering happens before YAML and before any state runs, so nothing at all is applied. You do not get a half-configured machine. You get a rendering error naming the file, with a pointer made of <====== marking the exact line Jinja choked on. If you picked the first answer, re-read the previous step: the file on the master's disk is not what the minion reads from. If you picked the third, a quiet empty value is something you have to ask for on purpose, using a different form that this lesson comes back to at the end. A bare pillar['key'] that is missing is loud, and being loud is the correct behaviour for a config file that would otherwise ship broken.
The fix is one command, and it is a normal part of working with pillar rather than a repair. saltutil is the module for housekeeping on minions, and refresh_pillar tells a minion to throw away its cached pillar and ask the master for a fresh one.
It is the same three-part Salt shape you already know: sudo salt, a quoted target, then module.function.
Before you run it, decide what a healthy answer looks like. This function reports whether the refresh was accepted, not what came back in it.
sudo salt 'minion1' saltutil.refresh_pillar
prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' saltutil.refresh_pillar ||| sudo salt "minion1" saltutil.refresh_pillar ||| sudo salt minion1 saltutil.refresh_pillar ||| sudo salt '*' saltutil.refresh_pillar ||| sudo salt "*" saltutil.refresh_pillar output: minion1: True hint: The housekeeping module is saltutil and the function names what it does: sudo salt 'minion1' saltutil.refresh_pillar
True, in the standard Salt return shape: the minion ID flush left with a colon, then its result indented four spaces underneath.
Read that True narrowly. It means the minion accepted the instruction and went to refetch. It does not tell you what keys arrived, and it would say the same thing if you had made a typo in the pillar file. Proving the values are actually there is a separate job, and it is the last thing you do in this lesson.
Get in the habit now: every time you edit anything under /srv/pillar, refresh before you apply. Experienced Salt hands type these two commands as a pair without thinking about it.
The pillar file you just wrote is the first thing your lab grades.
If a state still cannot see a key one second after a refresh, refresh once more and try again. The minion has to fetch and rebuild its pillar, and on a busy box that is not instant.
Now the other half. This state creates a config file on the minion and fills it from pillar.
Write it into the file root as appconf.sls. Same heredoc, same lack of sudo, and the quotes around 'EOF' matter here: they stop the shell from touching the curly braces before the file is written.
cat > /srv/salt/appconf.sls << 'EOF'
/etc/app.conf:
file.managed:
- contents: |
name={{ pillar['app_name'] }}
port={{ pillar['app_port'] }}
EOFThe heredoc prints nothing when it works. Silence is success, which is unnerving the first time, so read the file back and check your own typing before you go anywhere near the minion.
cat /srv/salt/appconf.sls
prompt: saltops@saltmaster:~$ answer: cat /srv/salt/appconf.sls output: /etc/app.conf: file.managed:
name={{ pillar['app_name'] }} port={{ pillar['app_port'] }} hint: Print your new file with cat and its full path: cat /srv/salt/appconf.sls
Five lines, and every one of them is doing a job. Read it as a staircase, because in YAML the depth is the meaning.
/etc/app.conf: at column 0 is the state ID, the label for this piece of work. Most state IDs are free-text names you invent. file.managed is one of the places where the ID does double duty: when you do not spell out a - name: line, Salt uses the ID as the path to manage. So this ID is also the destination on the minion.
file.managed at 2 spaces is the state function, the same one you used in the file states lesson. It means: this file should exist, with this content.
- contents: | at 4 spaces is the content, given inline. That trailing pipe character is YAML for a block of literal text, and it says everything indented beneath me is the text, keep the line breaks.
The two lines at 8 spaces are that text. Each one is half literal and half blank: name= and port= come out as typed, and each {{ pillar['...'] }} is a hole waiting for a value. Look at the whole file again and confirm the thing promised at the start of this lesson. There is no port number in it.
A state that references pillar values is the second thing your lab grades, and those two pillar[...] expressions are exactly what it looks for.
Mind the quotes inside the braces. {{ pillar['app_name'] }} needs the key wrapped in quotes because it is a lookup by name, the same way it would be in Python. {{ pillar[app_name] }} without them asks for a variable called app_name that does not exist, and the render fails. Spaces just inside the braces are optional, and every convention keeps them because they are easier to read.
The command that runs this is sudo salt 'minion1' state.apply appconf. That last word is the file you just wrote, minus its .sls extension. Salt looks for it in the file root.
There is a trap next to it, and this master is set up in a way that makes the trap real.
>>> It runs a highstate. state.apply with no argument is the same thing as state.highstate, so the minion reads the top file and applies whatever that file has assigned to it. You saw a top.sls in the listing at the start of this lesson, so that is not a theoretical risk on this box. A bare apply here would do work you did not ask for. If you picked the first answer, a refusal would honestly be the safer design, and plenty of people have wished for it after a bare apply on production. If you picked the third, Salt never applies state files just because they are sitting in the file root. A state file is inert until something names it, either you on the command line or the top file.
Everything is in place. Pillar has the values, the minion has a fresh copy of them, and the state has the blanks. Name the state on the command line so you apply exactly this one and nothing else.
Expect a single block back, because your state file holds a single state ID.
sudo salt 'minion1' state.apply appconf
prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' state.apply appconf ||| sudo salt "minion1" state.apply appconf ||| sudo salt minion1 state.apply appconf output: minion1: ---------- ID: /etc/app.conf Function: file.managed Result: True Comment: File /etc/app.conf updated Started: 16:34:21.811240 Duration: 4.297 ms Changes: ---------- diff: New file
Summary for minion1 ------------ Succeeded: 1 (changed=1) Failed: 0 ------------ Total states run: 1 Total run time: 4.297 ms hint: Same three-part shape, then the state file name without its extension: sudo salt 'minion1' state.apply appconf
It rendered and it ran. Result: True and Failed: 0 are the two words to find first, and both say the same thing: the condition you described is now true on that machine.
ID: /etc/app.conf echoes back the label from your file, which is also the path that got written. Comment: File /etc/app.conf updated is Salt in its own words. Total states run: 1 matches the one state ID you wrote, because Salt counts state IDs, not files.
Getting this state applied is the third thing your lab grades.
That screen is worth a second pass, because the interesting thing about it is a negative.
Result is the verdict. Changes is the receipt. The verdict says the condition holds. The receipt says what Salt actually had to move to get there, and here the receipt reads diff: then New file. Not a line-by-line difference, just the fact that nothing was there before. Run this against a machine that already had the file and the receipt is where you would see which lines changed.
Duration: 4.297 ms is worth a glance too. Four milliseconds. Writing a small file is nothing like installing a package, which takes seconds, and that difference is why a scheduled state run across a whole fleet is cheap when almost everything is already correct.
Now the negative. Scan every line of that output and look for the port number. It is not there. 8080 appears nowhere in the apply output, because the blank was filled in on the minion, inside the file, and Salt reported that a file was written rather than what went into it.
So Result: True is not proof that the substitution worked. It is proof that a file arrived. The only place the answer exists is on the minion, and going to look is the next thing you do.
Count what changed. You started with two directories the setup handed you and no connection between them. You now have a value in the pillar root, a state in the file root that refuses to hardcode that value, and a machine that took both and produced a finished config file.
That separation is the whole discipline. Instructions in /srv/salt, answers in /srv/pillar, joined at apply time and never before. It is why a Salt tree can go into version control where the whole team reads it. The values that differ per machine, or that nobody should read casually, stay in a directory with tighter permissions.
Everything else in this module is a variation on what you just did: pointing the same state at more machines, giving different machines different answers, and locking down who can see which answer. The join you just built is the mechanism under all of it.
Scaffolding off. No command is printed from here on.
Salt told you it wrote /etc/app.conf on minion1. Trust it and verify it anyway, because being able to ask a machine directly is the entire reason you built this. Do not log in. Reach for the blunt escape hatch from the modules lesson, the one that hands a raw shell command to a machine and returns whatever it printed. Pair it with the plainest way there is to print a file.
prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' cmd.run 'cat /etc/app.conf' ||| sudo salt "minion1" cmd.run "cat /etc/app.conf" ||| sudo salt 'minion1' cmd.run "cat /etc/app.conf" ||| sudo sal
Practice Pillar in States in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.