Learn › Salt Configuration Management › Jinja and Advanced States
Control state ordering with require, watch, and onchanges requisites.
There is a state on this master that installs nginx, writes its configuration, and keeps the service running. Apply it, change the configuration, apply it again, and you get a machine with the new config file on disk and the old config still loaded in memory.
Nothing errors. Nothing is skipped. Every state reports success. Salt did exactly what the file told it to do, and the file forgot to mention that the service should care.
What is missing is one word. In this lesson you will run the same apply three times without changing a character of the command, and watch that one word turn a state that does nothing into a state that acts.
The black boxes below are a practice terminal: a safe sandbox that checks the one command each step teaches. Every output printed below was captured from this lab's own master with one real minion attached, which is exactly the machine your lab boots. The lab user is saltops, the master is named saltmaster, and there is exactly one minion, named minion1.
Start by reading the master's file root. /srv/salt is the directory this master publishes to its minions, and it is where every state file in this track has lived.
ls -l gives the long listing: permissions, owner, group, size, time and name. Pass it two paths and it labels each one with a heading before its contents.
Before you run it, commit to one thing. You are about to write files into this directory. Will you need sudo to do it?
ls -l /srv/salt /srv/salt/files
prompt: saltops@saltmaster:~$ answer: ls -l /srv/salt /srv/salt/files ||| ls -l /srv/salt/ /srv/salt/files/ ||| ls -l /srv/salt /srv/salt/files/ output: /srv/salt: total 16 drwxr-xr-x 2 saltops saltops 4096 Jul 31 19:01 files -rw-r--r-- 1 saltops saltops 29 Jul 31 19:03 top.sls drwxr-xr-x 3 saltops saltops 4096 Jul 31 19:03 webserver -rw-r--r-- 1 saltops saltops 884 Jul 31 19:03 webserver.sls
/srv/salt/files: total 0 hint: Long listing, and you can hand ls more than one path at a time: ls -l /srv/salt /srv/salt/files
Read the owner column, not the file names. Every entry says saltops saltops, and saltops is you. You own this tree, so you write state files here with an ordinary redirect and no sudo at all.
That is worth fixing in your head now, because it is the one place in this track where sudo is wrong. Talking to Salt needs root, so salt and salt-key take sudo. Editing files you already own does not.
The rest of the listing is the work order. top.sls is 29 bytes, which is the three lines that map webserver onto every minion. webserver.sls is 884 bytes and holds five states. files is empty, and webserver is a directory holding the nginx config this master ships.
Salt reads a state file top to bottom and, by default, runs the states in the order you wrote them. That is a convenience. It is not a promise.
A requisite is a line inside one state that names another state and describes the relationship between them. It turns file order into a declared dependency: this state waits for that one, or this state reacts to that one.
Three requisites cover almost everything you will ever write, and the difference between them is one question each.
require asks: did that state succeed? If yes, run. If no, do not run at all.watch asks: did that state change anything? Everything require does, plus an extra action when the answer is yes.onchanges asks the same change question, but does nothing else. No change, no work.Every requisite is written the same way: the module name, a colon, and the ID of the state you are pointing at. - pkg: nginx means the state whose ID is nginx and whose module is pkg. You are naming a specific block in a specific file, so both halves have to match what you wrote.
Early Salt did not run states in file order at all. A state file was parsed into a dictionary and executed in whatever order that dictionary handed the keys over, which meant the order could change between runs of the same file.
That sounds like a bug and it was a design position: if order matters, say so. Requisites were not a convenience, they were the only way to get an install before a config, ever.
Automatic ordering that follows the file arrived later, under a master setting called state_auto_order, and it has been on by default for well over a decade. It is why the state files in the last few lessons worked without a single requisite in them.
The old behaviour is why every serious state file still declares its edges. File order is what happens when nobody says otherwise. A requisite is a contract that survives someone reordering the file, splitting it in two, or pulling half of it in from an include. Real state trees get rearranged constantly, and the states that survive that are the ones that wrote down what they depend on.
This is /srv/salt/webserver.sls as it ships, all 884 bytes of it. Read it, do not type it:
# Webserver state: install, configure, and run nginx.
# PROBLEM: These states have no ordering or dependency declarations.
# TODO: Add require so config deploys after nginx is installed.
# TODO: Add watch so nginx restarts when the config file changes.
# TODO: Add onchanges so the log entry only runs when something changes.
install_nginx:
pkg.installed:
- name: nginx
deploy_site_config:
file.managed:
- name: /etc/nginx/sites-available/lab-site.conf
- source: salt://webserver/files/nginx-site.conf
- makedirs: True
enable_site_link:
file.symlink:
- name: /etc/nginx/sites-enabled/lab-site.conf
- target: /etc/nginx/sites-available/lab-site.conf
run_nginx:
service.running:
- name: nginx
- enable: True
log_deploy:
cmd.run:
- name: echo "Site deployed at $(date)" >> /var/log/deploy.log
- creates: /var/log/deploy-initial.logFive states and not one edge between them. The three TODO lines are the three requisites this lesson teaches, and they are also three of the five things your lab grades.
In the practice terminal you are going to build the same pattern in miniature at the same path, so that every line of output on screen is about one requisite and nothing else. In the lab you will do it the other way round: keep these five states and add the missing edges to them.
Three states, three IDs, two edges. Write it into the file root with a heredoc:
cat > /srv/salt/webserver.sls << 'EOF'
nginx:
pkg.installed: []
/etc/nginx/conf.d/lab.conf:
file.managed:
- contents: |
# lab config v1
- require:
- pkg: nginx
nginx_service:
service.running:
- name: nginx
- watch:
- file: /etc/nginx/conf.d/lab.conf
EOFFour things to read in it.
nginx: with pkg.installed: [] is a state whose ID is nginx. The empty brackets are YAML for an empty list, which here means no options: the ID is already the package name, so there is nothing left to say.- contents: | puts the file's content inline instead of fetching it from the file root. The | keeps the following indented lines as literal text. One line of content, # lab config v1.- require: then - pkg: nginx is the first edge. The config file waits for the package state to succeed.- watch: then - file: /etc/nginx/conf.d/lab.conf is the second edge. The service watches the file state. Note the ID it names: this state used its path as its ID, so the path is what you point at.Indentation decides whether a requisite exists. require: and watch: sit at four spaces, level with - name: and - contents:. The states they point at sit at six spaces with their own dash. Put them one level too deep or too shallow and Salt either throws a YAML error or, worse, accepts the file and silently ignores the edge you thought you declared.
answer: - pkg: nginx ||| pkg: nginx hint: Module name, colon, then the ID of the state you are pointing at. The package state's ID is nginx.
Same three part shape as every Salt command: sudo salt, the target in quotes, then the work. The work is state.apply followed by the state file's name with no .sls on the end.
Before you run it, commit to an order. Three states, three blocks of output. Which block prints first?
sudo salt 'minion1' state.apply webserver
prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' state.apply webserver ||| sudo salt "minion1" state.apply webserver ||| sudo salt minion1 state.apply webserver ||| sudo salt 'minion1' state.sls webserver ||| sudo salt '*' state.apply webserver output: minion1: ---------- ID: nginx Function: pkg.installed Result: True Comment: The following packages were installed/updated: nginx Started: 19:03:57.377396 Duration: 9912.975 ms Changes: ---------- nginx: ---------- new: 1.22.1-9+deb12u9 old: nginx-common: ---------- new: 1.22.1-9+deb12u9 old: ---------- ID: /etc/nginx/conf.d/lab.conf Function: file.managed Result: True Comment: File /etc/nginx/conf.d/lab.conf updated Started: 19:04:07.293656 Duration: 6.773 ms Changes: ---------- diff: New file ---------- ID: nginx_service Function: service.running Name: nginx Result: True Comment: Service started Started: 19:04:07.316263 Duration: 1029.32 ms Changes: hint: Target first, then the function, then the state file name with no .sls: sudo salt 'minion1' state.apply webserver
Package, then file, then service. Read the Started: clocks and you can see the edge doing its work. The package state started at 19:03:57 and ran for 9912.975 ms. The file state did not start until 19:04:07, right after the package finished. The config file was never going to be written into a directory that nginx had not created yet.
Then read the three Comment: lines, because they are the sentences you will scan first for the rest of your career.
The following packages were installed/updated: nginx, with the versions that arrived listed under Changes:. Two packages, nginx and nginx-common, because installing one pulled in the other.File /etc/nginx/conf.d/lab.conf updated, with diff: New file. Salt prints a line by line diff when it edits an existing file. There was nothing to compare against here, so it says so in three words.Service started, with an empty Changes: block.That last line is one of the five things your lab grades: the state applied and nginx running. This capture stops there, so the totals rows that normally close a run are simply not shown.
Nothing on the minion has moved since that run finished. You press the up arrow and send the identical command a second time.
>>> Nothing. A watch is not a schedule, it is a reaction, and it needs something to react to. Salt runs the file state, finds the file already matches what you asked for, and records no changes. With no changes recorded, the watch never triggers. If you picked the restart, you have just described the most expensive misunderstanding in configuration management. A requisite that fired on every run would bounce every service on every machine, every time anything applied a state. That is an outage you scheduled yourself. If you picked the skip, Salt keeps no memory of past runs on the minion. It re-reads the machine in front of it every single time, which is exactly why running the same state twice is safe.
Nothing changes about what you type. Watch the Comment: lines and the Duration: numbers, because that is where the whole difference lands.
sudo salt 'minion1' state.apply webserver
prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' state.apply webserver ||| sudo salt "minion1" state.apply webserver ||| sudo salt minion1 state.apply webserver ||| sudo salt 'minion1' state.sls webserver ||| sudo salt '*' state.apply webserver output: minion1: ---------- ID: nginx Function: pkg.installed Result: True Comment: All specified packages are already installed Started: 19:04:09.354909 Duration: 17.748 ms Changes: ---------- ID: /etc/nginx/conf.d/lab.conf Function: file.managed Result: True Comment: File /etc/nginx/conf.d/lab.conf is in the correct state Started: 19:04:09.374646 Duration: 4.378 ms Changes: ---------- ID: nginx_service Function: service.running Name: nginx Result: True Comment: Running in OFFLINE mode. Nothing to do Started: 19:04:09.379222 Duration: 7.687 ms Changes:
Summary for minion1 ------------ Succeeded: 3 hint: The same apply as before, character for character: sudo salt 'minion1' state.apply webserver
Three states, three Result: True, and every single Changes: block is empty. Put this screen next to the last one and read only the parts that moved.
The following packages were installed/updated: nginx became All specified packages are already installed. 9912.975 ms became 17.748 ms.File /etc/nginx/conf.d/lab.conf updated became File /etc/nginx/conf.d/lab.conf is in the correct state.Service started to Running in OFFLINE mode. Nothing to do.That last comment is this fixture telling you something true about itself. OFFLINE mode is what the service module says when there is no running init system for it to talk to, and your minion is a container. Remember the shape of that line. On the next run the same state behaves completely differently, and the only thing that will have changed is the file underneath it.
Succeeded: 3 with no bracket after it. On a run where something changes, Salt adds a (changed=N) count in brackets. Nothing changed here, so there is nothing to count. Empty Changes: is the healthy outcome of a second run, and it is also the exact signal every requisite in your file is reading.
Stop and count what you can already do. There is a state in the file root with two declared edges. You applied it and read the clocks to prove the order was not luck. Then you applied the identical file again and read three comment lines to prove the machine was already right.
That is the working half of the job, and most people never get past it. Everything left in this lesson is about the other half: making a state notice that something moved.
The service watches the file state, and the file state carries its content inline. So the way to change the watched file is to change the state that manages it.
Rewrite the same file with one character different, v1 becoming v2:
cat > /srv/salt/webserver.sls << 'EOF'
nginx:
pkg.installed: []
/etc/nginx/conf.d/lab.conf:
file.managed:
- contents: |
# lab config v2
- require:
- pkg: nginx
nginx_service:
service.running:
- name: nginx
- watch:
- file: /etc/nginx/conf.d/lab.conf
EOFNothing else moved. Same three IDs, same two requisites, same package. One comment line inside the managed content is now different from what sits on the minion.
Before you apply it, commit to two things. How many of the three states report a change this time, and what does the service state say?
sudo salt 'minion1' state.apply webserver
prompt: saltops@saltmaster:~$ answer: sudo salt 'minion1' state.apply webserver ||| sudo salt "minion1" state.apply webserver ||| sudo salt minion1 state.apply webserver ||| sudo salt 'minion1' state.sls webserver ||| sudo salt '*' state.apply webserver output: ERROR: Minions returned with non-zero exit code minion1: ---------- ID: nginx Function: pkg.installed Result: True Comment: All specified packages are already installed Started: 19:04:10.444555 Duration: 24.132 ms Changes: ---------- ID: /etc/nginx/conf.d/lab.conf Function: file.managed Result: True Comment: File /etc/nginx/conf.d/lab.conf updated Started: 19:04:10.470727 Duration: 7.27 ms Changes: ---------- diff: --- +++ @@ -1 +1 @@ -# lab config v1 +# lab config v2 ---------- ID: nginx_service Function: service.running Name: nginx Result: False Comment: Failed to restart the service Started: 19:04:10.485889 Duration: 16032.434 ms Changes: ---------- nginx: False
Summary for minion1 ------------ hint: Still the same apply, unchanged: sudo salt 'minion1' state.apply webserver
Read the middle block first. File /etc/nginx/conf.d/lab.conf updated, and under Changes: a real diff this time: -# lab config v1 left, +# lab config v2 arrived. The file state recorded a change, which is the one thing the watch was waiting for.
Now read the bottom block against the run before it. Same state, same ID, same file, and the machine was in OFFLINE mode. Nothing to do a second ago. This time it tried to restart nginx and spent 16032.434 ms doing it. The requisite fired. That is the whole lesson, and it is visible in one number.
The restart did not succeed: Result: False, Failed to restart the service, and nginx: False under Changes:. That is this fixture being a container with no init system, the same fact the previous screen told you in politer words. A failed state also makes the whole run non-zero, which is the ERROR: Minions returned with non-zero exit code line at the very top.
Separate the two facts, because engineers lose hours by merging them. Whether the requisite fired and whether the action succeeded are different questions with different answers. Here the answers are yes and no. Reapplying after a config edit is the fifth thing your lab grades, and the capture stops at the dashed rule under Summary for minion1.
The service state has watch pointed at the file state. Suppose you change that single word to require and leave everything else alone. Then you edit the config again and apply.
Practice Requisites in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.