Deploying Saltstack : master and minion (archlinux on ARM)

Posted on September 23, 2016

As part of a project to upgrade my servers (Sheevaplug and Cubieboards), I wanted to automate most of the OS and software deployments.

Moreover the idea was also to factorize configurations for all servers and be able to change and replay deployments easily… That’s exactly the purpose of an orchestrator like Puppets, Ansible or Saltstack.

I choosed Saltstack as it seemed the most simple to use to me without any compromise on functionalities (regarding my needs).

Below are my notes about this first install, as well as a little script to to even automate the minions deployment.

Base install

Both Archlinux and Saltstack projects provide good documentation to start:
https://wiki.archlinux.org/index.php/Saltstack
https://docs.saltstack.com/en/latest/ref/configuration/index.html

Unfortunately the “raet” package was not working correctly on arm arch when I tried: it was taking 100% of cpu. So I installed both on Master and Minion the “zmq” version:

pacman -S salt-zmq

1	pacman -S salt-zmq

Next step : create a dns entry for the master, eg: “salt.local.lan” , so it is easier to change the host without impact on minion’s config file.

I also did the same for all minions, eg: “minion.local.lan”.

Saltstack use 2 TCP ports that must be opened for master and minon if they are on different subnets or vlan protected by a firewall.

Example for an iptables rules file :

-A INPUT -p tcp -i vlan1 --dport 4505:4506 -j ACCEPT
-A INPUT -p tcp -i vlan2 --dport 4505:4506 -j ACCEPT

1 2	-A INPUT -p tcp -i vlan1 --dport 4505:4506 -j ACCEPT -A INPUT -p tcp -i vlan2 --dport 4505:4506 -j ACCEPT

Enable communication between master and minion

To make the master deals with the minion, it is needed to :

Tell the minion who is it’s master (darth Sidious ?)
Make the master trusts the minion

Setting the minion’s master requires to set the “master” property in “/etc/salt/minion”:

master: salt.local.lan

1	master: salt.local.lan

Then the minion must have the master’s public key fingerprint in “/etc/etc/salt/minion” (the property is named “master_finger”). To get the master key (Saltstack must be running) :

sudo systemctl start salt-master
salt-key -F master

1 2	sudo systemctl start salt-master salt-key -F master

As soon as the key is set in the minion’s config file, saltstack can be started on the minion:

sudo systemctl start salt-minion

1	sudo systemctl start salt-minion

The last step to establish the communication is to allow the minion on the master side:

sudo salt-key -a minion.local.lan

1	sudo salt-key -a minion.local.lan

then test with a “ping” :

sudo salt '*' test.ping

1	sudo salt '*' test.ping

Final Adjustments

Everything works, so we can start Saltstack at boot time, on the master:

sudo systemctl enable salt-master

1	sudo systemctl enable salt-master

And on the minion:

sudo systemctl enable salt-minion

1	sudo systemctl enable salt-minion

Configure log rotation properly (logrotate) with the following in a new file /etc/logrotate.d/salt

/var/log/salt/key /var/log/salt/master /var/log/salt/minion {
 nocompress
 missingok
 postrotate
 /bin/kill -HUP `cat /var/run/salt/salt-master.pid 2&gt;/dev/null` 2&gt; /dev/null || true
 endscript
}

/var/log/salt/key /var/log/salt/master /var/log/salt/minion {

nocompress

missingok

postrotate

/bin/kill -HUP `cat /var/run/salt/salt-master.pid 2>/dev/null` 2> /dev/null || true

endscript

}

As some of my minions as well as my master run on low power arm computer (eg: raspberry or odroid c1), I changed the master configuration in /etc/salt/master to set the “timeout” property to 120 and the “worker_thread” to 8.

This allows to be more tolerant for slow answer as the master can wait up to 2 minutes for 8 command’s results.

At this time, salt is able to drive minions… but the true power of SaltStack is the templating of services: the states…

Configure States

Prepare folder architecture

States are configuration templates that are OS agnostic. Saltstack propose a directory and filename based logic to handle the whole thing.

There are basicaly 3 main directories to configure in “/etc/salt/master”:

file_roots : for states path
module_dirs: for modules (to create custom state’s actions)
pillar_roots: pillars (Salstack way of defining and assigning values to minions)

Example of structure :

/srv/salt/states/base
/srv/salt/pillar/base
/srv/salt/modules

/srv/salt/states/base

/srv/salt/pillar/base

/srv/salt/modules

A simple state

As an example, here is a state that remove “alarm” user and group on am Archlinuxarm distrib.

Create the folder “noalarmuser” in /srv/salt/states/base, then a file named “init.sls” with the following content:

alarm:
  user.absent:
    - purge: True

remove_alarm_group:
  group.absent:
   - name: alarm

alarm:

user.absent:

- purge: True

remove_alarm_group:

group.absent:

- name: alarm

To apply this state to a minion, it is as simple as :

salt 'minion.local.lan' state.apply noalarmuser

1	salt 'minion.local.lan' state.apply noalarmuser

There are 4 kind of information to build a state :

The state name, here defined as a the folder name “noalarmuser” which contains a default definition file “init.sls”
State actions identifiers : “alarm” and “remove_alarm_group”. They have to be unique.
Modules and functions to execute: “user.absent” and “group.absent”
Function’s parameters : “purge: True” and “name: alarm”

A majority (if not all ?) module’s function use a first parameter “name” which default value is taken from the action identifier.

For example, the first state action could have defined as :

any_unique_action_identifier:
  user.absent:
    - name: alarm
    - purge: True

any_unique_action_identifier:

user.absent:

- name: alarm

- purge: True

Assign states to minons

Saltstack allows to apply a state to one or several minions, but also allows to define which state should be applied to which minion.

Assignation of states to minion is done by defining a file name “top.sls” in the “base” directory. Example in “/srv/salt/states/base/top.sls”:

base: 
  '*.local.lan': 
    - noalarmuser

base:

'*.local.lan':

- noalarmuser

This allows to remove alarm user and groups to all minions which name ends with “.local.lan”.

To apply states that are set in the top.sls file to a specific minion:

salt 'minion.local.lan' state.apply

1	salt 'minion.local.lan' state.apply

Script to automate Salstack deployment on minions

I’m now using the following script (which is also attached to this article) to install and configure saltstack on any new Archlinux based machine to transform it into a minion.

To use it, just replace “xx:xx:xx:xx:xx:xx:xx:xx” by your master.pub key finger print:

#!/bin/bash 
 
if [[ $EUID -ne 0 ]]; then 
   echo "This script must be run as root" 
   exit 1 
fi  
 
# install salt and set master in config file 
pacman --noconfirm -Sy salt-zmq 
sed -i -e"s/^#master\s*:\s*salt/master: salt.local.lan/" /etc/salt/minion 
sed -i -e"s/^#master_finger\s*:\s*\'\'/master_finger: \''xx:xx:xx:xx:xx:xx:xx'\'/" /etc/salt/minion 
 
# add local domain to minion_id 
echo -n "$HOSTNAME.local.lan" &gt;&gt; /etc/salt/minion_id 

 
# add logrotate config 
cat &lt;&lt;EOL &gt;&gt; /etc/logrotate.d/salt 
/var/log/salt/key /var/log/salt/master /var/log/salt/minion  { 
        nocompress 
        missingok 
        postrotate 
                /bin/kill -HUP `cat /var/run/salt/salt-minion.pid 2&gt;/dev/null` 2&gt; /dev/null || true 
        endscript 
} 
EOL 
 
systemctl restart logrotate

# show minion id 
MINION_ID=`cat /etc/salt/minion_id` 
echo "" 
echo "minion_id set to : $MINION_ID" 
echo "   =&gt; check that this hostname is declared in local dns" 
echo "" 
  
# enable &amp; start service 
systemctl enable salt-minion 
systemctl start salt-minion 
sleep 2 
 
# final word 
echo "" 
ps ax | grep salt 
echo "" 
echo "" 
echo "type the following on the master to authorize this minion:" 
echo "salt-key -a $MINION_ID"

#!/bin/bash

if [[ $EUID -ne 0 ]]; then

echo "This script must be run as root"

exit 1

# install salt and set master in config file

pacman --noconfirm -Sy salt-zmq

sed -i -e"s/^#master\s*:\s*salt/master: salt.local.lan/" /etc/salt/minion

sed -i -e"s/^#master_finger\s*:\s*\'\'/master_finger: \''xx:xx:xx:xx:xx:xx:xx'\'/" /etc/salt/minion

# add local domain to minion_id

echo -n "$HOSTNAME.local.lan" >> /etc/salt/minion_id

# add logrotate config

cat <<EOL >> /etc/logrotate.d/salt

/var/log/salt/key /var/log/salt/master /var/log/salt/minion {

nocompress

missingok

postrotate

/bin/kill -HUP `cat /var/run/salt/salt-minion.pid 2>/dev/null` 2> /dev/null || true

endscript

}

EOL

systemctl restart logrotate

# show minion id

MINION_ID=`cat /etc/salt/minion_id`

echo ""

echo "minion_id set to : $MINION_ID"

echo " => check that this hostname is declared in local dns"

echo ""

# enable & start service

systemctl enable salt-minion

systemctl start salt-minion

sleep 2

# final word

echo ""

ps ax | grep salt

echo ""

echo "type the following on the master to authorize this minion:"

echo "salt-key -a $MINION_ID"

Download the script : salt-minion-install

Linux archlinux, arm, linux, orchestrator, saltstack

Deploying Saltstack : master and minion (archlinux on ARM)

Base install

Enable communication between master and minion

Final Adjustments

Configure States

Prepare folder architecture

A simple state

Assign states to minons

Script to automate Salstack deployment on minions

3 comments

Leave a Reply Cancel reply