Ubuntu Manpage: salt - Salt Documentation

Provided by: salt-common_2015.8.8+ds-1ubuntu0.1_all

NAME

       salt - Salt Documentation

INTRODUCTION TO SALT

We’re not just talking about NaCl..SS The 30 second summary

Salt is:

• a configuration management system, capable of maintaining remote nodes in defined states
(for example, ensuring that specific packages are installed and specific services are
running)

• a distributed remote execution system used to execute commands and query data on remote
nodes, either individually or by arbitrary selection criteria

It was developed in order to bring the best solutions found in the world of remote
execution together and make them better, faster, and more malleable. Salt accomplishes
this through its ability to handle large loads of information, and not just dozens but
hundreds and even thousands of individual servers quickly through a simple and manageable
interface.

Simplicity
Providing versatility between massive scale deployments and smaller systems may seem
daunting, but Salt is very simple to set up and maintain, regardless of the size of the
project. The architecture of Salt is designed to work with any number of servers, from a
handful of local network systems to international deployments across different data
centers. The topology is a simple server/client model with the needed functionality built
into a single set of daemons. While the default configuration will work with little to no
modification, Salt can be fine tuned to meet specific needs.

Parallel execution
The core functions of Salt:

• enable commands to remote systems to be called in parallel rather than serially

• use a secure and encrypted protocol

• use the smallest and fastest network payloads possible

• provide a simple programming interface

Salt also introduces more granular controls to the realm of remote execution, allowing
systems to be targeted not just by hostname, but also by system properties.

Building on proven technology
Salt takes advantage of a number of technologies and techniques. The networking layer is
built with the excellent ZeroMQ networking library, so the Salt daemon includes a viable
and transparent AMQ broker. Salt uses public keys for authentication with the master
daemon, then uses faster AES encryption for payload communication; authentication and
encryption are integral to Salt. Salt takes advantage of communication via msgpack,
enabling fast and light network traffic.

Python client interface
In order to allow for simple expansion, Salt execution routines can be written as plain
Python modules. The data collected from Salt executions can be sent back to the master
server, or to any arbitrary program. Salt can be called from a simple Python API, or from
the command line, so that Salt can be used to execute one-off commands as well as operate
as an integral part of a larger application.

Fast, flexible, scalable
The result is a system that can execute commands at high speed on target server groups
ranging from one to very many servers. Salt is very fast, easy to set up, amazingly
malleable and provides a single remote execution architecture that can manage the diverse
requirements of any number of servers. The Salt infrastructure brings together the best
of the remote execution world, amplifies its capabilities and expands its range, resulting
in a system that is as versatile as it is practical, suitable for any network.

Open
Salt is developed under the Apache 2.0 license, and can be used for open and proprietary
projects. Please submit your expansions back to the Salt project so that we can all
benefit together as Salt grows. Please feel free to sprinkle Salt around your systems and
let the deliciousness come forth.

Salt Community
Join the Salt!

There are many ways to participate in and communicate with the Salt community.

Salt has an active IRC channel and a mailing list.

Mailing List
Join the salt-users mailing list. It is the best place to ask questions about Salt and see
whats going on with Salt development! The Salt mailing list is hosted by Google Groups. It
is open to new members.

https://groups.google.com/forum/#!forum/salt-users

There is also a low-traffic list used to announce new releases called salt-announce

https://groups.google.com/forum/#!forum/salt-announce

IRC
The #salt IRC channel is hosted on the popular Freenode network. You can use the Freenode
webchat client right from your browser.

Logs of the IRC channel activity are being collected courtesy of Moritz Lenz.

If you wish to discuss the development of Salt itself join us in #salt-devel.

Follow on GitHub
The Salt code is developed via GitHub. Follow Salt for constant updates on what is
happening in Salt development:

https://github.com/saltstack/salt

Blogs
SaltStack Inc. keeps a blog with recent news and advancements:

http://www.saltstack.com/blog/

Thomas Hatch also shares news and thoughts on Salt and related projects in his personal
blog The Red45:

http://red45.wordpress.com/

Example Salt States
The official salt-states repository is: https://github.com/saltstack/salt-states

A few examples of salt states from the community:

• https://github.com/blast-hardcheese/blast-salt-states

• https://github.com/kevingranade/kevingranade-salt-state

• https://github.com/mattmcclean/salt-openstack/tree/master/salt

• https://github.com/rentalita/ubuntu-setup/

• https://github.com/brutasse/states

• https://github.com/bclermont/states

• https://github.com/pcrews/salt-data

Follow on ohloh
https://www.ohloh.net/p/salt

Other community links
• Salt Stack Inc.

• Subreddit

• Google+

• YouTube

• Facebook

• Twitter

• Wikipedia page

Hack the Source
If you want to get involved with the development of source code or the documentation
efforts, please review the hacking section!

INSTALLATION

SEE ALSO:
Installing Salt for development and contributing to the project.

Quick Install
On most distributions, you can set up a Salt Minion with the Salt Bootstrap.

Platform-specific Installation Instructions
These guides go into detail how to install Salt on a given platform.

Arch Linux
Installation
Salt (stable) is currently available via the Arch Linux Official repositories. There are
currently -git packages available in the Arch User repositories (AUR) as well.

Stable Release
Install Salt stable releases from the Arch Linux Official repositories as follows:

pacman -S salt-zmq

To install Salt stable releases using the RAET protocol, use the following:

pacman -S salt-raet

NOTE:
transports

Unlike other linux distributions, please be aware that Arch Linux's package manager
pacman defaults to RAET as the Salt transport. If you want to use ZeroMQ instead, make
sure to enter the associated number for the salt-zmq repository when prompted.

Tracking develop
To install the bleeding edge version of Salt (may include bugs!), use the -git package.
Installing the -git package as follows:

wget https://aur.archlinux.org/packages/sa/salt-git/salt-git.tar.gz
tar xf salt-git.tar.gz
cd salt-git/
makepkg -is

NOTE:
yaourt

If a tool such as Yaourt is used, the dependencies will be gathered and built
automatically.

The command to install salt using the yaourt tool is:

yaourt salt-git

Post-installation tasks
systemd

Activate the Salt Master and/or Minion via systemctl as follows:

systemctl enable salt-master.service
systemctl enable salt-minion.service

Start the Master

Once you've completed all of these steps you're ready to start your Salt Master. You
should be able to start your Salt Master now using the command seen here:

systemctl start salt-master

Now go to the Configuring Salt page.

Debian GNU/Linux / Raspbian
Debian GNU/Linux distribution and some devariatives such as Raspbian already have included
Salt packages to their repositories. However, current stable release codenamed "Jessie"
contains old outdated Salt release. It is recommended to use SaltStack repository for
Debian as described below.

Installation from official Debian and Raspbian repositories is described here.

Installation from the SaltStack Repository
2015.5 and later packages for Debian 8 ("Jessie") are available in the SaltStack
repository.

NOTE:
SaltStack repository contains only packages suitable for i386 (32-bit Intel-compatible
CPUs) and amd64 (64-bit) architectures. While Salt packages are built for all Debian
ports (have all suffix in package names), some of the dependencies are avaivable only
for amd64 systems.

IMPORTANT:
The repository folder structure changed in the 2015.8.3 release, though the previous
repository structure that was documented in 2015.8.1 can continue to be used.

To install using the SaltStack repository:

1. Run the following command to import the SaltStack repository key:

wget -O - https://repo.saltstack.com/apt/debian/8/amd64/latest/SALTSTACK-GPG-KEY.pub | sudo apt-key add -

2. Add the following line to /etc/apt/sources.list:

deb http://repo.saltstack.com/apt/debian/8/amd64/latest jessie main

3. Run sudo apt-get update.

4. Now go to the packages installation section.

Installation from the Community Repository
The SaltStack community maintains a Debian repository at debian.saltstack.com. Packages
for Debian Old Stable, Stable, and Unstable (Wheezy, Jessie, and Sid) for Salt 0.16 and
later are published in this repository.

NOTE:
Packages in this repository are community built, and it can take a little while until
the latest SaltStack release is available in this repository.

Jessie (Stable)
For Jessie, the following line is needed in either /etc/apt/sources.list or a file in
/etc/apt/sources.list.d:

deb http://debian.saltstack.com/debian jessie-saltstack main

Wheezy (Old Stable)
For Wheezy, the following line is needed in either /etc/apt/sources.list or a file in
/etc/apt/sources.list.d:

deb http://debian.saltstack.com/debian wheezy-saltstack main

Squeeze (Old Old Stable)
For Squeeze, you will need to enable the Debian backports repository as well as the
debian.saltstack.com repository. To do so, add the following to /etc/apt/sources.list or a
file in /etc/apt/sources.list.d:

deb http://debian.saltstack.com/debian squeeze-saltstack main
deb http://backports.debian.org/debian-backports squeeze-backports main

Stretch (Testing)
For Stretch, the following line is needed in either /etc/apt/sources.list or a file in
/etc/apt/sources.list.d:

deb http://debian.saltstack.com/debian stretch-saltstack main

Sid (Unstable)
For Sid, the following line is needed in either /etc/apt/sources.list or a file in
/etc/apt/sources.list.d:

deb http://debian.saltstack.com/debian unstable main

Import the repository key
You will need to import the key used for signing.

wget -q -O- "http://debian.saltstack.com/debian-salt-team-joehealy.gpg.key" | apt-key add -

NOTE:
You can optionally verify the key integrity with sha512sum using the public key
signature shown here. E.g:

echo "b702969447140d5553e31e9701be13ca11cc0a7ed5fe2b30acb8491567560ee62f834772b5095d735dfcecb2384a5c1a20045f52861c417f50b68dd5ff4660e6 debian-salt-team-joehealy.gpg.key" | sha512sum -c

Update the package database
apt-get update

Installation from the Debian / Raspbian Official Repository
Stretch (Testing) and Sid (Unstable) distributions are already contain mostly up-to-date
Salt packages built by Debian Salt Team. You can install Salt components directly from
Debian.

On Jessie (Stable) there is an option to install Salt minion from Stretch with
python-tornado dependency from jessie-backports repositories.

To install fresh release of Salt minion on Jessie:

1. Add jessie-backports and stretch repositories:

Debian:

echo 'deb http://httpredir.debian.org/debian jessie-backports main' >> /etc/apt/sources.list
echo 'deb http://httpredir.debian.org/debian stretch main' >> /etc/apt/sources.list

Raspbian:

echo 'deb http://archive.raspbian.org/raspbian/ stretch main' >> /etc/apt/sources.list

2. Make Jessie a default release:

echo 'APT::Default-Release "jessie";' > /etc/apt/apt.conf.d/10apt

3. Install Salt dependencies:

Debian:

apt-get update
apt-get install python-zmq python-tornado/jessie-backports salt-common/stretch

Raspbian:

apt-get update
apt-get install python-zmq python-tornado/stretch salt-common/stretch

4. Install Salt minion package from Stretch:

apt-get install salt-minion/stretch

Install Packages
Install the Salt master, minion or other packages from the repository with the apt-get
command. These examples each install one of Salt components, but more than one package
name may be given at a time:

• apt-get install salt-api

• apt-get install salt-cloud

• apt-get install salt-master

• apt-get install salt-minion

• apt-get install salt-ssh

• apt-get install salt-syndic

Post-installation tasks
Now, go to the Configuring Salt page.

Fedora
Beginning with version 0.9.4, Salt has been available in the primary Fedora repositories
and EPEL. It is installable using yum. Fedora will have more up to date versions of Salt
than other members of the Red Hat family, which makes it a great place to help improve
Salt!

WARNING: Fedora 19 comes with systemd 204. Systemd has known bugs fixed in later
revisions that prevent the salt-master from starting reliably or opening the network
connections that it needs to. It's not likely that a salt-master will start or run
reliably on any distribution that uses systemd version 204 or earlier. Running
salt-minions should be OK.

Installation
Salt can be installed using yum and is available in the standard Fedora repositories.

Stable Release
Salt is packaged separately for the minion and the master. It is necessary only to install
the appropriate package for the role the machine will play. Typically, there will be one
master and multiple minions.

yum install salt-master
yum install salt-minion

Installing from updates-testing
When a new Salt release is packaged, it is first admitted into the updates-testing
repository, before being moved to the stable repo.

To install from updates-testing, use the enablerepo argument for yum:

yum --enablerepo=updates-testing install salt-master
yum --enablerepo=updates-testing install salt-minion

Installation Using pip
Since Salt is on PyPI, it can be installed using pip, though most users prefer to install
using a package manager.

Installing from pip has a few additional requirements:

• Install the group 'Development Tools', dnf groupinstall 'Development Tools'

• Install the 'zeromq-devel' package if it fails on linking against that afterwards as
well.

A pip install does not make the init scripts or the /etc/salt directory, and you will need
to provide your own systemd service unit.

Installation from pip:

pip install salt

WARNING:
If installing from pip (or from source using setup.py install), be advised that the
yum-utils package is needed for Salt to manage packages. Also, if the Python
dependencies are not already installed, then you will need additional libraries/tools
installed to build some of them. More information on this can be found here.

Post-installation tasks
Master

To have the Master start automatically at boot time:

systemctl enable salt-master.service

To start the Master:

systemctl start salt-master.service

Minion

To have the Minion start automatically at boot time:

systemctl enable salt-minion.service

To start the Minion:

systemctl start salt-minion.service

Now go to the Configuring Salt page.

FreeBSD
Salt was added to the FreeBSD ports tree Dec 26th, 2011 by Christer Edwards <‐
christer.edwards@gmail.com>. It has been tested on FreeBSD 7.4, 8.2, 9.0, 9.1, 10.0 and
later releases.

Installation
Salt is available in binary package form from both the FreeBSD pkgng repository or
directly from SaltStack. The instructions below outline installation via both methods:

FreeBSD repo
The FreeBSD pkgng repository is preconfigured on systems 10.x and above. No configuration
is needed to pull from these repositories.

pkg install py27-salt

These packages are usually available within a few days of upstream release.

SaltStack repo
SaltStack also hosts internal binary builds of the Salt package, available from
https://repo.saltstack.com/freebsd/. To make use of this repository, add the following
file to your system:

/usr/local/etc/pkg/repos/saltstack.conf:

saltstack: {
url: "https://repo.saltstack.com/freebsd/${ABI}/",
mirror_type: "http",
enabled: yes
priority: 10
}

You should now be able to install Salt from this new repository:

pkg install py27-salt

These packages are usually available earlier than upstream FreeBSD. Also available are
release candidates and development releases. Use these pre-release packages with caution.

Post-installation tasks
Master

Copy the sample configuration file:

cp /usr/local/etc/salt/master.sample /usr/local/etc/salt/master

rc.conf

Activate the Salt Master in /etc/rc.conf:

sysrc salt_master_enable="YES"

Start the Master

Start the Salt Master as follows:

service salt_master start

Minion

Copy the sample configuration file:

cp /usr/local/etc/salt/minion.sample /usr/local/etc/salt/minion

rc.conf

Activate the Salt Minion in /etc/rc.conf:

sysrc salt_minion_enable="YES"

Start the Minion

Start the Salt Minion as follows:

service salt_minion start

Now go to the Configuring Salt page.

Gentoo
Salt can be easily installed on Gentoo via Portage:

emerge app-admin/salt

Post-installation tasks
Now go to the Configuring Salt page.

OpenBSD
Salt was added to the OpenBSD ports tree on Aug 10th 2013. It has been tested on OpenBSD
5.5 onwards.

Salt is dependent on the following additional ports. These will be installed as
dependencies of the sysutils/salt port:

devel/py-futures
devel/py-progressbar
net/py-msgpack
net/py-zmq
security/py-crypto
security/py-M2Crypto
textproc/py-MarkupSafe
textproc/py-yaml
www/py-jinja2
www/py-requests
www/py-tornado

Installation
To install Salt from the OpenBSD pkg repo, use the command:

pkg_add salt

Post-installation tasks
Master

To have the Master start automatically at boot time:

rcctl enable salt_master

To start the Master:

rcctl start salt_master

Minion

To have the Minion start automatically at boot time:

rcctl enable salt_minion

To start the Minion:

rcctl start salt_minion

Now go to the Configuring Salt page.

OS X
Dependency Installation
It should be noted that Homebrew explicitly discourages the use of sudo:
Homebrew is designed to work without using sudo. You can decide to use it but we
strongly recommend not to do so. If you have used sudo and run into a bug then it is
likely to be the cause. Please don’t file a bug report unless you can reproduce it
after reinstalling Homebrew from scratch without using sudo

So when using Homebrew, if you want support from the Homebrew community, install this way:

brew install saltstack

When using MacPorts, install this way:

sudo port install salt

When only using the OS X system's pip, install this way:

sudo pip install salt

Salt-Master Customizations
To run salt-master on OS X, the root user maxfiles limit must be increased:

NOTE:
On OS X 10.10 (Yosemite) and higher, maxfiles should not be adjusted. The default
limits are sufficient in all but the most extreme scenarios. Overriding these values
with the setting below will cause system instability!

sudo launchctl limit maxfiles 4096 8192

And sudo add this configuration option to the /etc/salt/master file:

max_open_files: 8192

Now the salt-master should run without errors:

sudo salt-master --log-level=all

Post-installation tasks
Now go to the Configuring Salt page.

RHEL / CentOS / Scientific Linux / Amazon Linux / Oracle Linux
Salt should work properly with all mainstream derivatives of Red Hat Enterprise Linux,
including CentOS, Scientific Linux, Oracle Linux, and Amazon Linux. Report any bugs or
issues on the issue tracker.

Installation from the SaltStack Repository
2015.5 and later packages for RHEL 5, 6, and 7 are available in the SaltStack repository.

IMPORTANT:
The repository folder structure changed in the 2015.8.3 release, though the previous
repository structure that was documented in 2015.8.1 can continue to be used.

To install using the SaltStack repository:

1. Run one of the following commands based on your version to import the SaltStack
repository key:

Version 7:

rpm --import https://repo.saltstack.com/yum/redhat/7/x86_64/latest/SALTSTACK-GPG-KEY.pub

Version 6:

rpm --import https://repo.saltstack.com/yum/redhat/6/x86_64/latest/SALTSTACK-GPG-KEY.pub

Version 5:

wget https://repo.saltstack.com/yum/redhat/5/x86_64/latest/SALTSTACK-EL5-GPG-KEY.pub
rpm --import SALTSTACK-EL5-GPG-KEY.pub
rm -f SALTSTACK-EL5-GPG-KEY.pub

2. Save the following file to /etc/yum.repos.d/saltstack.repo:

Version 7 and 6:

[saltstack-repo]
name=SaltStack repo for RHEL/CentOS $releasever
baseurl=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest
enabled=1
gpgcheck=1
gpgkey=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest/SALTSTACK-GPG-KEY.pub

Version 5:

3. Run sudo yum clean expire-cache.

4. Run sudo yum update.

5. Install the salt-minion, salt-master, or other Salt components:

• yum install salt-master

• yum install salt-minion

• yum install salt-ssh

• yum install salt-syndic

• yum install salt-cloud

NOTE:
As of 2015.8.0, EPEL repository is no longer required for installing on RHEL systems.
SaltStack repository provides all needed dependencies.

WARNING:
If installing on Red Hat Enterprise Linux 7 with disabled (not subscribed on) 'RHEL
Server Releases' or 'RHEL Server Optional Channel' repositories, append CentOS 7 GPG
key URL to SaltStack yum repository configuration to install required base packages:

[saltstack-repo]
name=SaltStack repo for Red Hat Enterprise Linux $releasever
baseurl=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest
enabled=1
gpgcheck=1
gpgkey=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest/SALTSTACK-GPG-KEY.pub
https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest/base/RPM-GPG-KEY-CentOS-7

NOTE:
systemd and python-systemd are required by Salt, but are not installed by the Red Hat 7
@base installation or by the Salt installation. These dependencies might need to be
installed before Salt.

Installation from the Community Repository
Beginning with version 0.9.4, Salt has been available in EPEL. For RHEL/CentOS 5, Fedora
COPR is a single community repository that provides Salt packages due to the removal from
EPEL5.

NOTE:
Packages in these repositories are built by community, and it can take a little while
until the latest stable SaltStack release become available.

RHEL/CentOS 6 and 7, Scientific Linux, etc.
WARNING:
Salt 2015.8 is currently not available in EPEL due to unsatisfied dependencies:
python-crypto 2.6.1 or higher, and python-tornado version 4.2.1 or higher. These
packages are not currently available in EPEL for Red Hat Enterprise Linux 6 and 7.

Enabling EPEL
If the EPEL repository is not installed on your system, you can download the RPM for
RHEL/CentOS 6 or for RHEL/CentOS 7 and install it using the following command:

rpm -Uvh epel-release-X-Y.rpm

Replace epel-release-X-Y.rpm with the appropriate filename.

Installing Stable Release
Salt is packaged separately for the minion and the master. It is necessary to install only
the appropriate package for the role the machine will play. Typically, there will be one
master and multiple minions.

• yum install salt-master

• yum install salt-minion

• yum install salt-ssh

• yum install salt-syndic

• yum install salt-cloud

Installing from epel-testing
When a new Salt release is packaged, it is first admitted into the epel-testing
repository, before being moved to the stable EPEL repository.

To install from epel-testing, use the enablerepo argument for yum:

yum --enablerepo=epel-testing install salt-minion

Installation Using pip
Since Salt is on PyPI, it can be installed using pip, though most users prefer to install
using RPM packages (which can be installed from EPEL).

Installing from pip has a few additional requirements:

• Install the group 'Development Tools', yum groupinstall 'Development Tools'

• Install the 'zeromq-devel' package if it fails on linking against that afterwards as
well.

A pip install does not make the init scripts or the /etc/salt directory, and you will need
to provide your own systemd service unit.

Installation from pip:

pip install salt

ZeroMQ 4
We recommend using ZeroMQ 4 where available. SaltStack provides ZeroMQ 4.0.5 and pyzmq
14.5.0 in the SaltStack Repository as well as a separate zeromq4 COPR repository.

If this repository is added before Salt is installed, then installing either salt-master
or salt-minion will automatically pull in ZeroMQ 4.0.5, and additional steps to upgrade
ZeroMQ and pyzmq are unnecessary.

WARNING:
RHEL/CentOS 5 Users Using COPR repos on RHEL/CentOS 5 requires that the python-hashlib
package be installed. Not having it present will result in checksum errors because YUM
will not be able to process the SHA256 checksums used by COPR.

NOTE:
For RHEL/CentOS 5 installations, if using the SaltStack repo or Fedora COPR to install
Salt (as described above), then it is not necessary to enable the zeromq4 COPR, because
those repositories already include ZeroMQ 4.

Package Management
Salt's interface to yum makes heavy use of the repoquery utility, from the yum-utils
package. This package will be installed as a dependency if salt is installed via EPEL.
However, if salt has been installed using pip, or a host is being managed using salt-ssh,
then as of version 2014.7.0 yum-utils will be installed automatically to satisfy this
dependency.

Post-installation tasks
Master
To have the Master start automatically at boot time:

RHEL/CentOS 5 and 6

chkconfig salt-master on

RHEL/CentOS 7

systemctl enable salt-master.service

To start the Master:

RHEL/CentOS 5 and 6

service salt-master start

RHEL/CentOS 7

systemctl start salt-master.service

Minion
To have the Minion start automatically at boot time:

RHEL/CentOS 5 and 6

chkconfig salt-minion on

RHEL/CentOS 7

systemctl enable salt-minion.service

To start the Minion:

RHEL/CentOS 5 and 6

service salt-minion start

RHEL/CentOS 7

systemctl start salt-minion.service

Now go to the Configuring Salt page.

Solaris
Salt was added to the OpenCSW package repository in September of 2012 by Romeo Theriault
<romeot@hawaii.edu> at version 0.10.2 of Salt. It has mainly been tested on Solaris 10
(sparc), though it is built for and has been tested minimally on Solaris 10 (x86), Solaris
9 (sparc/x86) and 11 (sparc/x86). (Please let me know if you're using it on these
platforms!) Most of the testing has also just focused on the minion, though it has
verified that the master starts up successfully on Solaris 10.

Comments and patches for better support on these platforms is very welcome.

As of version 0.10.4, Solaris is well supported under salt, with all of the following
working well:

1. remote execution

2. grain detection

3. service control with SMF

4. 'pkg' states with 'pkgadd' and 'pkgutil' modules

5. cron modules/states

6. user and group modules/states

7. shadow password management modules/states

Salt is dependent on the following additional packages. These will automatically be
installed as dependencies of the py_salt package:

• py_yaml

• py_pyzmq

• py_jinja2

• py_msgpack_python

• py_m2crypto

• py_crypto

• python

Installation
To install Salt from the OpenCSW package repository you first need to install pkgutil
assuming you don't already have it installed:

On Solaris 10:

pkgadd -d http://get.opencsw.org/now

On Solaris 9:

wget http://mirror.opencsw.org/opencsw/pkgutil.pkg
pkgadd -d pkgutil.pkg all

Once pkgutil is installed you'll need to edit it's config file /etc/opt/csw/pkgutil.conf
to point it at the unstable catalog:

- #mirror=http://mirror.opencsw.org/opencsw/testing
+ mirror=http://mirror.opencsw.org/opencsw/unstable

OK, time to install salt.

# Update the catalog
root> /opt/csw/bin/pkgutil -U
# Install salt
root> /opt/csw/bin/pkgutil -i -y py_salt

Minion Configuration
Now that salt is installed you can find it's configuration files in /etc/opt/csw/salt/.

You'll want to edit the minion config file to set the name of your salt master server:

- #master: salt
+ master: your-salt-server

If you would like to use pkgutil as the default package provider for your Solaris minions,
you can do so using the providers option in the minion config file.

You can now start the salt minion like so:

On Solaris 10:

svcadm enable salt-minion

On Solaris 9:

/etc/init.d/salt-minion start

You should now be able to log onto the salt master and check to see if the salt-minion key
is awaiting acceptance:

salt-key -l un

Accept the key:

salt-key -a <your-salt-minion>

Run a simple test against the minion:

salt '<your-salt-minion>' test.ping

Troubleshooting
Logs are in /var/log/salt

Ubuntu
Installation from the SaltStack Repository
2015.5 and later packages for Ubuntu 14 (Trusty) and Ubuntu 12 (Precise) are available in
the SaltStack repository.

NOTE:
While Salt packages are built for all Ubuntu supported CPU architectures (i386 and
amd64), some of the dependencies avaivable from SaltStack corporate repository are only
suitable for amd64 systems.

IMPORTANT:
The repository folder structure changed in the 2015.8.3 release, though the previous
repository structure that was documented in 2015.8.1 can continue to be used.

To install using the SaltStack repository:

1. Run the following command to import the SaltStack repository key:

Ubuntu 14:

wget -O - https://repo.saltstack.com/apt/ubuntu/14.04/amd64/latest/SALTSTACK-GPG-KEY.pub | sudo apt-key add -

Ubuntu 12:

wget -O - https://repo.saltstack.com/apt/ubuntu/12.04/amd64/latest/SALTSTACK-GPG-KEY.pub | sudo apt-key add -

2. Add the following line to /etc/apt/sources.list:

Ubuntu 14:

deb http://repo.saltstack.com/apt/ubuntu/14.04/amd64/latest trusty main

Ubuntu 12:

deb http://repo.saltstack.com/apt/ubuntu/12.04/amd64/latest precise main

3. Run sudo apt-get update.

4. Now go to the packages installation section.

Installation from the Community Repository
Packages for Ubuntu are also published in the saltstack PPA. If you have the
add-apt-repository utility, you can add the repository and import the key in one step:

sudo add-apt-repository ppa:saltstack/salt

In addition to the main repository, there are secondary repositories for each individual
major release. These repositories receive security and point releases but will not upgrade
to any subsequent major release. There are currently several available repos: salt16,
salt17, salt2014-1, salt2014-7, salt2015-5. For example to follow 2015.5.x releases:

sudo add-apt-repository ppa:saltstack/salt2015-5

add-apt-repository: command not found?

The add-apt-repository command is not always present on Ubuntu systems. This
can be fixed by installing python-software-properties:

sudo apt-get install python-software-properties

The following may be required as well:

sudo apt-get install software-properties-common

Note that since Ubuntu 12.10 (Raring Ringtail), add-apt-repository is found in the
software-properties-common package, and is part of the base install. Thus,
add-apt-repository should be able to be used out-of-the-box to add the PPA.

Alternately, manually add the repository and import the PPA key with these commands:

echo deb http://ppa.launchpad.net/saltstack/salt/ubuntu `lsb_release -sc` main | sudo tee /etc/apt/sources.list.d/saltstack.list
wget -q -O- "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x4759FA960E27C0A6" | sudo apt-key add -

After adding the repository, update the package management database:

sudo apt-get update

• apt-get install salt-api

• apt-get install salt-cloud

• apt-get install salt-master

• apt-get install salt-minion

• apt-get install salt-ssh

• apt-get install salt-syndic

Post-installation tasks
Now go to the Configuring Salt page.

Windows
Salt has full support for running the Salt Minion on Windows.

There are no plans for the foreseeable future to develop a Salt Master on Windows. For now
you must run your Salt Master on a supported operating system to control your Salt Minions
on Windows.

Many of the standard Salt modules have been ported to work on Windows and many of the Salt
States currently work on Windows, as well.

Windows Installer
Salt Minion Windows installers can be found here. The output of md5sum <salt minion exe>
should match the contents of the corresponding md5 file.

Latest stable build from the selected branch:

Earlier builds from supported branches

Archived builds from unsupported branches

NOTE:
The installation executable installs dependencies that the Salt minion requires.

The 64bit installer has been tested on Windows 7 64bit and Windows Server 2008R2 64bit.
The 32bit installer has been tested on Windows 2003 Server 32bit. Please file a bug
report on our GitHub repo if issues for other platforms are found.

The installer asks for 2 bits of information; the master hostname and the minion name. The
installer will update the minion config with these options and then start the minion.

The salt-minion service will appear in the Windows Service Manager and can be started and
stopped there or with the command line program sc like any other Windows service.

If the minion won't start, try installing the Microsoft Visual C++ 2008 x64 SP1
redistributable. Allow all Windows updates to run salt-minion smoothly.

Silent Installer Options
The installer can be run silently by providing the /S option at the command line. The
installer also accepts the following options for configuring the Salt Minion silently:

• /master= A string value to set the IP address or host name of the master. Default value
is 'salt'

• /minion-name= A string value to set the minion name. Default is 'hostname'

• /start-service= Either a 1 or 0. '1' will start the service, '0' will not. Default is to
start the service after installation.

Here's an example of using the silent installer:

Salt-Minion-2015.5.6-Setup-amd64.exe /S /master=yoursaltmaster /minion-name=yourminionname /start-service=0

Running the Salt Minion on Windows as an Unprivileged User
Notes: - These instructions were tested with Windows Server 2008 R2 - They are
generalizable to any version of Windows that supports a salt-minion

A. Create the Unprivileged User that the Salt Minion will Run As
1. Click Start > Control Panel > User Accounts.

2. Click Add or remove user accounts.

3. Click Create new account.

4. Enter salt-user (or a name of your preference) in the New account name field.

5. Select the Standard user radio button.

6. Click the Create Account button.

7. Click on the newly created user account.

8. Click the Create a password link.

9. In the New password and Confirm new password fields, provide a password (e.g
"SuperSecretMinionPassword4Me!").

10. In the Type a password hint field, provide appropriate text (e.g. "My Salt Password").

11. Click the Create password button.

12. Close the Change an Account window.

B. Add the New User to the Access Control List for the Salt Folder
1. In a File Explorer window, browse to the path where Salt is installed (the default path
is C:\Salt).

2. Right-click on the Salt folder and select Properties.

3. Click on the Security tab.

4. Click the Edit button.

5. Click the Add button.

6. Type the name of your designated Salt user and click the OK button.

7. Check the box to Allow the Modify permission.

8. Click the OK button.

9. Click the OK button to close the Salt Properties window.

C. Update the Windows Service User for the salt-minion Service
1. Click Start > Administrative Tools > Services.

2. In the Services list, right-click on salt-minion and select Properties.

3. Click the Log On tab.

4. Click the This account radio button.

5. Provide the account credentials created in section A.

6. Click the OK button.

7. Click the OK button to the prompt confirming that the user has been granted the Log On
As A Service right.

8. Click the OK button to the prompt confirming that The new logon name will not take
effect until you stop and restart the service.

9. Right-Click on salt-minion and select Stop.

10. Right-Click on salt-minion and select Start.

Setting up a Windows build environment
This document will explain how to set up a development environment for salt on Windows.
The development environment allows you to work with the source code to customize or fix
bugs. It will also allow you to build your own installation.

The Easy Way
Prerequisite Software
To do this the easy way you only need to install Git for Windows.

Create the Build Environment
1. Clone the Salt-Windows-Dev repo from github.

Open a command line and type:

git clone https://github.com/saltstack/salt-windows-dev

2. Build the Python Environment

Go into the salt-windows-dev directory. Right-click the file named dev_env.ps1 and
select Run with PowerShell

If you get an error, you may need to change the execution policy.

Open a powershell window and type the following:

Set-ExecutionPolicy RemoteSigned

This will download and install Python with all the dependencies needed to develop and
build salt.

3. Build the Salt Environment

Right-click on the file named dev_env_salt.ps1 and select Run with Powershell

This will clone salt into C:\Salt-Dev\salt and set it to the 2015.5 branch. You could
optionally run the command from a powershell window with a -Version switch to pull a
different version. For example:

dev_env_salt.ps1 -Version '2014.7'

To view a list of available branches and tags, open a command prompt in your
C:Salt-Devsalt directory and type:

git branch -a
git tag -n

The Hard Way
Prerequisite Software
Install the following software:

1. Git for Windows

2. Nullsoft Installer

Download the Prerequisite zip file for your CPU architecture from the SaltStack download
site:

• Salt32.zip

• Salt64.zip

These files contain all software required to build and develop salt. Unzip the contents of
the file to C:\Salt-Dev\temp.

Create the Build Environment
1. Build the Python Environment

• Install Python:

Browse to the C:\Salt-Dev\temp directory and find the Python installation file for
your CPU Architecture under the corresponding subfolder. Double-click the file to
install python.

Make sure the following are in your PATH environment variable:

C:\Python27
C:\Python27\Scripts

• Install Pip

Open a command prompt and navigate to C:\Salt-Dev\temp Run the following command:

python get-pip.py

• Easy Install compiled binaries.

M2Crypto, PyCrypto, and PyWin32 need to be installed using Easy Install. Open a
command prompt and navigate to C:\Salt-Dev\temp\<cpuarch>. Run the following
commands:

easy_install -Z <M2Crypto file name>
easy_install -Z <PyCrypto file name>
easy_install -Z <PyWin32 file name>

NOTE:
You can type the first part of the file name and then press the tab key to
auto-complete the name of the file.

• Pip Install Additional Prerequisites

All remaining prerequisites need to be pip installed. These prerequisites are as
follow:

• MarkupSafe

• Jinja

• MsgPack

• PSUtil

• PyYAML

• PyZMQ

• WMI

• Requests

• Certifi

Open a command prompt and navigate to C:\Salt-Dev\temp. Run the following commands:

pip install <cpuarch>\<MarkupSafe file name>
pip install <Jinja file name>
pip install <cpuarch>\<MsgPack file name>
pip install <cpuarch>\<psutil file name>
pip install <cpuarch>\<PyYAML file name>
pip install <cpuarch>\<pyzmq file name>
pip install <WMI file name>
pip install <requests file name>
pip install <certifi file name>

2. Build the Salt Environment

• Clone Salt

Open a command prompt and navigate to C:\Salt-Dev. Run the following command to clone
salt:

git clone https://github.com/saltstack/salt

• Checkout Branch

Checkout the branch or tag of salt you want to work on or build. Open a command
prompt and navigate to C:\Salt-Dev\salt. Get a list of available tags and branches by
running the following commands:

git fetch --all

To view a list of available branches:
git branch -a

To view a list of availabel tags:
git tag -n

Checkout the branch or tag by typing the following command:

git checkout <branch/tag name>

• Clean the Environment

When switching between branches residual files can be left behind that will interfere
with the functionality of salt. Therefore, after you check out the branch you want to
work on, type the following commands to clean the salt environment:

Developing with Salt
There are two ways to develop with salt. You can run salt's setup.py each time you make a
change to source code or you can use the setup tools develop mode.

Configure the Minion
Both methods require that the minion configuration be in the C:\salt directory. Copy the
conf and var directories from C:\Salt-Dev\salt\pkg\ windows\buildenv to C:\salt. Now go
into the C:\salt\conf directory and edit the file name minion (no extension). You need to
configure the master and id parameters in this file. Edit the following lines:

master: <ip or name of your master>
id: <name of your minion>

Setup.py Method
Go into the C:\Salt-Dev\salt directory from a cmd prompt and type:

python setup.py install --force

This will install python into your python installation at C:\Python27. Everytime you make
an edit to your source code, you'll have to stop the minion, run the setup, and start the
minion.

To start the salt-minion go into C:\Python27\Scripts from a cmd prompt and type:

salt-minion

For debug mode type:

salt-minion -l debug

To stop the minion press Ctrl+C.

Setup Tools Develop Mode (Preferred Method)
To use the Setup Tools Develop Mode go into C:\Salt-Dev\salt from a cmd prompt and type:

pip install -e .

This will install pointers to your source code that resides at C:\Salt-Dev\salt. When you
edit your source code you only have to restart the minion.

Build the windows installer
This is the method of building the installer as of version 2014.7.4.

Clean the Environment
Make sure you don't have any leftover salt files from previous versions of salt in your
Python directory.

1. Remove all files that start with salt in the C:\Python27\Scripts directory

2. Remove all files and directorys that start with salt in the
C:\Python27\Lib\site-packages directory

Install Salt
Install salt using salt's setup.py. From the C:\Salt-Dev\salt directory type the following
command:

python setup.py install --force

Build the Installer
From cmd prompt go into the C:\Salt-Dev\salt\pkg\windows directory. Type the following
command for the branch or tag of salt you're building:

BuildSalt.bat <branch or tag>

This will copy python with salt installed to the buildenv\bin directory, make it portable,
and then create the windows installer . The .exe for the windows installer will be placed
in the installer directory.

Testing the Salt minion
1. Create the directory C:\salt (if it doesn't exist already)

2. Copy the example conf and var directories from pkg/windows/buildenv/ into C:\salt

3. Edit C:\salt\conf\minion

master: ipaddress or hostname of your salt-master

4. Start the salt-minion

cd C:\Python27\Scripts
python salt-minion

5. On the salt-master accept the new minion's key

sudo salt-key -A

This accepts all unaccepted keys. If you're concerned about security just accept the
key for this specific minion.

6. Test that your minion is responding

On the salt-master run:

sudo salt '*' test.ping

You should get the following response: {'your minion hostname': True}

Single command bootstrap script
On a 64 bit Windows host the following script makes an unattended install of salt,
including all dependencies:

Not up to date.

This script is not up to date. Please use the installer found above

# (All in one line.)

"PowerShell (New-Object System.Net.WebClient).DownloadFile('http://csa-net.dk/salt/bootstrap64.bat','C:\bootstrap.bat');(New-Object -com Shell.Application).ShellExecute('C:\bootstrap.bat');"

You can execute the above command remotely from a Linux host using winexe:

winexe -U "administrator" //fqdn "PowerShell (New-Object ......);"

For more info check http://csa-net.dk/salt

Packages management under Windows 2003
On windows Server 2003, you need to install optional component "wmi windows installer
provider" to have full list of installed packages. If you don't have this, salt-minion
can't report some installed software.

SUSE
With openSUSE 13.2, Salt 2014.1.11 is available in the primary repositories. The
devel:language:python repo will have more up to date versions of salt, all package
development will be done there.

Installation
Salt can be installed using zypper and is available in the standard openSUSE repositories.

zypper install salt-master
zypper install salt-minion

Post-installation tasks openSUSE
Master

To have the Master start automatically at boot time:

systemctl enable salt-master.service

To start the Master:

systemctl start salt-master.service

Minion

To have the Minion start automatically at boot time:

systemctl enable salt-minion.service

To start the Minion:

systemctl start salt-minion.service

Post-installation tasks SLES
Master

To have the Master start automatically at boot time:

chkconfig salt-master on

To start the Master:

rcsalt-master start

Minion

To have the Minion start automatically at boot time:

chkconfig salt-minion on

To start the Minion:

rcsalt-minion start

Unstable Release
openSUSE
For openSUSE Factory run the following as root:

zypper addrepo http://download.opensuse.org/repositories/devel:languages:python/openSUSE_Factory/devel:languages:python.repo
zypper refresh
zypper install salt salt-minion salt-master

For openSUSE 13.2 run the following as root:

zypper addrepo http://download.opensuse.org/repositories/devel:languages:python/openSUSE_13.2/devel:languages:python.repo
zypper refresh
zypper install salt salt-minion salt-master

For openSUSE 13.1 run the following as root:

zypper addrepo http://download.opensuse.org/repositories/devel:languages:python/openSUSE_13.1/devel:languages:python.repo
zypper refresh
zypper install salt salt-minion salt-master

For bleeding edge python Factory run the following as root:

zypper addrepo http://download.opensuse.org/repositories/devel:languages:python/bleeding_edge_python_Factory/devel:languages:python.repo
zypper refresh
zypper install salt salt-minion salt-master

Suse Linux Enterprise
For SLE 12 run the following as root:

zypper addrepo http://download.opensuse.org/repositories/devel:languages:python/SLE_12/devel:languages:python.repo
zypper refresh
zypper install salt salt-minion salt-master

For SLE 11 SP3 run the following as root:

zypper addrepo http://download.opensuse.org/repositories/devel:languages:python/SLE_11_SP3/devel:languages:python.repo
zypper refresh
zypper install salt salt-minion salt-master

For SLE 11 SP2 run the following as root:

zypper addrepo http://download.opensuse.org/repositories/devel:languages:python/SLE_11_SP2/devel:languages:python.repo
zypper refresh
zypper install salt salt-minion salt-master

Now go to the Configuring Salt page.

Dependencies
Salt should run on any Unix-like platform so long as the dependencies are met.

• Python 2.6 >= 2.6 <3.0

• msgpack-python - High-performance message interchange format

• YAML - Python YAML bindings

• Jinja2 - parsing Salt States (configurable in the master settings)

• MarkupSafe - Implements a XML/HTML/XHTML Markup safe string for Python

• apache-libcloud - Python lib for interacting with many of the popular cloud service
providers using a unified API

• Requests - HTTP library

Depending on the chosen Salt transport, ZeroMQ or RAET, dependencies vary:

• ZeroMQ:

• ZeroMQ >= 3.2.0

• pyzmq >= 2.2.0 - ZeroMQ Python bindings

• PyCrypto - The Python cryptography toolkit

• RAET:

• libnacl - Python bindings to libsodium

• ioflo - The flo programming interface raet and salt-raet is built on

• RAET - The worlds most awesome UDP protocol

Salt defaults to the ZeroMQ transport, and the choice can be made at install time, for
example:

python setup.py --salt-transport=raet install

This way, only the required dependencies are pulled by the setup script if need be.

If installing using pip, the --salt-transport install option can be provided like:

pip install --install-option="--salt-transport=raet" salt

NOTE:
Salt does not bundle dependencies that are typically distributed as part of the base
OS. If you have unmet dependencies and are using a custom or minimal installation, you
might need to install some additional packages from your OS vendor.

Optional Dependencies
• mako - an optional parser for Salt States (configurable in the master settings)

• gcc - dynamic Cython module compiling

Upgrading Salt
When upgrading Salt, the master(s) should always be upgraded first. Backward
compatibility for minions running newer versions of salt than their masters is not
guaranteed.

Whenever possible, backward compatibility between new masters and old minions will be
preserved. Generally, the only exception to this policy is in case of a security
vulnerability.

TUTORIALS

Introduction
Salt Masterless Quickstart
Running a masterless salt-minion lets you use Salt's configuration management for a single
machine without calling out to a Salt master on another machine.

Since the Salt minion contains such extensive functionality it can be useful to run it
standalone. A standalone minion can be used to do a number of things:

• Stand up a master server via States (Salting a Salt Master)

• Use salt-call commands on a system without connectivity to a master

• Masterless States, run states entirely from files local to the minion

It is also useful for testing out state trees before deploying to a production setup.

Bootstrap Salt Minion
The salt-bootstrap script makes bootstrapping a server with Salt simple for any OS with a
Bourne shell:

curl -L https://bootstrap.saltstack.com -o install_salt.sh
sudo sh install_salt.sh

See the salt-bootstrap documentation for other one liners. When using Vagrant to test out
salt, the Vagrant salt provisioner will provision the VM for you.

Telling Salt to Run Masterless
To instruct the minion to not look for a master, the file_client configuration option
needs to be set in the minion configuration file. By default the file_client is set to
remote so that the minion gathers file server and pillar data from the salt master. When
setting the file_client option to local the minion is configured to not gather this data
from the master.

file_client: local

Now the salt minion will not look for a master and will assume that the local system has
all of the file and pillar resources.

NOTE:
When running Salt in masterless mode, do not run the salt-minion daemon. Otherwise, it
will attempt to connect to a master and fail. The salt-call command stands on its own
and does not need the salt-minion daemon.

Create State Tree
Following the successful installation of a salt-minion, the next step is to create a state
tree, which is where the SLS files that comprise the possible states of the minion are
stored.

The following example walks through the steps necessary to create a state tree that
ensures that the server has the Apache webserver installed.

NOTE:
For a complete explanation on Salt States, see the tutorial.

1. Create the top.sls file:

/srv/salt/top.sls:

base:
'*':
- webserver

2. Create the webserver state tree:

/srv/salt/webserver.sls:

apache: # ID declaration
pkg: # state declaration
- installed # function declaration

NOTE:
The apache package has different names on different platforms, for instance on
Debian/Ubuntu it is apache2, on Fedora/RHEL it is httpd and on Arch it is apache

The only thing left is to provision our minion using salt-call and the highstate command.

Salt-call
The salt-call command is used to run module functions locally on a minion instead of
executing them from the master. Normally the salt-call command checks into the master to
retrieve file server and pillar data, but when running standalone salt-call needs to be
instructed to not check the master for this data:

salt-call --local state.highstate

The --local flag tells the salt-minion to look for the state tree in the local file system
and not to contact a Salt Master for instructions.

To provide verbose output, use -l debug:

salt-call --local state.highstate -l debug

The minion first examines the top.sls file and determines that it is a part of the group
matched by * glob and that the webserver SLS should be applied.

It then examines the webserver.sls file and finds the apache state, which installs the
Apache package.

The minion should now have Apache installed, and the next step is to begin learning how to
write more complex states.

Basics
Standalone Minion
Since the Salt minion contains such extensive functionality it can be useful to run it
standalone. A standalone minion can be used to do a number of things:

• Use salt-call commands on a system without connectivity to a master

• Masterless States, run states entirely from files local to the minion

Telling Salt Call to Run Masterless
The salt-call command is used to run module functions locally on a minion instead of
executing them from the master. Normally the salt-call command checks into the master to
retrieve file server and pillar data, but when running standalone salt-call needs to be
instructed to not check the master for this data. To instruct the minion to not look for a
master when running salt-call the file_client configuration option needs to be set. By
default the file_client is set to remote so that the minion knows that file server and
pillar data are to be gathered from the master. When setting the file_client option to
local the minion is configured to not gather this data from the master.

file_client: local

Now the salt-call command will not look for a master and will assume that the local system
has all of the file and pillar resources.

Running States Masterless
The state system can be easily run without a Salt master, with all needed files local to
the minion. To do this the minion configuration file needs to be set up to know how to
return file_roots information like the master. The file_roots setting defaults to
/srv/salt for the base environment just like on the master:

file_roots:
base:
- /srv/salt

Now set up the Salt State Tree, top file, and SLS modules in the same way that they would
be set up on a master. Now, with the file_client option set to local and an available
state tree then calls to functions in the state module will use the information in the
file_roots on the minion instead of checking in with the master.

Remember that when creating a state tree on a minion there are no syntax or path changes
needed, SLS modules written to be used from a master do not need to be modified in any way
to work with a minion.

This makes it easy to "script" deployments with Salt states without having to set up a
master, and allows for these SLS modules to be easily moved into a Salt master as the
deployment grows.

The declared state can now be executed with:

salt-call state.highstate

Or the salt-call command can be executed with the --local flag, this makes it unnecessary
to change the configuration file:

salt-call state.highstate --local

External Pillars
External pillars are supported when running in masterless mode.

Opening the Firewall up for Salt
The Salt master communicates with the minions using an AES-encrypted ZeroMQ connection.
These communications are done over TCP ports 4505 and 4506, which need to be accessible on
the master only. This document outlines suggested firewall rules for allowing these
incoming connections to the master.

NOTE:
No firewall configuration needs to be done on Salt minions. These changes refer to the
master only.

Fedora 18 and beyond / RHEL 7 / CentOS 7
Starting with Fedora 18 FirewallD is the tool that is used to dynamically manage the
firewall rules on a host. It has support for IPv4/6 settings and the separation of runtime
and permanent configurations. To interact with FirewallD use the command line client
firewall-cmd.

firewall-cmd example:

firewall-cmd --permanent --zone=<zone> --add-port=4505-4506/tcp

Please choose the desired zone according to your setup. Don't forget to reload after you
made your changes.

firewall-cmd --reload

RHEL 6 / CentOS 6
The lokkit command packaged with some Linux distributions makes opening iptables firewall
ports very simple via the command line. Just be careful to not lock out access to the
server by neglecting to open the ssh port.

lokkit example:

lokkit -p 22:tcp -p 4505:tcp -p 4506:tcp

The system-config-firewall-tui command provides a text-based interface to modifying the
firewall.

system-config-firewall-tui:

system-config-firewall-tui

openSUSE
Salt installs firewall rules in /etc/sysconfig/SuSEfirewall2.d/services/salt. Enable
with:

SuSEfirewall2 open
SuSEfirewall2 start

If you have an older package of Salt where the above configuration file is not included,
the SuSEfirewall2 command makes opening iptables firewall ports very simple via the
command line.

SuSEfirewall example:

SuSEfirewall2 open EXT TCP 4505
SuSEfirewall2 open EXT TCP 4506

The firewall module in YaST2 provides a text-based interface to modifying the firewall.

YaST2:

yast2 firewall

iptables
Different Linux distributions store their iptables (also known as netfilter) rules in
different places, which makes it difficult to standardize firewall documentation. Included
are some of the more common locations, but your mileage may vary.

Fedora / RHEL / CentOS:

/etc/sysconfig/iptables

Arch Linux:

/etc/iptables/iptables.rules

Debian

Follow these instructions: https://wiki.debian.org/iptables

Once you've found your firewall rules, you'll need to add the two lines below to allow
traffic on tcp/4505 and tcp/4506:

-A INPUT -m state --state new -m tcp -p tcp --dport 4505 -j ACCEPT
-A INPUT -m state --state new -m tcp -p tcp --dport 4506 -j ACCEPT

Ubuntu

Salt installs firewall rules in /etc/ufw/applications.d/salt.ufw. Enable with:

ufw allow salt

pf.conf
The BSD-family of operating systems uses packet filter (pf). The following example
describes the additions to pf.conf needed to access the Salt master.

pass in on $int_if proto tcp from any to $int_if port 4505
pass in on $int_if proto tcp from any to $int_if port 4506

Once these additions have been made to the pf.conf the rules will need to be reloaded.
This can be done using the pfctl command.

pfctl -vf /etc/pf.conf

Whitelist communication to Master
There are situations where you want to selectively allow Minion traffic from specific
hosts or networks into your Salt Master. The first scenario which comes to mind is to
prevent unwanted traffic to your Master out of security concerns, but another scenario is
to handle Minion upgrades when there are backwards incompatible changes between the
installed Salt versions in your environment.

Here is an example Linux iptables ruleset to be set on the Master:

# Allow Minions from these networks
-I INPUT -s 10.1.2.0/24 -p tcp -m multiport --dports 4505,4506 -j ACCEPT
-I INPUT -s 10.1.3.0/24 -p tcp -m multiport --dports 4505,4506 -j ACCEPT
# Allow Salt to communicate with Master on the loopback interface
-A INPUT -i lo -p tcp -m multiport --dports 4505,4506 -j ACCEPT
# Reject everything else
-A INPUT -p tcp -m multiport --dports 4505,4506 -j REJECT

NOTE:
The important thing to note here is that the salt command needs to communicate with the
listening network socket of salt-master on the loopback interface. Without this you
will see no outgoing Salt traffic from the master, even for a simple salt '*'
test.ping, because the salt client never reached the salt-master to tell it to carry
out the execution.

Using cron with Salt
The Salt Minion can initiate its own highstate using the salt-call command.

$ salt-call state.highstate

This will cause the minion to check in with the master and ensure it is in the correct
'state'.

Use cron to initiate a highstate
If you would like the Salt Minion to regularly check in with the master you can use the
venerable cron to run the salt-call command.

# PATH=/bin:/sbin:/usr/bin:/usr/sbin

00 00 * * * salt-call state.highstate

The above cron entry will run a highstate every day at midnight.

NOTE:
Be aware that you may need to ensure the PATH for cron includes any scripts or commands
that need to be executed.

Remote execution tutorial
Before continuing make sure you have a working Salt installation by following the
installation and the configuration instructions.

Stuck?

There are many ways to get help from the Salt community including our mailing
list and our IRC channel #salt.

Order your minions around
Now that you have a master and at least one minion communicating with each other you can
perform commands on the minion via the salt command. Salt calls are comprised of three
main components:

salt '<target>' <function> [arguments]

TARGETING MINIONS

       Targeting  minions  is specifying which minions should run a command or execute a state by
       matching against hostnames, or system information, or defined groups, or even combinations
       thereof.

       For example the command salt web1 apache.signal restart to restart the Apache httpd server
       specifies the machine web1 as the target and the command will only  be  run  on  that  one
       minion.

       Similarly  when  using  States, the following top file specifies that only the web1 minion
       should execute the contents of webserver.sls:

          base:
            'web1':
              - webserver

       There are many ways to target individual minions or groups of minions in Salt:

   Matching the minion id
       Each minion needs a unique identifier. By default when a minion starts for the first  time
       it  chooses  its FQDN as that identifier. The minion id can be overridden via the minion's
       id configuration setting.

       TIP:
          minion id and minion keys

          The minion id is used to generate the minion's  public/private  keys  and  if  it  ever
          changes the master must then accept the new key as though the minion was a new host.

   Globbing
       The default matching that Salt utilizes is shell-style globbing around the minion id. This
       also works for states in the top file.

       NOTE:
          You must wrap salt calls that use globbing in single-quotes to prevent the  shell  from
          expanding the globs before Salt is invoked.

       Match all minions:

          salt '*' test.ping

       Match all minions in the example.net domain or any of the example domains:

          salt '*.example.net' test.ping
          salt '*.example.*' test.ping

       Match all the webN minions in the example.net domain (web1.example.net, web2.example.net …
       webN.example.net):

          salt 'web?.example.net' test.ping

       Match the web1 through web5 minions:

          salt 'web[1-5]' test.ping

       Match the web1 and web3 minions:

          salt 'web[1,3]' test.ping

       Match the web-x, web-y, and web-z minions:

          salt 'web-[x-z]' test.ping

       NOTE:
          For additional targeting methods please review the compound matchers documentation.

   Regular Expressions
       Minions can be matched using Perl-compatible regular expressions  (which  is  globbing  on
       steroids and a ton of caffeine).

       Match both web1-prod and web1-devel minions:

          salt -E 'web1-(prod|devel)' test.ping

       When  using regular expressions in a State's top file, you must specify the matcher as the
       first option. The  following  example  executes  the  contents  of  webserver.sls  on  the
       above-mentioned minions.

          base:
            'web1-(prod|devel)':
            - match: pcre
            - webserver

   Lists
       At the most basic level, you can specify a flat list of minion IDs:

          salt -L 'web1,web2,web3' test.ping

   Grains
       Salt  comes  with an interface to derive information about the underlying system.  This is
       called the grains interface, because it presents salt with grains of  information.  Grains
       are  collected for the operating system, domain name, IP address, kernel, OS type, memory,
       and many other system properties.

       The grains interface is made available to Salt modules and components so  that  the  right
       salt minion commands are automatically available on the right systems.

       Grain  data  is  relatively  static, though if system information changes (for example, if
       network settings are changed), or if a new value is assigned to a custom grain, grain data
       is refreshed.

       NOTE:
          Grains resolve to lowercase letters. For example, FOO, and foo target the same grain.

       IMPORTANT:
          See Is Targeting using Grain Data Secure? for important security information.

       Match all CentOS minions:

          salt -G 'os:CentOS' test.ping

       Match  all  minions  with  64-bit  CPUs,  and return number of CPU cores for each matching
       minion:

          salt -G 'cpuarch:x86_64' grains.item num_cpus

       Additionally, globs can be used in  grain  matches,  and  grains  that  are  nested  in  a
       dictionary can be matched by adding a colon for each level that is traversed. For example,
       the following will match hosts that have a grain called ec2_tags, which itself is  a  dict
       with a key named environment, which has a value that contains the word production:

          salt -G 'ec2_tags:environment:*production*'

   Listing Grains
       Available grains can be listed by using the 'grains.ls' module:

          salt '*' grains.ls

       Grains data can be listed by using the 'grains.items' module:

          salt '*' grains.items

   Grains in the Minion Config
       Grains can also be statically assigned within the minion configuration file.  Just add the
       option grains and pass options to it:

          grains:
            roles:
              - webserver
              - memcache
            deployment: datacenter4
            cabinet: 13
            cab_u: 14-15

       Then status data specific to your servers can be retrieved via Salt, or used inside of the
       State  system  for  matching.  It  also makes targeting, in the case of the example above,
       simply based on specific data about your deployment.

   Grains in /etc/salt/grains
       If you do not want to place your custom static grains in the minion config file,  you  can
       also put them in /etc/salt/grains on the minion. They are configured in the same way as in
       the above example, only without a top-level grains: key:

          roles:
            - webserver
            - memcache
          deployment: datacenter4
          cabinet: 13
          cab_u: 14-15

   Matching Grains in the Top File
       With correctly configured grains on the Minion, the top file  used  in  Pillar  or  during
       Highstate can be made very efficient. For example, consider the following configuration:

          'node_type:web':
            - match: grain
            - webserver

          'node_type:postgres':
            - match: grain
            - database

          'node_type:redis':
            - match: grain
            - redis

          'node_type:lb':
            - match: grain
            - lb

       For  this  example  to  work,  you  would need to have defined the grain node_type for the
       minions you wish to match. This simple example is nice,  but  too  much  of  the  code  is
       similar. To go one step further, Jinja templating can be used to simplify the top file.

          {% set the_node_type = salt['grains.get']('node_type', '') %}

          {% if the_node_type %}
            'node_type:{{ the_node_type }}':
              - match: grain
              - {{ the_node_type }}
          {% endif %}

       Using Jinja templating, only one match entry needs to be defined.

       NOTE:
          The example above uses the grains.get function to account for minions which do not have
          the node_type grain set.

   Writing Grains
       The grains interface is derived by executing all of the "public" functions  found  in  the
       modules located in the grains package or the custom grains directory. The functions in the
       modules of the grains must return a Python dict, where the keys in the dict are the  names
       of the grains and the values are the values.

       Custom  grains  should  be  placed  in  a  _grains  directory located under the file_roots
       specified by the master config file.  The default path would be /srv/salt/_grains.  Custom
       grains will be distributed to the minions when state.highstate is run, or by executing the
       saltutil.sync_grains or saltutil.sync_all functions.

       Grains are easy to write, and only need to return a dictionary.  A common  approach  would
       be code something similar to the following:

          #!/usr/bin/env python
          def yourfunction():
               # initialize a grains dictionary
               grains = {}
               # Some code for logic that sets grains like
               grains['yourcustomgrain'] = True
               grains['anothergrain'] = 'somevalue'
               return grains

       Before adding a grain to Salt, consider what the grain is and remember that grains need to
       be static data. If the data is something that is likely to change, consider  using  Pillar
       instead.

       WARNING:
          Custom grains will not be available in the top file until after the first highstate. To
          make custom grains available on a minion's first highstate, it is  recommended  to  use
          this example to ensure that the custom grains are synced when the minion starts.

   Loading Custom Grains
       If  you have multiple functions specifying grains that are called from a main function, be
       sure to prepend grain function names with an underscore. This prevents Salt from including
       the loaded grains from the grain functions in the final grain data structure. For example,
       consider this custom grain file:

          #!/usr/bin/env python
          def _my_custom_grain():
              my_grain = {'foo': 'bar', 'hello': 'world'}
              return my_grain

          def main():
              # initialize a grains dictionary
              grains = {}
              grains['my_grains'] = _my_custom_grain()
              return grains

       The output of this example renders like so:

          # salt-call --local grains.items
          local:
              ----------
              <Snipped for brevity>
              my_grains:
                  ----------
                  foo:
                      bar
                  hello:
                      world

       However, if you don't  prepend  the  my_custom_grain  function  with  an  underscore,  the
       function  will be rendered twice by Salt in the items output: once for the my_custom_grain
       call itself, and again when it is called in the main function:

          # salt-call --local grains.items
          local:
          ----------
              <Snipped for brevity>
              foo:
                  bar
              <Snipped for brevity>
              hello:
                  world
              <Snipped for brevity>
              my_grains:
                  ----------
                  foo:
                      bar
                  hello:
                      world

   Precedence
       Core grains can be overridden by custom grains. As there  are  several  ways  of  defining
       custom  grains, there is an order of precedence which should be kept in mind when defining
       them. The order of evaluation is as follows:

       1. Core grains.

       2. Custom grains in /etc/salt/grains.

       3. Custom grains in /etc/salt/minion.

       4. Custom grain modules in _grains directory, synced to minions.

       Each successive evaluation overrides the previous ones, so any grains  defined  by  custom
       grains  modules  synced  to  minions that have the same name as a core grain will override
       that core grain. Similarly, grains from /etc/salt/minion override  both  core  grains  and
       custom grain modules, and grains in _grains will override any grains of the same name.

   Examples of Grains
       The  core  module  in  the  grains package is where the main grains are loaded by the Salt
       minion and provides the principal example of how to write grains:

       https://github.com/saltstack/salt/blob/develop/salt/grains/core.py

   Syncing Grains
       Syncing grains can  be  done  a  number  of  ways,  they  are  automatically  synced  when
       state.highstate  is  called,  or  (as  noted  above) the grains can be manually synced and
       reloaded by calling the saltutil.sync_grains or saltutil.sync_all functions.

   Targeting with Pillar
       Pillar data can be used when targeting minions.  This  allows  for  ultimate  control  and
       flexibility when targeting minions.

          salt -I 'somekey:specialvalue' test.ping

       Like with Grains, it is possible to use globbing as well as match nested values in Pillar,
       by adding colons for each level that is being traversed. The  below  example  would  match
       minions  with  a  pillar  named  foo,  which  is a dict containing a key bar, with a value
       beginning with baz:

          salt -I 'foo:bar:baz*' test.ping

   Subnet/IP Address Matching
       Minions can easily be matched based on IP address, or by subnet (using CIDR notation).

          salt -S 192.168.40.20 test.ping
          salt -S 10.0.0.0/24 test.ping

       Ipcidr matching can also be used in compound matches

          salt -C 'S@10.0.0.0/24 and G@os:Debian' test.ping

       It is also possible to use in both pillar and state-matching

          '172.16.0.0/12':
             - match: ipcidr
             - internal

       NOTE:
          Only IPv4 matching is supported at this time.

   Compound matchers
       Compound matchers allow very granular minion targeting using any of Salt's  matchers.  The
       default  matcher  is  a glob match, just as with CLI and top file matching. To match using
       anything other than a glob, prefix the match string with the appropriate letter  from  the
       table below, followed by an @ sign.

        ┌───────┬───────────────────┬──────────────────────────────────────────┬────────────────┐
        │Letter │ Match Type        │ Example                                  │ Alt Delimiter? │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │G      │ Grains glob       │ G@os:Ubuntu                              │ Yes            │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │E      │ PCRE Minion ID    │ E@web\d+\.(dev|qa|prod)\.loc             │ No             │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │P      │ Grains PCRE       │ P@os:(RedHat|Fedora|CentOS)              │ Yes            │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │L      │ List of minions   │ L@minion1.example.com,minion3.domain.com │ No             │
        │       │                   │ or bl*.domain.com                        │                │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │I      │ Pillar glob       │ I@pdata:foobar                           │ Yes            │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │J      │ Pillar PCRE       │ J@pdata:^(foo|bar)$                      │ Yes            │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │S      │ Subnet/IP address │ S@192.168.1.0/24 or S@192.168.1.100      │ No             │
        ├───────┼───────────────────┼──────────────────────────────────────────┼────────────────┤
        │R      │ Range cluster     │ R@%foo.bar                               │ No             │
        └───────┴───────────────────┴──────────────────────────────────────────┴────────────────┘

       Matchers can be joined using boolean and, or, and not operators.

       For  example,  the following string matches all Debian minions with a hostname that begins
       with webserv, as well as any minions that  have  a  hostname  which  matches  the  regular
       expression web-dc1-srv.*:

          salt -C 'webserv* and G@os:Debian or E@web-dc1-srv.*' test.ping

       That same example expressed in a top file looks like the following:

          base:
            'webserv* and G@os:Debian or E@web-dc1-srv.*':
              - match: compound
              - webserver

       New in version 2015.8.0.

       Excluding a minion based on its ID is also possible:

          salt -C 'not web-dc1-srv' test.ping

       Versions  prior  to 2015.8.0 a leading not was not supported in compound matches. Instead,
       something like the following was required:

          salt -C '* and not G@kernel:Darwin' test.ping

       Excluding a minion based on its ID was also possible:

          salt -C '* and not web-dc1-srv' test.ping

   Precedence Matching
       Matchers can be grouped together with parentheses to explicitly declare precedence amongst
       groups.

          salt -C '( ms-1 or G@id:ms-3 ) and G@id:ms-3' test.ping

       NOTE:
          Be  certain  to  note  that  spaces  are  required between the parentheses and targets.
          Failing to obey this rule may result in incorrect targeting!

   Alternate Delimiters
       New in version 2015.8.0.

       Matchers that target based on a key value pair use a colon (:) as  a  delimiter.  Matchers
       with  a  Yes  in  the  Alt  Delimiters  column in the previous table support specifying an
       alternate delimiter character.

       This is done by specifying an alternate delimiter character between  the  leading  matcher
       character  and  the @ pattern separator character. This avoids incorrect interpretation of
       the pattern in the case that : is part of the grain or pillar data structure traversal.

          salt -C 'J|@foo|bar|^foo:bar$ or J!@gitrepo!https://github.com:example/project.git' test.ping

   Node groups
       Nodegroups are declared  using  a  compound  target  specification.  The  compound  target
       documentation can be found here.

       The  nodegroups  master  config  file  parameter  is  used to define nodegroups. Here's an
       example nodegroup configuration within /etc/salt/master:

          nodegroups:
            group1: 'L@foo.domain.com,bar.domain.com,baz.domain.com or bl*.domain.com'
            group2: 'G@os:Debian and foo.domain.com'
            group3: 'G@os:Debian and N@group1'
            group4:
              - 'G@foo:bar'
              - 'or'
              - 'G@foo:baz'

       NOTE:
          The L within group1 is matching a list of minions, while the G in  group2  is  matching
          specific grains. See the compound matchers documentation for more details.

       New in version 2015.8.0.

       NOTE:
          Nodgroups  can  reference  other  nodegroups as seen in group3.  Ensure that you do not
          have circular references.  Circular references  will  be  detected  and  cause  partial
          expansion with a logged error message.

       New in version 2015.8.0.

       Compound  nodegroups  can  be  either  string  values or lists of string values.  When the
       nodegroup is A string value will be tokenized by splitting on whitespace.  This may  be  a
       problem  if  whitespace  is necessary as part of a pattern.  When a nodegroup is a list of
       strings then tokenization will happen for each list element as a whole.

       To match a nodegroup on the CLI, use the -N command-line option:

          salt -N group1 test.ping

       NOTE:
          The N@ classifier cannot be used in compound mathes within the CLI or top file,  it  is
          only recognized in the nodegroups master config file parameter.

       To  match  a  nodegroup  in your top file, make sure to put - match: nodegroup on the line
       directly following the nodegroup name.

          base:
            group1:
              - match: nodegroup
              - webserver

       NOTE:
          When adding or modifying nodegroups to a master configuration file, the master must  be
          restarted for those changes to be fully recognized.

          A  limited amount of functionality, such as targeting with -N from the command-line may
          be available without a restart.

   Using Nodegroups in SLS files
       To use Nodegroups in Jinja logic for SLS files, the pillar_opts option in /etc/salt/master
       must  be  set  to "True". This will pass the master's configuration as Pillar data to each
       minion.

       NOTE:
          If the master's configuration contains any sensitive data, this will be passed to  each
          minion.   Do  not enable this option if you have any configuration data that you do not
          want to get on your minions.

          Also, if you make  changes  to  your  nodegroups,  you  might  need  to  run  salt  '*'
          saltutil.refresh_pillar after restarting the master.

       Once  pillar_opts  is  enabled, you can find the nodegroups under the "master" pillar.  To
       make sure that only the correct minions are targeted, you should use each matcher for  the
       nodegroup definition.  For example, to check if a minion is in the 'webserver' nodegroup:

          nodegroups:
            webserver: 'G@os:Debian and L@minion1,minion2'

          {% if grains.id in salt['pillar.get']('master:nodegroups:webserver', [])
          and grains.os in salt['pillar.get']('master:nodegroups:webserver', []) %}
          ...
          {% endif %}

       NOTE:
          If  you  do  not  include  all  of  the matchers used to define a nodegroup, Salt might
          incorrectly target minions that meet some of the nodegroup requirements, but not all of
          them.

   Batch Size
       The  -b (or --batch-size) option allows commands to be executed on only a specified number
       of minions at a time. Both percentages and finite numbers are supported.

          salt '*' -b 10 test.ping

          salt -G 'os:RedHat' --batch-size 25% apache.signal restart

       This will only run test.ping on 10 of the targeted minions at  a  time  and  then  restart
       apache  on 25% of the minions matching os:RedHat at a time and work through them all until
       the task is complete. This makes jobs like rolling  web  server  restarts  behind  a  load
       balancer or doing maintenance on BSD firewalls using carp much easier with salt.

       The  batch  system  maintains a window of running minions, so, if there are a total of 150
       minions targeted and the batch size is 10, then the command is sent to  10  minions,  when
       one  minion  returns then the command is sent to one additional minion, so that the job is
       constantly running on 10 minions.

   SECO Range
       SECO range is a cluster-based metadata store developed and maintained by Yahoo!

       The Range project is hosted here:

       https://github.com/ytoolshed/range

       Learn more about range here:

       https://github.com/ytoolshed/range/wiki/

   Prerequisites
       To utilize range support in Salt, a range server is required. Setting up a range server is
       outside the scope of this document. Apache modules are included in the range distribution.

       With  a  working  range  server, cluster files must be defined. These files are written in
       YAML and define hosts contained inside a cluster. Full documentation on writing YAML range
       files is here:

       https://github.com/ytoolshed/range/wiki/%22yamlfile%22-module-file-spec

       Additionally,  the  Python  seco range libraries must be installed on the salt master. One
       can verify that they have been installed correctly via the following command:

          python -c 'import seco.range'

       If no errors are returned, range is installed successfully on the salt master.

   Preparing Salt
       Range support must be enabled on the salt master by setting the hostname and port  of  the
       range server inside the master configuration file:

          range_server: my.range.server.com:80

       Following this, the master must be restarted for the change to have an effect.

   Targeting with Range
       Once a cluster has been defined, it can be targeted with a salt command by using the -R or
       --range flags.

       For example, given the following range YAML file being served from a range server:

          $ cat /etc/range/test.yaml
          CLUSTER: host1..100.test.com
          APPS:
            - frontend
            - backend
            - mysql

       One might target host1 through host100 in the test.com domain with Salt as follows:

          salt --range %test:CLUSTER test.ping

       The following salt command would target three hosts: frontend, backend, and mysql:

          salt --range %test:APPS test.ping

STORING STATIC DATA IN THE PILLAR

       Pillar is an interface for Salt designed to offer global values that can be distributed to
       all minions. Pillar data is managed in a similar way as the Salt State Tree.

       Pillar was added to Salt in version 0.9.8

       NOTE:
          Storing sensitive data

          Unlike  state  tree, pillar data is only available for the targeted minion specified by
          the matcher type.  This makes it useful  for  storing  sensitive  data  specific  to  a
          particular minion.

   Declaring the Master Pillar
       The  Salt  Master  server maintains a pillar_roots setup that matches the structure of the
       file_roots used in the Salt file server. Like the Salt file server the pillar_roots option
       in  the  master config is based on environments mapping to directories. The pillar data is
       then mapped to minions based on matchers in a top file which is laid out in the  same  way
       as  the  state  top  file. Salt pillars can use the same matcher types as the standard top
       file.

       The configuration for the pillar_roots in the master config file is identical in  behavior
       and function as file_roots:

          pillar_roots:
            base:
              - /srv/pillar

       This  example  configuration  declares  that  the  base environment will be located in the
       /srv/pillar directory. It must not be in a subdirectory of the state tree.

       The top file used matches the name of the top file used  for  States,  and  has  the  same
       structure:

       /srv/pillar/top.sls

          base:
            '*':
              - packages

       In  the above top file, it is declared that in the base environment, the glob matching all
       minions will have the pillar data found in the packages pillar available to  it.  Assuming
       the  pillar_roots  value  of  /srv/pillar  taken  from above, the packages pillar would be
       located at /srv/pillar/packages.sls.

       Any number of matchers can be added to the base  environment.  For  example,  here  is  an
       expanded version of the Pillar top file stated above:

       /srv/pillar/top.sls:

          base:
            '*':
              - packages
            'web*':
              - vim

       In   this   expanded   top  file,  minions  that  match  web*  will  have  access  to  the
       /srv/pillar/pacakges.sls file, as well as the /srv/pillar/vim.sls file.

       Another example shows how to use other standard top matching  types  to  deliver  specific
       salt pillar data to minions with different properties.

       Here  is  an  example  using  the  grains matcher to target pillars to minions by their os
       grain:

          dev:
            'os:Debian':
              - match: grain
              - servers

       /srv/pillar/packages.sls

          {% if grains['os'] == 'RedHat' %}
          apache: httpd
          git: git
          {% elif grains['os'] == 'Debian' %}
          apache: apache2
          git: git-core
          {% endif %}

          company: Foo Industries

       IMPORTANT:
          See Is Targeting using Grain Data Secure? for important security information.

       The above pillar sets two key/value pairs. If a minion is running RedHat, then the  apache
       key  is  set to httpd and the git key is set to the value of git. If the minion is running
       Debian, those values are changed to apache2 and git-core respectively.  All  minions  that
       have  this  pillar  targeting  to  them via a top file will have the key of company with a
       value of Foo Industries.

       Consequently this data can be used from within modules, renderers, State  SLS  files,  and
       more via the shared pillar dict:

          apache:
            pkg.installed:
              - name: {{ pillar['apache'] }}

          git:
            pkg.installed:
              - name: {{ pillar['git'] }}

       Finally,  the above states can utilize the values provided to them via Pillar.  All pillar
       values targeted to a minion are available via the 'pillar'  dictionary.  As  seen  in  the
       above  example,  Jinja  substitution can then be utilized to access the keys and values in
       the Pillar dictionary.

       Note that you cannot just list key/value-information in top.sls. Instead, target a  minion
       to  a  pillar file and then list the keys and values in the pillar. Here is an example top
       file that illustrates this point:

          base:
            '*':
               - common_pillar

       And the actual pillar file at '/srv/pillar/common_pillar.sls':

          foo: bar
          boo: baz

   Pillar namespace flattened
       The separate pillar files all share the same namespace. Given a top.sls of:

          base:
            '*':
              - packages
              - services

       a packages.sls file of:

          bind: bind9

       and a services.sls file of:

          bind: named

       Then a request for the bind pillar  will  only  return  named;  the  bind9  value  is  not
       available.  It  is  better to structure your pillar files with more hierarchy. For example
       your package.sls file could look like:

          packages:
            bind: bind9

   Pillar Namespace Merges
       With some care, the pillar namespace can merge content from multiple pillar files under  a
       single key, so long as conflicts are avoided as described above.

       For  example, if the above example were modified as follows, the values are merged below a
       single key:

          base:
            '*':
              - packages
              - services

       And a packages.sls file like:

          bind:
            package-name: bind9
            version: 9.9.5

       And a services.sls file like:

          bind:
            port: 53
            listen-on: any

       The resulting pillar will be as follows:

          $ salt-call pillar.get bind
          local:
              ----------
              listen-on:
                  any
              package-name:
                  bind9
              port:
                  53
              version:
                  9.9.5

       NOTE:
          Pillar files are applied in the order they are  listed  in  the  top  file.   Therefore
          conflicting  keys will be overwritten in a 'last one wins' manner!  For example, in the
          above scenario conflicting key values in services  will  overwrite  those  in  packages
          because it's at the bottom of the list.

   Including Other Pillars
       New in version 0.16.0.

       Pillar  SLS files may include other pillar files, similar to State files. Two syntaxes are
       available for this purpose. The simple form simply includes the additional pillar as if it
       were part of the same file:

          include:
            - users

       The  full  include  form  allows  two  additional options -- passing default values to the
       templating engine for the included pillar file as well as an optional key under  which  to
       nest the results of the included pillar:

          include:
            - users:
                defaults:
                    sudo: ['bob', 'paul']
                key: users

       With this form, the included file (users.sls) will be nested within the 'users' key of the
       compiled pillar. Additionally, the 'sudo' value will be available as a  template  variable
       to users.sls.

   Viewing Minion Pillar
       Once  the pillar is set up the data can be viewed on the minion via the pillar module, the
       pillar module comes with functions, pillar.items and pillar.raw.  pillar.items will return
       a freshly reloaded pillar and pillar.raw will return the current pillar without a refresh:

          salt '*' pillar.items

       NOTE:
          Prior  to  version  0.16.2,  this  function is named pillar.data. This function name is
          still supported for backwards compatibility.

   Pillar get Function
       New in version 0.14.0.

       The pillar.get function works much in the same way as the get method in a python dict, but
       with an enhancement: nested dict components can be extracted using a : delimiter.

       If a structure like this is in pillar:

          foo:
            bar:
              baz: qux

       Extracting it from the raw pillar in an sls formula or file template is done this way:

          {{ pillar['foo']['bar']['baz'] }}

       Now, with the new pillar.get function the data can be safely gathered and a default can be
       set, allowing the template to fall back if the value is not available:

          {{ salt['pillar.get']('foo:bar:baz', 'qux') }}

       This makes handling nested structures much easier.

       NOTE:
          pillar.get() vs salt['pillar.get']()

          It should be noted that within templating, the pillar variable is  just  a  dictionary.
          This  means  that  calling  pillar.get() inside of a template will just use the default
          dictionary .get() function which does not include the extra : delimiter  functionality.
          It  must be called using the above syntax (salt['pillar.get']('foo:bar:baz', 'qux')) to
          get the salt function, instead of the default dictionary behavior.

   Refreshing Pillar Data
       When pillar data is changed on the master the minions need to refresh  the  data  locally.
       This is done with the saltutil.refresh_pillar function.

          salt '*' saltutil.refresh_pillar

       This  function  triggers  the  minion to asynchronously refresh the pillar and will always
       return None.

   Set Pillar Data at the Command Line
       Pillar data can be set at the command line like the following example:

          salt '*' state.highstate pillar='{"cheese": "spam"}'

       This will create a dict with a key of 'cheese' and a  value  of  'spam'.  A  list  can  be
       created like this:

          salt '*' state.highstate pillar='["cheese", "milk", "bread"]'

       NOTE:
          Be  aware  that  when  sending  sensitive  data via pillar on the command-line that the
          publication containing that data will be received  by  all  minions  and  will  not  be
          restricted  to  the  targeted  minions.  This  may represent a security concern in some
          cases.

   Master Config In Pillar
       For convenience the data stored in the master configuration file can be made available  in
       all  minion's  pillars.  This makes global configuration of services and systems very easy
       but may not be desired if sensitive data is  stored  in  the  master  configuration.  This
       option is disabled by default.

       To enable the master config from being added to the pillar set pillar_opts to True:

          pillar_opts: True

   Minion Config in Pillar
       Minion  configuration  options  can be set on pillars. Any option that you want to modify,
       should be in the first level of the pillars, in the same way you set the  options  in  the
       config  file.  For  example, to configure the MySQL root password to be used by MySQL Salt
       execution module, set the following pillar variable:

          mysql.pass: hardtoguesspassword

   Master Provided Pillar Error
       By default if there is an error rendering a pillar,  the  detailed  error  is  hidden  and
       replaced with:

          Rendering SLS 'my.sls' failed. Please see master log for details.

       The  error  is protected because it's possible to contain templating data which would give
       that minion information it shouldn't know, like a password!

       To have the master provide the detailed error that could potentially carry protected  data
       set pillar_safe_render_error to False:

          pillar_safe_render_error: False

REACTOR SYSTEM

       Salt  version  0.11.0 introduced the reactor system. The premise behind the reactor system
       is that with Salt's events and the ability to execute commands, a logic  engine  could  be
       put in place to allow events to trigger actions, or more accurately, reactions.

       This  system  binds  sls  files  to  event tags on the master. These sls files then define
       reactions. This means that the reactor system has two parts.  First,  the  reactor  option
       needs  to  be  set  in the master configuration file.  The reactor option allows for event
       tags to be associated with sls reaction files. Second, these reaction files  use  highdata
       (like the state system) to define reactions to be executed.

   Event System
       A  basic  understanding of the event system is required to understand reactors.  The event
       system is a local ZeroMQ PUB interface which fires salt events. This event bus is an  open
       system used for sending information notifying Salt and other systems about operations.

       The  event system fires events with a very specific criteria. Every event has a tag. Event
       tags allow for fast top level filtering of events. In addition to the tag, each event  has
       a  data  structure.  This  data  structure is a dict, which contains information about the
       event.

   Mapping Events to Reactor SLS Files
       Reactor SLS files and event tags are associated in the master  config  file.   By  default
       this is /etc/salt/master, or /etc/salt/master.d/reactor.conf.

       New in version 2014.7.0: Added Reactor support for salt:// file paths.

       In  the  master  config  section 'reactor:' is a list of event tags to be matched and each
       event tag has a list of reactor SLS files to be run.

          reactor:                            # Master config section "reactor"

            - 'salt/minion/*/start':          # Match tag "salt/minion/*/start"
              - /srv/reactor/start.sls        # Things to do when a minion starts
              - /srv/reactor/monitor.sls      # Other things to do

            - 'salt/cloud/*/destroyed':       # Globs can be used to matching tags
              - /srv/reactor/destroy/*.sls    # Globs can be used to match file names

            - 'myco/custom/event/tag':        # React to custom event tags
              - salt://reactor/mycustom.sls   # Put reactor files under file_roots

       Reactor sls files are similar to state and pillar sls files.  They are by default  yaml  +
       Jinja templates and are passed familiar context variables.

       They differ because of the addition of the tag and data variables.

       • The tag variable is just the tag in the fired event.

       • The data variable is the event's data dict.

       Here is a simple reactor sls:

          {% if data['id'] == 'mysql1' %}
          highstate_run:
            local.state.highstate:
              - tgt: mysql1
          {% endif %}

       This  simple reactor file uses Jinja to further refine the reaction to be made.  If the id
       in the event data is mysql1 (in other words, if the name of the minion is mysql1) then the
       following  reaction  is  defined.  The same data structure and compiler used for the state
       system is used for the reactor system. The only difference is that the data is matched  up
       to the salt command API and the runner system.  In this example, a command is published to
       the mysql1 minion with a function of state.highstate. Similarly, a runner can be called:

          {% if data['data']['orchestrate'] == 'refresh' %}
          orchestrate_run:
            runner.state.orchestrate
          {% endif %}

       This example will  execute  the  state.orchestrate  runner  and  initiate  an  orchestrate
       execution.

   Fire an event
       To fire an event from a minion call event.send

          salt-call event.send 'foo' '{orchestrate: refresh}'

       After  this  is  called, any reactor sls files matching event tag foo will execute with {{
       data['data']['orchestrate'] }} equal to 'refresh'.

       See salt.modules.event for more information.

   Knowing what event is being fired
       The best way to see exactly what events are fired and what data is available in each event
       is to use the state.event runner.

       SEE ALSO:
          Common Salt Events

       Example usage:

          salt-run state.event pretty=True

       Example output:

          salt/job/20150213001905721678/new       {
              "_stamp": "2015-02-13T00:19:05.724583",
              "arg": [],
              "fun": "test.ping",
              "jid": "20150213001905721678",
              "minions": [
                  "jerry"
              ],
              "tgt": "*",
              "tgt_type": "glob",
              "user": "root"
          }
          salt/job/20150213001910749506/ret/jerry {
              "_stamp": "2015-02-13T00:19:11.136730",
              "cmd": "_return",
              "fun": "saltutil.find_job",
              "fun_args": [
                  "20150213001905721678"
              ],
              "id": "jerry",
              "jid": "20150213001910749506",
              "retcode": 0,
              "return": {},
              "success": true
          }

   Debugging the Reactor
       The best window into the Reactor is to run the master in the foreground with debug logging
       enabled. The output will include when the master sees the event, what the master  does  in
       response  to  that  event,  and  it will also include the rendered SLS file (or any errors
       generated while rendering the SLS file).

       1. Stop the master.

       2. Start the master manually:

             salt-master -l debug

       3. Look for log entries in the form:

             [DEBUG   ] Gathering reactors for tag foo/bar
             [DEBUG   ] Compiling reactions for tag foo/bar
             [DEBUG   ] Rendered data from file: /path/to/the/reactor_file.sls:
             <... Rendered output appears here. ...>

          The rendered output is the result of the Jinja parsing and is a good way  to  view  the
          result  of  referencing  Jinja variables. If the result is empty then Jinja produced an
          empty result and the Reactor will ignore it.

   Understanding the Structure of Reactor Formulas
       I.e., when to use `arg` and `kwarg` and when to specify the function arguments directly.

       While the reactor system uses the same basic data  structure  as  the  state  system,  the
       functions  that  will be called using that data structure are different functions than are
       called via Salt's state system. The Reactor can  call  Runner  modules  using  the  runner
       prefix,  Wheel modules using the wheel prefix, and can also cause minions to run Execution
       modules using the local prefix.

       Changed in version 2014.7.0: The cmd prefix was renamed  to  local  for  consistency  with
       other parts of Salt. A backward-compatible alias was added for cmd.

       The  Reactor  runs on the master and calls functions that exist on the master. In the case
       of Runner and Wheel functions the Reactor can just call  those  functions  directly  since
       they exist on the master and are run on the master.

       In  the  case of functions that exist on minions and are run on minions, the Reactor still
       needs to call a function on the master in order to send the necessary data to  the  minion
       so the minion can execute that function.

       The  Reactor  calls  functions  exposed  in  Salt's Python API documentation. and thus the
       structure of Reactor files very transparently reflects the function  signatures  of  those
       functions.

   Calling Execution modules on Minions
       The  Reactor  sends  commands  down  to minions in the exact same way Salt's CLI interface
       does. It calls a function locally on the master that sends the name  of  the  function  as
       well  as a list of any arguments and a dictionary of any keyword arguments that the minion
       should use to execute that function.

       Specifically, the Reactor calls the async version of  this  function.  You  can  see  that
       function  has 'arg' and 'kwarg' parameters which are both values that are sent down to the
       minion.

       Executing remote commands maps to the LocalClient interface which  is  used  by  the  salt
       command.  This  interface  more  specifically  maps  to the cmd_async method inside of the
       LocalClient class. This means that the arguments passed are being passed to the  cmd_async
       method, not the remote method. A field starts with local to use the LocalClient subsystem.
       The result is, to execute a remote command, a reactor formula would look like this:

          clean_tmp:
            local.cmd.run:
              - tgt: '*'
              - arg:
                - rm -rf /tmp/*

       The arg option takes a list of arguments as they would be presented on the  command  line,
       so the above declaration is the same as running this salt command:

          salt '*' cmd.run 'rm -rf /tmp/*'

       Use the expr_form argument to specify a matcher:

          clean_tmp:
            local.cmd.run:
              - tgt: 'os:Ubuntu'
              - expr_form: grain
              - arg:
                - rm -rf /tmp/*

          clean_tmp:
            local.cmd.run:
              - tgt: 'G@roles:hbase_master'
              - expr_form: compound
              - arg:
                - rm -rf /tmp/*

       Any other parameters in the LocalClient().cmd() method can be specified as well.

   Calling Runner modules and Wheel modules
       Calling  Runner modules and Wheel modules from the Reactor uses a more direct syntax since
       the function is being executed locally instead of sending a command to a remote system  to
       be executed there. There are no 'arg' or 'kwarg' parameters (unless the Runner function or
       Wheel function accepts a parameter with either of those names.)

       For example:

          clear_the_grains_cache_for_all_minions:
            runner.cache.clear_grains

       If the runner takes arguments then they can be specified as well:

          spin_up_more_web_machines:
            runner.cloud.profile:
              - prof: centos_6
              - instances:
                - web11       # These VM names would be generated via Jinja in a
                - web12       # real-world example.

   Passing event data to Minions or Orchestrate as Pillar
       An interesting trick to pass data from the Reactor script to state.highstate or  state.sls
       is  to  pass  it  as inline Pillar data since both functions take a keyword argument named
       pillar.

       The following example uses Salt's Reactor to listen for the event that is fired  when  the
       key for a new minion is accepted on the master using salt-key.

       /etc/salt/master.d/reactor.conf:

          reactor:
            - 'salt/key':
              - /srv/salt/haproxy/react_new_minion.sls

       The  Reactor then fires a state.sls command targeted to the HAProxy servers and passes the
       ID of the new minion from the event to the state file via inline Pillar.

       /srv/salt/haproxy/react_new_minion.sls:

          {% if data['act'] == 'accept' and data['id'].startswith('web') %}
          add_new_minion_to_pool:
            local.state.sls:
              - tgt: 'haproxy*'
              - arg:
                - haproxy.refresh_pool
              - kwarg:
                  pillar:
                    new_minion: {{ data['id'] }}
          {% endif %}

       The above command is equivalent to the following command at the CLI:

          salt 'haproxy*' state.sls haproxy.refresh_pool 'pillar={new_minion: minionid}'

       This works with Orchestrate files as well:

          call_some_orchestrate_file:
            runner.state.orchestrate:
              - mods: some_orchestrate_file
              - pillar:
                  stuff: things

       Which is equivalent to the following command at the CLI:

          salt-run state.orchestrate some_orchestrate_file pillar='{stuff: things}'

       Finally, that data is available in the state file using the normal Pillar  lookup  syntax.
       The  following  example  is  grabbing web server names and IP addresses from Salt Mine. If
       this state is invoked from the Reactor then the custom Pillar value  from  above  will  be
       available  and the new minion will be added to the pool but with the disabled flag so that
       HAProxy won't yet direct traffic to it.

       /srv/salt/haproxy/refresh_pool.sls:

          {% set new_minion = salt['pillar.get']('new_minion') %}

          listen web *:80
              balance source
              {% for server,ip in salt['mine.get']('web*', 'network.interfaces', ['eth0']).items() %}
              {% if server == new_minion %}
              server {{ server }} {{ ip }}:80 disabled
              {% else %}
              server {{ server }} {{ ip }}:80 check
              {% endif %}
              {% endfor %}

   A Complete Example
       In this example, we're going to assume that we have a group  of  servers  that  will  come
       online  at  random  and  need  to have keys automatically accepted. We'll also add that we
       don't want all servers being automatically accepted. For this example, we'll  assume  that
       all  hosts  that have an id that starts with 'ink' will be automatically accepted and have
       state.highstate executed. On top of this, we're going to add that a host  coming  up  that
       was replaced (meaning a new key) will also be accepted.

       Our  master configuration will be rather simple. All minions that attempte to authenticate
       will match the tag of salt/auth. When it comes to the minion key being accepted, we get  a
       more refined tag that includes the minion id, which we can use for matching.

       /etc/salt/master.d/reactor.conf:

          reactor:
            - 'salt/auth':
              - /srv/reactor/auth-pending.sls
            - 'salt/minion/ink*/start':
              - /srv/reactor/auth-complete.sls

       In this sls file, we say that if the key was rejected we will delete the key on the master
       and then also tell the master to ssh in to the minion and tell it to restart  the  minion,
       since a minion process will die if the key is rejected.

       We  also say that if the key is pending and the id starts with ink we will accept the key.
       A minion that is waiting on a pending key will retry authentication every ten  seconds  by
       default.

       /srv/reactor/auth-pending.sls:

          {# Ink server faild to authenticate -- remove accepted key #}
          {% if not data['result'] and data['id'].startswith('ink') %}
          minion_remove:
            wheel.key.delete:
              - match: {{ data['id'] }}
          minion_rejoin:
            local.cmd.run:
              - tgt: salt-master.domain.tld
              - arg:
                - ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no "{{ data['id'] }}" 'sleep 10 && /etc/init.d/salt-minion restart'
          {% endif %}

          {# Ink server is sending new key -- accept this key #}
          {% if 'act' in data and data['act'] == 'pend' and data['id'].startswith('ink') %}
          minion_add:
            wheel.key.accept:
              - match: {{ data['id'] }}
          {% endif %}

       No  if  statements  are  needed  here  because  we already limited this action to just Ink
       servers in the master configuration.

       /srv/reactor/auth-complete.sls:

          {# When an Ink server connects, run state.highstate. #}
          highstate_run:
            local.state.highstate:
              - tgt: {{ data['id'] }}
              - ret: smtp

       The above will also return the highstate result data using the smtp_return  returner  (use
       virtualname  like  when using from the command line with --return).  The returner needs to
       be  configured  on  the  minion  for  this  to   work.    See   salt.returners.smtp_return
       documentation for that.

   Syncing Custom Types on Minion Start
       Salt  will  sync  all  custom  types  (by running a saltutil.sync_all) on every highstate.
       However, there is a chicken-and-egg issue where, on the initial highstate, a  minion  will
       not  yet  have  these custom types synced when the top file is first compiled. This can be
       worked around with a simple reactor which watches  for  minion_start  events,  which  each
       minion fires when it first starts up and connects to the master.

       On the master, create /srv/reactor/sync_grains.sls with the following contents:

          sync_grains:
            local.saltutil.sync_grains:
              - tgt: {{ data['id'] }}

       And in the master config file, add the following reactor configuration:

          reactor:
            - 'minion_start':
              - /srv/reactor/sync_grains.sls

       This  will  cause  the  master  to  instruct each minion to sync its custom grains when it
       starts, making these grains available when the initial highstate is executed.

       Other   types   can   be   synced    by    replacing    local.saltutil.sync_grains    with
       local.saltutil.sync_modules,  local.saltutil.sync_all, or whatever else suits the intended
       use case.

THE SALT MINE

       The Salt Mine is used to collect arbitrary data from Minions and store it on  the  Master.
       This data is then made available to all Minions via the salt.modules.mine module.

       Mine data is gathered on the Minion and sent back to the Master where only the most recent
       data is maintained (if long term data is  required  use  returners  or  the  external  job
       cache).

   Mine vs Grains
       Mine  data is designed to be much more up-to-date than grain data. Grains are refreshed on
       a very limited basis and are largely static data. Mines are designed to replace slow  peer
       publishing  calls  when  Minions need data from other Minions. Rather than having a Minion
       reach out to all the other Minions for a piece of data, the  Salt  Mine,  running  on  the
       Master, can collect it from all the Minions every mine-interval, resulting in almost fresh
       data at any given time, with much less overhead.

   Mine Functions
       To enable the Salt Mine the mine_functions option needs to be applied to  a  Minion.  This
       option  can  be  applied  via the Minion's configuration file, or the Minion's Pillar. The
       mine_functions option dictates what functions are being executed and allows for  arguments
       to be passed in. If no arguments are passed, an empty list must be added:

          mine_functions:
            test.ping: []
            network.ip_addrs:
              interface: eth0
              cidr: '10.0.0.0/8'

   Mine Functions Aliases
       Function  aliases  can  be  used  to  provide friendly names, usage intentions or to allow
       multiple calls of the same function with different arguments.  There is a different syntax
       for passing positional and key-value arguments.  Mixing positional and key-value arguments
       is not supported.

       New in version 2014.7.0.

          mine_functions:
            network.ip_addrs: [eth0]
            networkplus.internal_ip_addrs: []
            internal_ip_addrs:
              mine_function: network.ip_addrs
              cidr: 192.168.0.0/16
            ip_list:
              - mine_function: grains.get
              - ip_interfaces

   Mine Interval
       The Salt Mine functions are executed when the Minion starts and at a given interval by the
       scheduler. The default interval is every 60 minutes and can be adjusted for the Minion via
       the mine_interval option:

          mine_interval: 60

   Mine in Salt-SSH
       As of the 2015.5.0 release of salt, salt-ssh supports mine.get.

       Because the Minions cannot provide their own mine_functions configuration, we retrieve the
       args for specified mine functions in one of three places, searched in the following order:

       1. Roster data

       2. Pillar

       3. Master config

       The  mine_functions  are  formatted  exactly  the same as in normal salt, just stored in a
       different location. Here is an example of a flat roster containing mine_functions:

          test:
            host: 104.237.131.248
            user: root
            mine_functions:
              cmd.run: ['echo "hello!"']
              network.ip_addrs:
                interface: eth0

       NOTE:
          Because of the differences in the architecture of salt-ssh, mine.get calls are somewhat
          inefficient.  Salt  must make a new salt-ssh call to each of the Minions in question to
          retrieve the requested data, much like a publish call. However, unlike publish, it must
          run  the requested function as a wrapper function, so we can retrieve the function args
          from the pillar of the Minion in question. This  results  in  a  non-trivial  delay  in
          retrieving the requested data.

   Example
       One  way  to  use data from Salt Mine is in a State. The values can be retrieved via Jinja
       and used in the SLS file. The following example is a partial  HAProxy  configuration  file
       and  pulls  IP  addresses from all Minions with the "web" grain to add them to the pool of
       load balanced servers.

       /srv/pillar/top.sls:

          base:
            'G@roles:web':
              - web

       /srv/pillar/web.sls:

          mine_functions:
            network.ip_addrs: [eth0]

       /etc/salt/minion.d/mine.conf:

          mine_interval: 5

       /srv/salt/haproxy.sls:

          haproxy_config:
            file.managed:
              - name: /etc/haproxy/config
              - source: salt://haproxy_config
              - template: jinja

       /srv/salt/haproxy_config:

          <...file contents snipped...>

          {% for server, addrs in salt['mine.get']('roles:web', 'network.ip_addrs', expr_form='pillar').items() %}
          server {{ server }} {{ addrs[0] }}:80 check
          {% endfor %}

          <...file contents snipped...>

EXTERNAL AUTHENTICATION SYSTEM

       Salt's External Authentication System (eAuth) allows for  Salt  to  pass  through  command
       authorization to any external authentication system, such as PAM or LDAP.

       NOTE:
          eAuth using the PAM external auth system requires salt-master to be run as root as this
          system needs root access to check authentication.

   Access Control System
       NOTE:
          When to Use client_acl and external_auth

          client_acl is useful for allowing local system  users  to  run  Salt  commands  without
          giving  them root access. If you can log into the Salt master directly, then client_acl
          will allow you to use Salt without root privileges. If the local system  is  configured
          to authenticate against a remote system, like LDAP or Active Directory, then client_acl
          will interact with the remote system transparently.

          external_auth is useful for salt-api or for making your own  scripts  that  use  Salt's
          Python  API.  It can be used at the CLI (with the -a flag) but it is more cumbersome as
          there are more steps involved.  The only time it is useful at the CLI is when the local
          system is not configured to authenticate against an external service but you still want
          Salt to authenticate against an external service.

       The external authentication system allows for specific  users  to  be  granted  access  to
       execute  specific  functions  on  specific  minions.  Access  is  configured in the master
       configuration file and uses the access control system:

          external_auth:
            pam:
              thatch:
                - 'web*':
                  - test.*
                  - network.*
              steve:
                - .*

       The above configuration allows the user thatch  to  execute  functions  in  the  test  and
       network  modules  on  the  minions  that  match  the  web*  target.   User  steve is given
       unrestricted access to minion commands.

       Salt respects the current PAM configuration in place, and  uses  the  'login'  service  to
       authenticate.

       NOTE:
          The PAM module does not allow authenticating as root.

       To allow access to wheel modules or runner modules the following @ syntax must be used:

          external_auth:
            pam:
              thatch:
                - '@wheel'   # to allow access to all wheel modules
                - '@runner'  # to allow access to all runner modules
                - '@jobs'    # to allow access to the jobs runner and/or wheel module

       NOTE:
          The runner/wheel markup is different, since there are no minions to scope the acl to.

       NOTE:
          Globs  will  not match wheel or runners! They must be explicitly allowed with @wheel or
          @runner.

       The external authentication system can then be used from the command-line by any  user  on
       the same system as the master with the -a option:

          $ salt -a pam web\* test.ping

       The system will ask the user for the credentials required by the authentication system and
       then publish the command.

       To apply permissions to a group of users in an external authentication system, append a  %
       to the ID:

          external_auth:
            pam:
              admins%:
                - '*':
                  - 'pkg.*'

       WARNING:
          All   users   that   have   external  authentication  privileges  are  allowed  to  run
          saltutil.findjob. Be aware that this could  inadvertently  expose  some  data  such  as
          minion IDs.

   Tokens
       With  external  authentication alone, the authentication credentials will be required with
       every call to Salt. This can be alleviated with Salt tokens.

       Tokens are short term authorizations and can be easily created by just adding a -T  option
       when authenticating:

          $ salt -T -a pam web\* test.ping

       Now a token will be created that has a expiration of 12 hours (by default).  This token is
       stored in a file named salt_token in the active user's home directory.

       Once the  token  is  created,  it  is  sent  with  all  subsequent  communications.   User
       authentication does not need to be entered again until the token expires.

       Token expiration time can be set in the Salt master config file.

   LDAP and Active Directory
       NOTE:
          LDAP usage requires that you have installed python-ldap.

       Salt  supports  both user and group authentication for LDAP (and Active Directory accessed
       via its LDAP interface)

   OpenLDAP and similar systems
       LDAP configuration happens in the Salt master configuration file.

       Server configuration values and their defaults:

          # Server to auth against
          auth.ldap.server: localhost

          # Port to connect via
          auth.ldap.port: 389

          # Use TLS when connecting
          auth.ldap.tls: False

          # LDAP scope level, almost always 2
          auth.ldap.scope: 2

          # Server specified in URI format
          auth.ldap.uri: ''    # Overrides .ldap.server, .ldap.port, .ldap.tls above

          # Verify server's TLS certificate
          auth.ldap.no_verify: False

          # Bind to LDAP anonymously to determine group membership
          # Active Directory does not allow anonymous binds without special configuration
          auth.ldap.anonymous: False

          # FOR TESTING ONLY, this is a VERY insecure setting.
          # If this is True, the LDAP bind password will be ignored and
          # access will be determined by group membership alone with
          # the group memberships being retrieved via anonymous bind
          auth.ldap.auth_by_group_membership_only: False

          # Require authenticating user to be part of this Organizational Unit
          # This can be blank if your LDAP schema does not use this kind of OU
          auth.ldap.groupou: 'Groups'

          # Object Class for groups.  An LDAP search will be done to find all groups of this
          # class to which the authenticating user belongs.
          auth.ldap.groupclass: 'posixGroup'

          # Unique ID attribute name for the user
          auth.ldap.accountattributename: 'memberUid'

          # These are only for Active Directory
          auth.ldap.activedirectory: False
          auth.ldap.persontype: 'person'

       There are two phases to LDAP authentication.  First, Salt authenticates to  search  for  a
       users's  Distinguished  Name  and  group membership.  The user it authenticates as in this
       phase is often a special LDAP system user with read-only access  to  the  LDAP  directory.
       After  Salt  searches  the  directory  to  determine  the  actual user's DN and groups, it
       re-authenticates as the user running the Salt commands.

       If you are already aware of the structure of your DNs and permissions in your  LDAP  store
       are set such that users can look up their own group memberships, then the first and second
       users can be the same.   To  tell  Salt  this  is  the  case,  omit  the  auth.ldap.bindpw
       parameter.  You can template the binddn like this:

          auth.ldap.basedn: dc=saltstack,dc=com
          auth.ldap.binddn: uid={{ username }},cn=users,cn=accounts,dc=saltstack,dc=com

       Salt will use the password entered on the salt command line in place of the bindpw.

       To  use  two  separate  users,  specify  the LDAP lookup user in the binddn directive, and
       include a bindpw like so

          auth.ldap.binddn: uid=ldaplookup,cn=sysaccounts,cn=etc,dc=saltstack,dc=com
          auth.ldap.bindpw: mypassword

       As mentioned before, Salt uses a filter to find  the  DN  associated  with  a  user.  Salt
       substitutes the {{ username }} value for the username when querying LDAP

          auth.ldap.filter: uid={{ username }}

       For  OpenLDAP,  to  determine  group membership, one can specify an OU that contains group
       data. This is prepended to the basedn to create a  search  path.   Then  the  results  are
       filtered  against  auth.ldap.groupclass,  default  posixGroup,  and  the  account's 'name'
       attribute, memberUid by default.

          auth.ldap.groupou: Groups

   Active Directory
       Active Directory handles group membership differently, and does not  utilize  the  groupou
       configuration variable.  AD needs the following options in the master config:

          auth.ldap.activedirectory: True
          auth.ldap.filter: sAMAccountName={{username}}
          auth.ldap.accountattributename: sAMAccountName
          auth.ldap.groupclass: group
          auth.ldap.persontype: person

       To  determine  group membership in AD, the username and password that is entered when LDAP
       is requested as the eAuth mechanism on the command line is  used  to  bind  to  AD's  LDAP
       interface.  If  this  fails, then it doesn't matter what groups the user belongs to, he or
       she is denied access. Next, the distinguishedName of  the  user  is  looked  up  with  the
       following LDAP search:

          (&(<value of auth.ldap.accountattributename>={{username}})
            (objectClass=<value of auth.ldap.persontype>)
          )

       This  should  return  a  distinguishedName that we can use to filter for group membership.
       Then the following LDAP query is executed:

          (&(member=<distinguishedName from search above>)
            (objectClass=<value of auth.ldap.groupclass>)
          )

          external_auth:
            ldap:
              test_ldap_user:
                  - '*':
                      - test.ping

       To configure an LDAP group, append a % to the ID:

          external_auth:
            ldap:
              test_ldap_group%:
                - '*':
                  - test.echo

ACCESS CONTROL SYSTEM

New in version 0.10.4.

Salt maintains a standard system used to open granular control to non administrative users
to execute Salt commands. The access control system has been applied to all systems used
to configure access to non administrative control interfaces in Salt.These interfaces
include, the peer system, the external auth system and the client acl system.

The access control system mandated a standard configuration syntax used in all of the
three aforementioned systems. While this adds functionality to the configuration in
0.10.4, it does not negate the old configuration.

Now specific functions can be opened up to specific minions from specific users in the
case of external auth and client ACLs, and for specific minions in the case of the peer
system.

The access controls are manifested using matchers in these configurations:

client_acl:
fred:
- web\*:
- pkg.list_pkgs
- test.*
- apache.*

In the above example, fred is able to send commands only to minions which match the
specified glob target. This can be expanded to include other functions for other minions
based on standard targets.

external_auth:
pam:
dave:
- test.ping
- mongo\*:
- network.*
- log\*:
- network.*
- pkg.*
- 'G@os:RedHat':
- kmod.*
steve:
- .*

The above allows for all minions to be hit by test.ping by dave, and adds a few functions
that dave can execute on other minions. It also allows steve unrestricted access to salt
commands.

NOTE:
Functions are matched using regular expressions.

JOB MANAGEMENT

New in version 0.9.7.

Since Salt executes jobs running on many systems, Salt needs to be able to manage jobs
running on many systems.

The Minion proc System
Salt Minions maintain a proc directory in the Salt cachedir. The proc directory maintains
files named after the executed job ID. These files contain the information about the
current running jobs on the minion and allow for jobs to be looked up. This is located in
the proc directory under the cachedir, with a default configuration it is under
/var/cache/salt/proc.

Functions in the saltutil Module
Salt 0.9.7 introduced a few new functions to the saltutil module for managing jobs. These
functions are:

1. running Returns the data of all running jobs that are found in the proc directory.

2. find_job Returns specific data about a certain job based on job id.

3. signal_job Allows for a given jid to be sent a signal.

4. term_job Sends a termination signal (SIGTERM, 15) to the process controlling the
specified job.

5. kill_job Sends a kill signal (SIGKILL, 9) to the process controlling the specified job.

These functions make up the core of the back end used to manage jobs at the minion level.

The jobs Runner
A convenience runner front end and reporting system has been added as well. The jobs
runner contains functions to make viewing data easier and cleaner.

The jobs runner contains a number of functions...

active
The active function runs saltutil.running on all minions and formats the return data about
all running jobs in a much more usable and compact format. The active function will also
compare jobs that have returned and jobs that are still running, making it easier to see
what systems have completed a job and what systems are still being waited on.

# salt-run jobs.active

lookup_jid
When jobs are executed the return data is sent back to the master and cached. By default
it is cached for 24 hours, but this can be configured via the keep_jobs option in the
master configuration. Using the lookup_jid runner will display the same return data that
the initial job invocation with the salt command would display.

# salt-run jobs.lookup_jid <job id number>

list_jobs
Before finding a historic job, it may be required to find the job id. list_jobs will parse
the cached execution data and display all of the job data for jobs that have already, or
partially returned.

# salt-run jobs.list_jobs

Scheduling Jobs
In Salt versions greater than 0.12.0, the scheduling system allows incremental executions
on minions or the master. The schedule system exposes the execution of any execution
function on minions or any runner on the master.

Scheduling is enabled via the schedule option on either the master or minion config files,
or via a minion's pillar data. Schedules that are impletemented via pillar data, only need
to refresh the minion's pillar data, for example by using saltutil.refresh_pillar.
Schedules implemented in the master or minion config have to restart the application in
order for the schedule to be implemented.

NOTE:
The scheduler executes different functions on the master and minions. When running on
the master the functions reference runner functions, when running on the minion the
functions specify execution functions.

A scheduled run has no output on the minion unless the config is set to info level or
higher. Refer to minion logging settings.

Specify maxrunning to ensure that there are no more than N copies of a particular routine
running. Use this for jobs that may be long-running and could step on each other or
otherwise double execute. The default for maxrunning is 1.

States are executed on the minion, as all states are. You can pass positional arguments
and provide a yaml dict of named arguments.

schedule:
job1:
function: state.sls
seconds: 3600
args:
- httpd
kwargs:
test: True

This will schedule the command: state.sls httpd test=True every 3600 seconds (every hour)

schedule:
job1:
function: state.sls
seconds: 3600
args:
- httpd
kwargs:
test: True
splay: 15

This will schedule the command: state.sls httpd test=True every 3600 seconds (every hour)
splaying the time between 0 and 15 seconds

schedule:
job1:
function: state.sls
seconds: 3600
args:
- httpd
kwargs:
test: True
splay:
start: 10
end: 15

This will schedule the command: state.sls httpd test=True every 3600 seconds (every hour)
splaying the time between 10 and 15 seconds

New in version 2014.7.0.

Frequency of jobs can also be specified using date strings supported by the python
dateutil library. This requires python-dateutil to be installed on the minion.

schedule:
job1:
function: state.sls
args:
- httpd
kwargs:
test: True
when: 5:00pm

This will schedule the command: state.sls httpd test=True at 5:00pm minion localtime.

schedule:
job1:
function: state.sls
args:
- httpd
kwargs:
test: True
when:
- Monday 5:00pm
- Tuesday 3:00pm
- Wednesday 5:00pm
- Thursday 3:00pm
- Friday 5:00pm

This will schedule the command: state.sls httpd test=True at 5pm on Monday, Wednesday, and
Friday, and 3pm on Tuesday and Thursday.

schedule:
job1:
function: state.sls
seconds: 3600
args:
- httpd
kwargs:
test: True
range:
start: 8:00am
end: 5:00pm

This will schedule the command: state.sls httpd test=True every 3600 seconds (every hour)
between the hours of 8am and 5pm. The range parameter must be a dictionary with the date
strings using the dateutil format. This requires python-dateutil to be installed on the
minion.

New in version 2014.7.0.

The scheduler also supports ensuring that there are no more than N copies of a particular
routine running. Use this for jobs that may be long-running and could step on each other
or pile up in case of infrastructure outage.

The default for maxrunning is 1.

schedule:
long_running_job:
function: big_file_transfer
jid_include: True

States
schedule:
log-loadavg:
function: cmd.run
seconds: 3660
args:
- 'logger -t salt < /proc/loadavg'
kwargs:
stateful: False
shell: /bin/sh

Highstates
To set up a highstate to run on a minion every 60 minutes set this in the minion config or
pillar:

schedule:
highstate:
function: state.highstate
minutes: 60

Time intervals can be specified as seconds, minutes, hours, or days.

Runners
Runner executions can also be specified on the master within the master configuration
file:

schedule:
run_my_orch:
function: state.orchestrate
hours: 6
splay: 600
args:
- orchestration.my_orch

The above configuration is analogous to running salt-run state.orch orchestration.my_orch
every 6 hours.

Scheduler With Returner
The scheduler is also useful for tasks like gathering monitoring data about a minion, this
schedule option will gather status data and send it to a MySQL returner database:

schedule:
uptime:
function: status.uptime
seconds: 60
returner: mysql
meminfo:
function: status.meminfo
minutes: 5
returner: mysql

Since specifying the returner repeatedly can be tiresome, the schedule_returner option is
available to specify one or a list of global returners to be used by the minions when
scheduling. In Salt versions greater than 0.12.0, the scheduling system allows
incremental executions on minions or the master. The schedule system exposes the execution
of any execution function on minions or any runner on the master.