Manage Salt minions in large environments in symphony

Requirements for the Salt setup

If you are new to the tool, follow these instructions to install Salt.

The Salt master machine must have the name salt in /etc/hosts. To enable communication, open TCP ports 9200 and 9300. The tutorial takes place on two managed systems, named paris and paris2, and is based on an Ubuntu OS.

The SaltStack documentation does not explain which options to use in the configuration files for Salt, which are written with YAML formatting. New Salt users can struggle to install software from a custom repository and undertake other configuration tasks. This tutorial covers the approach for configuration files.

Salt state files, which use the .sls suffix, describe the desired operating configuration of the Salt minions to which they are applied. Salt state files use YAML formatting, which requires two-space indentation to delimit each line. To determine the arguments that belong in the Salt state files, review the software development kit (SDK) documentation for two main Salt states: salt.states.pkgrepo and salt.states.pkg.

Then, follow along to install Cassandra on the paris and paris2 systems.

Summon the Salt minions

First, list all the machines that Salt should manage. Create a list with the command sudo salt-key.

Salt responds:

Accepted Keys:
paris
paris2
Denied Keys:
Unaccepted Keys:
Rejected Keys:

Create the top file, /srv/salt/top.sls. The top file shows machines managed by Salt and their roles in the application stack. To follow along with this tutorial, you might have to create the folder /srv/salt.

An example top file, formatted in YAML, looks like this:

base:
  'paris2':

- cassandradeb

  'paris':

   - cassandradeb

These parameters each mean something for the configuration:

• Base means all environments. To limit the environments that receive this configuration, users can name them dev, prod or some other convention. Base is the simplest configuration.
• Salt minions are included by their names, paris and paris2. Programmers can use wildcards, regular expressions and other code to select the right minions in a diverse IT deployment for a given configuration.
• Finally, the YAML file includes cassandradeb. This parameter tells Salt to look for the file /srv/salt/cassandradeb.sls for further instructions. Do not use the name cassandra, as that name will conflict with the built-in Salt Cassandra module if you were to call those functions directly.

Bring Cassandra into the picture

Create the /srv/salt/cassandradeb.sls state file that this Salt configuration top file relies on.

With some common software for IT systems, such as Apache HTTP, the installation is simple, because Salt has background information on how to install the software. For the Apache example, the repository for Ubuntu lists everything Salt needs to know. However, with software such as Cassandra, the Salt user needs to tell the configuration management tool where to find Cassandra's instructions and which version to use.

You also use Salt to push out the configuration necessary to make a cluster. To understand what to include, look at the arguments for the state objects from the SDK, as linked in the requirements section,  salt.states.pkgrepo.managed and salt.states.pkg.install.

With Salt, salt.pkgrepo.managed manages the repositories on Ubuntu machines and other Linux distributions. On Windows, Salt uses salt.modules.win_pkg. The salt.pkgrepo.managed state object copies a repository file to the minion and then runs the apt-get package manager update to include the file in the repo location /etc/apt/sources.list.d/.

The pkg.install state directs the actual installation. This state knows the Salt minion's OS, whether that is Debian, CentOS, Windows or another distribution, and runs yum, apt-get or whatever package manager is appropriate for the OS to install the software. These two state objects are shown in the configuration file:

base:
  pkgrepo.managed:
    - humanname: Cassandra
    - name: deb http://www.apache.org/dist/cassandra/debian 311x main
    - dist: 311x
    - file: /etc/apt/sources.list.d/cassandra.sources.list
    - keyid: A278B781FE4B2BDA
    - keyserver: keyserver.ubuntu.com

cassandra:
  pkg.installed:
    - refresh: True
    - allow_updates: True

Salt with Python

You can run Salt from Python code. This configuration file is the same as calling salt.states.pkg.installed, with these arguments.

salt.states.pkg.installed(name, version=None, refresh=None, fromrepo=None, skip_verify=False, skip_suggestions=False, pkgs=None, sources=None, allow_updates=False, pkg_verify=False, normalize=True, ignore_epoch=False, reinstall=False, update_holds=False, **kwargs)

The arguments are base, name, cassandra, refresh and allow_updates:

• Base means that the configuration should look at this section for instructions. Otherwise, the code would have to use the fromrepo argument passed to installed.
• Name gives the location of the repository. This file gets copied to file: /etc/apt/sources.list.d/cassandra.sources.list. The key (signature) for that is keyid: A278B781FE4B2BDA, which is downloaded from keyserver: keyserver.ubuntu.com.
• Cassandra is the name you could pass to sudo apt-get install if you were to install Cassandra manually.
• Refresh tells Salt to run sudo apt-get update. A refresh command is not always required, such as when the instructions include more than one package install, in which case the refresh step could be redundant.
• The allow_updates argument tells Salt that the software in question can be updated by its vendor directly.

Install Cassandra

Salt offers various ways to target minions, such as by name or with wildcards. This tutorial uses the system's name: paris2. Try it instead using the wildcard paris* or even just '*'. The wildcard '*' by itself tells Salt to install whatever software is referenced in the state files listed in top.sls on all machines that run salt-minion.

The instruction state.apply tells the configuration management tool to install the software or check that it is installed:

sudo salt 'paris2' state.apply

Salt responds as shown:

...

Summary for paris
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  14.317 s

Cassandra is now installed on the paris2 systems. Alternatively, try sending the machine specific instructions, rather than relying on state.apply.