Website Provisioning – Automate Apache with SaltStack

Website Provisioning - Apache and SaltstackEver since I’ve started hosting my own domains, I’ve searched for ways to automate the website provisioning.

Normally, the documentation includes the information on configuring your infrastructure based on roles. But if you have a webserver role for all your minions it doesn’t help with the individual provisioning I am aiming at.

Planing Phase

The first rule: plan for what you need with what you have.

For this example I’ll be using the following set-up:

  • e-tel.eu hosted on srv01.so.ai with SSL and HSTS enabled
  • www.bitleader.com hosted on srv01.so.ai with SSL enabled
  • bitleader.one hosted on srv02.so.ai without SSL
  • so.ai hosted on srv03.so.ai without SSL
  • all the hostnames are unique (not the fqdns)
  • all the SSL key files are found in /etc/ssl/salt-managed/keys/website.key
  • all the SSL certificate files are found in /etc/ssl/salt-managed/certs/website.crt
  • the SSL chain file is located in /etc/ssl/salt-managed/bundle.crt
  • all the servers run either Ubuntu 14.04 or Ubuntu 16.04

Data Required

The main idea is to split the provisioning data in two main categories:

  1. Identical across servers and websites
  2. Website and/or server specific

The website and server specific data will be stored in pillars, the rest directly in the state files (in the sls files).

Salt-Master Configuration

pillar_source_merging_strategy

To be able to overwrite part of the pillars with data from other pillar files, you need to set it on recurse. See this link on information about it).

file_roots

Make sure you include a filebase folder. If you don’t, you’ll need to change the examples accordingly.

Pillars

pillars/top.sls

In your pillars/top.sls you should have:

pillars/apache/init.sls

In the folder pillars/apache you create the file init.sls:

In order for you to be able to load pillars with different content for the different hosts, you need to specify this in the pillars/apache/init.sls file. The common setting are the only ones that will be directly in this file.

pillars/apache/{host}.sls

You’ll need to have a sls file for every host:

The host settings will be light. The convention used:

  • ssl will be used to enable/disable SSL
  • force_ssl will be used to redirect HTTP traffic to HTTPS (redirect 301)
  • hsts will be used to enable HSTS for the specified hsts_max_age

Note! You can overwrite all the settings from init.sls in the host settings file (for example, hsts_max_age).

States

You need to create the folder and all the files. For a better structure, the following are split into separate state files: package dependencies, states for the default website and states for the other websites.

salt/top.sls

You include the Apache state in the salt/top.sls file:

salt/apache/init.sls

The folder, package and file states will be included in this file:

salt/apache/packages.sls

All the package dependencies, as well as the Apache service, are monitored here. Depending on the Apache modules you need, this is also where you can enable or disable them.

Note! You’ll need to change the package names and add a service name if you plan on using RH/CentOS!

Note! If you’re using salt < 2016.3 you need to replace apache_module.enabled with apache_module.enable and apache_module.disabled with apache_module.disable

The Default Website

The last state file is salt/apache/default.sls. But before we start with it, some more conventions:

  • all the configuration files, that will be deployed on the server, are located on the Salt-Master under /srv/salt/filebase/apache/.
  • all the SSL certificates are located on the Salt-Master under /srv/salt/filebase/apache/ssl/certs/*.crt
  • all the SSL keys are located on the Salt-Master under /srv/salt/filebase/apache/ssl/keys/*.key
  • the SSL certificates are named either fqdn.crt for the default server certificate, or website.crt for the hosted virtual servers
  • the same applies for SSL keys, their form will be fqdn.key and website.key

Note! This guide doesn’t cover generating your certificate. Information on how to do that can be found on DigitalOcean for example.

salt/apache/default.sls scale with your setup

For the actual Apache configuration files also wanted to automatically make the tuning of the prefork module. Wish I could link the source of the formulas, but alas, I have lost it.

salt/apache/websites.sls

All states related to a hosted website are in this file.

The template files

According to the sls definitions, the following files are needed:

  • salt/filebase/apache/mpm_prefork.conf
  • salt/filebase/apache/site_available_template.conf
  • salt/filebase/apache/logrotate
  • salt/filebase/apache/ssl/keys/srv01.so.ai.key
  • salt/filebase/apache/ssl/keys/srv02.so.ai.key
  • salt/filebase/apache/ssl/keys/srv03.so.ai.key
  • salt/filebase/apache/ssl/keys/e-tel.eu.key
  • salt/filebase/apache/ssl/keys/www.bitleader.com.key
  • salt/filebase/apache/ssl/keys/bitleader.one.key
  • salt/filebase/apache/ssl/keys/so.ai.key
  • salt/filebase/apache/ssl/certs/srv01.so.ai.crt
  • salt/filebase/apache/ssl/certs/srv02.so.ai.crt
  • salt/filebase/apache/ssl/certs/srv03.so.ai.crt
  • salt/filebase/apache/ssl/certs/e-tel.eu.crt
  • salt/filebase/apache/ssl/certs/www.bitleader.com.crt
  • salt/filebase/apache/ssl/certs/bitleader.one.crt
  • salt/filebase/apache/ssl/certs/so.ai.crt
  • salt/filebase/apache/ssl/certs/bundle.crt

Olny three of these files are SaltStack templates:

salt/filebase/apache/logrotate

Include the logs for the default website and all the virtual ones:

salt/filebase/apache/mpm_prefork.conf

Now to apply the settings calculated at the previous step:

salt/filebase/apache/site_available_template.conf

All the website provisioning variables go in this template:

Full Website Provisioning Run

And this is how the full automated website provisioning runs, once it’s set up:

All the files for setting up the website provisioning with SaltStack are available on tlex/Website-Provisioning.