Ansible terms to keep in mind 

Like all new subjects or topics, it is a good idea to get familiar with the terminology of that subject or topic. We will go through some of the Ansible terms that we will be using throughout the book, and if at any point you are not able to follow, you might want to come back to this chapter and refresh your understanding for that particular term. 

Playbooks

A playbook, in the classic sense, is about offensive and defensive plays in football. The players keep a record of the plays (plan of action) in a book, usually in the form of a diagram.

In Ansible, a playbook is a series of ordered steps or instructions for an IT process. Think of a nicely-written instruction manual that can be read and understood by humans and computers alike. 

In the subsequent chapters, all the automation we will focus on regarding security will lead us toward building both simple and complex playbooks. 

This is what an Ansible playbook command looks like:

ansible-playbook -i inventory playbook.yml

Ignore the -i flag for now and notice the extension of the playbook file. 

As stated in http://docs.ansible.com/ansible/playbooks_intro.html:

"Playbooks are expressed in YAML format (see YAML syntax (http://docs.ansible.com/ansible/YAMLSyntax.html)) and have a minimum of syntax, which intentionally tries to not be a programming language or script, but rather a model of a configuration or a process."

Ansible modules

Ansible ships with a number of modules (called the module library) that can be executed directly on remote hosts or through playbooks.Tasks in playbooks call modules to do the work. 

Ansible has many modules, most of which are community contributed and maintained. Core modules are maintained by the Ansible core engineering team and will always ship with Ansible itself.

Users can also write their own modules. These modules can control system resources, like services, packages, or files (anything really), or handle executing system commands.

Here is the list of modules available by Ansible: http://docs.ansible.com/ansible/latest/modules_by_category.html#module-index.
If you use Dash (https://kapeli.com/dash) or Zeal (https://zealdocs.org/), you can download the offline version for easy reference.

Modules can be executed via the command line as well. We will be using modules to write all the tasks inside our playbooks. All modules technically return JSON format data. 

Modules should be idempotent and should avoid making any changes if they detect that the current state matches the desired final state. When using Ansible playbooks, these modules can trigger change events in the form of notifying handlers to run additional tasks.

Documentation for each module can be accessed from the command line with the ansible-doc tool:

$ ansible-doc apt

We can list all the modules available on our host:

$ ansible-doc -l

Start the Apache web server on all nodes grouped under webservers by executing the httpd module. Note the use of the -m flag:

$ ansible webservers -m service -a "name=httpd state=started"

This snippet shows the exact same command but inside a playbook in YAML syntax:

- name: restart webserver
  service:
    name: httpd
    state: started

Each module contains multiple parameters and options, get to know more about the features of the modules by looking at their documentation and examples.

YAML syntax for writing Ansible playbooks

Ansible playbooks are written in YAML, which stands for YAML Ain't Markup Language. 

According to the official document (http://yaml.org/spec/current.html):

YAML Ain’t Markup Language (abbreviated YAML) is a data serialization language designed to be human-friendly and work well with modern programming languages for everyday tasks.

Ansible uses YAML because it is easier for humans to read and write than other common data formats, such as XML or JSON. All YAML files (regardless of their association with Ansible or not) can optionally begin with --- and end with .... This is part of the YAML format and indicates the start and end of a document.

YAML files should end with .yaml or .yml. YAML is case sensitive.
You can also use linters, such as www.yamllint.com, or your text editor plugins for linting YAML syntax, which help you to troubleshoot any syntax errors and so on. 

Here is an example of a simple playbook to showcase YAML syntax from Ansible documentation (http://docs.ansible.com/ansible/playbooks_intro.html#playbook-language-example):

- hosts: webservers
  vars:
    http_port: 80
    max_clients: 200
  remote_user: root

  tasks:
  - name: Ensure apache is at the latest version
    yum:
      name: httpd
      state: latest
  - name: Write the apache config file
    template:
      src: /srv/httpd.j2
      dest: /etc/httpd.conf

    notify:
    - restart apache

  - name: Ensure apache is running (and enable it at boot)
    service:
      name: httpd
      state: started
      enabled: yes

  handlers:
    - name: Restart apache
      service:
        name: httpd
        state: restarted

Ansible roles

While playbooks offer a great way to execute plays in a pre-defined order, there is a brilliant feature on Ansible that takes the whole idea to a completely different level. Roles are a convenient way to bundle tasks, supporting assets such as files and templates, coupled with an automatic set of search paths.

By using a concept most programmers would be familiar with, of including files and folders and ascribing what is being included, a playbook becomes infinitely more readable and understandable. Roles are basically made up of tasks, handlers, and configurations, but by adding an additional layer to how a playbook is structured, we can easily get the big picture overview as well as the low-level details. 

This allows for reusable code and a division of work in a team tasked with writing playbooks. For example, the database guru writes a role (almost like a partial playbook) for setting up the database and the security guru writes one on hardening such a database.

While it is possible to write a playbook in one very large file, eventually you want to reuse files and start to organize things.

Large and complex playbooks are hard to maintain and it is very difficult to reuse sections of a large playbook. Breaking a playbook into roles allows very efficient code reuse and makes playbooks much easier to understand.

The benefits of using roles while building large playbooks include:

• Collaborating on writing playbooks
• Reusing existing roles
• Roles can be updated, improved upon independently
• Handling variables, templates, and files is easier

LAMP usually stands for Linux, Apache, MySQL, PHP. A popular combination of software that is used to build applications for the web. Nowadays, another common combination in the PHP world is LEMP, which is Linux, NGINX, MySQL, PHP.

This is an example of what a possible LAMP stack site.yml can look like:

- name: LAMP stack setup on ubuntu 16.04
  hosts: all
  gather_facts: False
  remote_user: "{{remote_username}}"
  become: yes

 roles:
   - common
   - web
   - db
   - php

Note the list of roles. Just by reading the role names we can get an idea of the kind of tasks possibly under that role. 

Templates with Jinja2

Ansible uses Jinja2 templating to enable dynamic expressions and access to variables. Jinja2 variables and expressions within playbooks and tasks allow us to create roles that are very flexible. By passing variables to a role written this way, we can have the same role perform different tasks or configurations. Using a templating language, such as Jinja2, we are able to write playbooks that are succinct and easier to read.

By ensuring that all the templating takes place on the Ansible controller, Jinja2 is not required on the target machine. Only the required data is copied over, which reduces the data that needs to be transferred. As we know, less data transfer usually results in faster execution and feedback. 

Jinja templating examples

A mark of a good templating language is the ability to allow control of the content without appearing to be a fully-fledged programming language. Jinja2 excels in that by providing us with the ability to do conditional output, such as iterations using loops, among other things. 

Let's look at some basic examples (obviously Ansible playbook-related) to see what that looks like. 

Conditional example

Execute only when the operating system family is Debian:

tasks:
  - name: "shut down Debian flavored systems"
    command: /sbin/shutdown -t now
    when: ansible_os_family == "Debian"

Loops example

The following task adds users using the Jinja2 templating. This allows for dynamic functionality in playbooks. We can use variables to store data when required, we just need to update the variables rather than the entire playbook:

- name: add several users
  user:
    name: "{{ item.name }}"
    state: present
    groups: "{{ item.groups }}"
  with_items:
    - { name: 'testuser1', groups: 'wheel' }
    - { name: 'testuser2', groups: 'root' }

LAMP stack playbook example – combining all the concepts

We will look at how to write a LAMP stack playbook using the skills we have learned so far. Here is the high-level hierarchy structure of the entire playbook:

inventory               # inventory file
group_vars/             #
   all.yml              # variables
site.yml                # master playbook (contains list of roles)
roles/                  #
    common/             # common role
        tasks/          #
            main.yml    # installing basic tasks
    web/                # apache2 role
        tasks/          #
            main.yml    # install apache
        templates/      #
            web.conf.j2 # apache2 custom configuration
        vars/           # 
            main.yml    # variables for web role 
        handlers/       #
            main.yml    # start apache2
    php/                # php role
        tasks/          # 
            main.yml    # installing php and restart apache2
    db/                 # db role
        tasks/          #
            main.yml    # install mysql and include harden.yml
            harden.yml  # security hardening for mysql
        handlers/       #
            main.yml    # start db and restart apache2
        vars/           #
            main.yml    # variables for db role

Let's start with creating an inventory file. The following inventory file is created using static manual entry. Here is a very basic static inventory file where we will define a since host and set the IP address used to connect to it.

Configure the following inventory file as required:

[lamp]
lampstack    ansible_host=192.168.56.10

The following file is group_vars/lamp.yml, which has the configuration of all the global variables:

remote_username: "hodor"

The following file is the site.yml, which is the main playbook file to start:

- name: LAMP stack setup on Ubuntu 16.04
 hosts: lamp
 gather_facts: False
 remote_user: "{{ remote_username }}"
 become: True

 roles:
   - common
   - web
   - db
   - php

The following is the roles/common/tasks/main.yml file, which will install python2, curl, and git:

# In ubuntu 16.04 by default there is no python2
- name: install python 2
  raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal)

- name: install curl and git
  apt:
    name: "{{ item }}"
    state: present
    update_cache: yes

  with_items:
    - curl
    - git

The following task, roles/web/tasks/main.yml, performs multiple operations, such as installation and configuration of apache2. It also adds the service to the startup process:

- name: install apache2 server
  apt:
    name: apache2
    state: present

- name: update the apache2 server configuration
  template: 
    src: web.conf.j2
    dest: /etc/apache2/sites-available/000-default.conf
    owner: root
    group: root
    mode: 0644

- name: enable apache2 on startup
  systemd:
    name: apache2
    enabled: yes
  notify:
    - start apache2

The notify parameter will trigger the handlers found in roles/web/handlers/main.yml:

- name: start apache2
  systemd:
    state: started
    name: apache2

- name: stop apache2
  systemd:
    state: stopped
    name: apache2

- name: restart apache2
  systemd:
    state: restarted
    name: apache2
    daemon_reload: yes

The template files will be taken from role/web/templates/web.conf.j2, which uses Jinja templating, it also takes values from local variables:

<VirtualHost *:80><VirtualHost *:80>
    ServerAdmin {{server_admin_email}}
    DocumentRoot {{server_document_root}}

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

The local variables file is located in roles/web/vars/main.yml:

server_admin_email: [email protected]
server_document_root: /var/www/html

Similarly, we will write database roles as well. The following file roles/db/tasks/main.yml includes installation of the database server with assigned passwords when prompted. At the end of the file, we included harden.yml, which executes another set of tasks:

- name: set mysql root password
  debconf:
    name: mysql-server
    question: mysql-server/root_password
    value: "{{ mysql_root_password | quote }}"
    vtype: password

- name: confirm mysql root password
  debconf: 
    name: mysql-server
    question: mysql-server/root_password_again
    value: "{{ mysql_root_password | quote }}"
    vtype: password

- name: install mysqlserver
  apt:
    name: "{{ item }}"
    state: present 
  with_items:
    - mysql-server
    - mysql-client

- include: harden.yml

The harden.yml performs hardening of MySQL server configuration:

- name: deletes anonymous mysql user
  mysql_user:
    user: ""
    state: absent
    login_password: "{{ mysql_root_password }}"
    login_user: root

- name: secures the mysql root user
  mysql_user: 
    user: root
    password: "{{ mysql_root_password }}"
    host: "{{ item }}"
    login_password: "{{mysql_root_password}}"
    login_user: root
 with_items:
   - 127.0.0.1
   - localhost
   - ::1
   - "{{ ansible_fqdn }}"

- name: removes the mysql test database
  mysql_db:
    db: test
    state: absent
    login_password: "{{ mysql_root_password }}"
    login_user: root

- name: enable mysql on startup
  systemd:
    name: mysql
    enabled: yes

  notify:
    - start mysql

The db server role also has roles/db/handlers/main.yml and local variables similar to the web role:

- name: start mysql
  systemd:
    state: started
    name: mysql

- name: stop mysql
  systemd:
    state: stopped
    name: mysql

- name: restart mysql
  systemd:
    state: restarted
    name: mysql
    daemon_reload: yes

The following file is roles/db/vars/main.yml, which has the mysql_root_password while configuring the server. We will see how we can secure these plaintext passwords using ansible-vault in future chapters:

mysql_root_password: R4nd0mP4$$w0rd

Now, we will install PHP and configure it to work with apache2 by restarting the roles/php/tasks/main.yml service:

- name: install php7
  apt:
    name: "{{ item }}"
    state: present
  with_items:
    - php7.0-mysql
    - php7.0-curl
    - php7.0-json
    - php7.0-cgi
    - php7.0
    - libapache2-mod-php7

- name: restart apache2
  systemd:
    state: restarted
    name: apache2
    daemon_reload: yes

To run this playbook, we need to have Ansible installed in the system path. Please refer to http://docs.ansible.com/ansible/intro_installation.html for installation instructions. 

Then execute the following command against the Ubuntu 16.04 server to set up LAMP stack. Provide the password when it prompts for system access for user hodor:

$ ansible-playbook -i inventory site.yml

After successful completion of the playbook execution, we will be ready to use LAMP stack in a Ubuntu 16.04 machine. You might have observed that each task or role is configurable as we need throughout the playbook. Roles give the power to generalize the playbook and customize easily using variables and templating.