Playbook parsing

The whole purpose of an inventory source is to have systems to manipulate. The manipulation comes from playbooks (or in the case of ansible ad hoc execution, simple single-task plays). You should already have a base understanding of playbook construction so we won't spend a lot of time covering that; however, we will delve into some specifics of how a playbook is parsed. Specifically, we will cover the following:

• Order of operations
• Relative path assumptions
• Play behavior keys
• Host selection for plays and tasks
• Play and task names

Order of operations

Ansible is designed to be as easy as possible for a human to understand. The developers strive to strike the best balance of human comprehension and machine efficiency. To that end, nearly everything in Ansible can be assumed to be executed in a top to bottom order; that is, the operation listed at the top of a file will be accomplished before the operation listed at the bottom of a file. Having said that, there are a few caveats and even a few ways to influence the order of operations.

A playbook has only two main operations it can accomplish. It can either run a play, or it can include another playbook from somewhere on the filesystem. The order in which these are accomplished is simply the order in which they appear in the playbook file, from top to bottom. It is important to note that while the operations are executed in order, the entire playbook and any included playbooks are completely parsed before any executions. This means that any included playbook file has to exist at the time of the playbook parsing. They cannot be generated in an earlier operation. This is specific to playbook includes, not necessarily to task includes that may appear within a play, which will be covered in a later chapter.

Within a play, there are a few more operations. While a playbook is strictly ordered from top to bottom, a play has a more nuanced order of operations. Here is a list of the possible operations and the order in which they will happen:

• Variable loading
• Fact gathering
• The pre_tasks execution
• Handlers notified from the pre_tasks execution
• Roles execution
• Tasks execution
• Handlers notified from roles or tasks execution
• The post_tasks execution
• Handlers notified from post_tasks execution

Here is an example play with most of these operations shown:

--- 
- hosts: localhost 
  gather_facts: false 
 
  vars: 
    - a_var: derp 
 
  pre_tasks: 
    - name: pretask 
      debug: 
        msg: "a pre task" 
      changed_when: true 
      notify: say hi 
 
  roles: 
    - role: simple 
      derp: newval 
 
  tasks: 
    - name: task 
      debug: 
        msg: "a task" 
      changed_when: true 
      notify: say hi 
 
  post_tasks: 
    - name: posttask 
      debug: 
        msg: "a post task" 
      changed_when: true 
      notify: say hi 

Regardless of the order in which these blocks are listed in a play, the order detailed above is the order in which they will be processed. Handlers (the tasks that can be triggered by other tasks that result in a change) are a special case. There is a utility module, meta, which can be used to trigger handler processing at a specific point:

- meta: flush_handlers 

This will instruct Ansible to process any pending handlers at that point before continuing on with the next task or next block of actions within a play. Understanding the order and being able to influence the order with flush_handlers is another key skill to have when there is a need for orchestrate complicated actions, where things such as service restarts are very sensitive to order. Consider the initial rollout of a service. The play will have tasks that modify config files and indicate that the service should be restarted when these files change. The play will also indicate that the service should be running. The first time this play happens, the config file will change and the service will change from not running to running. Then, the handlers will trigger, which will cause the service to restart immediately. This can be disruptive to any consumers of the service. It would be better to flush the handlers before a final task to ensure the service is running. This way, the restart will happen before the initial start, and thus, the service will start up once and stay up.

Relative path assumptions

When Ansible parses a playbook, there are certain assumptions that can be made about the relative paths of items referenced by the statements in a playbook. In most cases, paths for things such as variable files to include, task files to include, playbook files to include, files to copy, templates to render, scripts to execute, and so on, are all relative to the directory where the file referencing them lives. Let's explore this with an example playbook and directory listing to show where the things are:

• Directory structure:

  . 
  ├── a_vars_file.yaml 
  ├── mastery-hosts 
  ├── relative.yaml 
  └── tasks 
      ├── a.yaml 
      └── b.yaml 
  

• Contents of _vars_file.yaml:

  --- 
  something: "better than nothing" 
  

• Contents of relative.yaml:

  --- 
  - name: relative path play 
    hosts: localhost 
    gather_facts: false 
    vars_files: 
      - a_vars_file.yaml 
    tasks: 
      - name: who am I 
        debug: 
          msg: "I am mastery task" 
      - name: var from file 
        debug:         
          var: something 
   
      - include: tasks/a.yaml 
  

• Contents of tasks/a.yaml:

  --- 
  - name: where am I 
    debug: 
      msg: "I am task a" 
   
  - include: b.yaml 
  

• Contents of tasks/b.yaml:

  --- 
  - name: who am I 
    debug: 
      msg: "I am task b" 
  

Execution of the playbook is shown as follows:

We can clearly see the relative reference to paths and how they are relative to the file referencing them. When using roles, there are some additional relative path assumptions; however, we'll cover that in detail in a later chapter.

Play behavior directives

When Ansible parses a play, there are a few directives it looks for to define various behaviors for a play. These directives are written at the same level as the hosts: directive. Here are subset of the keys that can be used is described:

• any_errors_fatal: This Boolean directive is used to instruct Ansible to treat any failure as a fatal error to prevent any further tasks from being attempted. This changes the default where Ansible will continue until all the tasks are complete or all the hosts have failed.
• connection: This string directive defines which connection system to use for a given play. A common choice to make here is local, which instructs Ansible to do all the operations locally, but with the context of the system from the inventory.
• gather_facts: This Boolean directive controls whether or not Ansible will perform the fact gathering phase of operation, where a special task will run on a host to discover various facts about the system. Skipping fact gathering, when you are sure that you do not need any of the discovered data, can be a significant time-saver in a larger environment.
• max_fail_percentage: This number directive is similar to any_errors_fatal, but is more fine-grained. This allows you to define just what percentage of your hosts can fail before the whole operation is halted.
• no_log: This is a Boolean directive to control whether or not Ansible will log (to the screen and/or a configured log file) the command given or the results received from a task. This is important if your task or return deals with secrets. This key can also be applied to a task directly.
• port: This is a number directive to define what port SSH (or other remote connection plugin) should use to connect unless otherwise configured in inventory data.
• remote_user: This is a string directive that defines which user to log in with on the remote system. The default is to connect as the same user that ansible-playbook was started with.
• serial: This directive takes a number and controls how many systems Ansible will execute a task on before moving to the next task in a play. This is a drastic change from the normal order of operation where a task is executed across every system in a play before moving to the next. This is very useful in rolling update scenarios, which will be detailed in later chapters.
• become: This is a Boolean directive used to configure whether privilege escalation (sudo or otherwise) should be used on the remote host to execute tasks. This key can also be defined at a task level. Related directives include become_user, become_method, and become_flags. These can be used to configure how the escalation will occur.
• strategy: This directive sets the execution strategy to be used for the play.

Many of these keys will be used in example playbooks through this book.

Note

For a full list of available play directives, see the online documentation at: https://docs.ansible.com/ansible/playbooks_directives.html#play.

Execution strategies

With the release of Ansible 2.0, a new way to control play execution behavior was introduced, strategy. A strategy defines how Ansible coordinates each task across the set of hosts. Each strategy is a plugin, and two come with Ansible, linear and free. The linear strategy, which is the default strategy, is how Ansible has always behaved. As a play is executed, all the hosts for a given play execute the first task. Once all are complete, Ansible moves to the next task. The serial directive can create batches of hosts to operate in this way, but the base strategy remains the same. All the targets for a given batch must complete a task before the next task is executed. The free strategy breaks from this traditional behavior. When using the free strategy, as soon as a host completes a task, Ansible will execute the next task for that host, without waiting for any other hosts to complete. This will happen for every host in the set, for every task in the play. The hosts will complete the tasks as fast as each are able to, minimizing the execution time of each specific host. While most playbooks will use the default linear strategy, there are situations where the free strategy would be advantageous. For example, upgrading a service across a large set of hosts. If the play has numerous tasks to perform the upgrade, which starts with shutting down the service, then it would be more important for each host to suffer as little downtime as possible. Allowing each host to independently move through the play as fast as it is able too will ensure that each host is down only for as long as necessary. Without using free, the entire fleet will be down for as long as the slowest host in the fleet takes to complete the tasks.

Note

As the free strategy does not coordinate task completion across hosts, it is not possible to depend on the data that is generated during a task on one host to be available for use in a later task on a different host. There is no guarantee that the first host will have completed the task that generates the data.

Execution strategies are implemented as a plugin, and as such, custom strategies can be developed to extend Ansible behavior. Development of such plugins is beyond the scope of this book.

Host selection for plays and tasks

The first thing most plays define (after a name, of course) is a host pattern for the play. This is the pattern used to select hosts out of the inventory object to run the tasks on. Generally, this is straightforward; a host pattern contains one or more blocks indicating a host, group, wildcard pattern, or regex to use for the selection. Blocks are separated by a colon, wildcards are just an asterisk, and regex patterns start with a tilde:

hostname:groupname:*.example:~(web|db)\.example\.com 

Advanced usage can include group index selection or even ranges within a group:

Webservers[0]:webservers[2:4] 

Each block is treated as an inclusion block, that is, all the hosts found in the first pattern are added to all the hosts found in the next pattern, and so on. However, this can be manipulated with control characters to change their behavior. The use of an ampersand allows an inclusion selection (all the hosts that exist in both patterns). The use of an exclamation point allows exclusion selection (all the hosts that exist in the previous patterns that are NOT in the exclusion pattern):

Webservers:&dbservers 
Webservers:!dbservers 

Once Ansible parses the patterns, it will then apply restrictions, if any. Restrictions come in the form of limits or failed hosts. This result is stored for the duration of the play, and it is accessible via the play_hosts variable. As each task is executed, this data is consulted and an additional restriction may be placed upon it to handle serial operations. As failures are encountered, either failure to connect or a failure in execute tasks, the failed host is placed in a restriction list so that the host will be bypassed in the next task. If, at any time, a host selection routine gets restricted down to zero hosts, the play execution will stop with an error. A caveat here is that if the play is configured to have a max_fail_precentage or any_errors_fatal parameter, then the playbook execution stops immediately after the task where this condition is met.

Play and task names

While not strictly necessary, it is a good practice to label your plays and tasks with names. These names will show up in the command-line output of ansible-playbook, and will show up in the log file if ansible-playbook is directed to log to a file. Task names also come in handy to direct ansible-playbook to start at a specific task and to reference handlers.

There are two main points to consider when naming plays and tasks:

• Names of plays and tasks should be unique
• Beware of what kind of variables can be used in play and task names

Naming plays and tasks uniquely is a best practice in general that will help to quickly identify where a problematic task may reside in your hierarchy of playbooks, roles, task files, handlers, and so on. Uniqueness is more important when notifying a handler or when starting at a specific task. When task names have duplicates, the behavior of Ansible may be nondeterministic or at least nonobvious.

With uniqueness as a goal, many playbook authors will look to variables to satisfy this constraint. This strategy may work well but authors need to take care as to the source of the variable data they are referencing. Variable data can come from a variety of locations (which we will cover later in this chapter), and the values assigned to variables can be defined at a variety of times. For the sake of play and task names, it is important to remember that only variables for which the values can be determined at playbook parse time will parse and render correctly. If the data of a referenced variable is discovered via a task or other operation, the variable string will be displayed unparsed in the output. Let's look at an example playbook that utilizes variables for play and task names:

--- 
- name: play with a {{ var_name }} 
  hosts: localhost 
  gather_facts: false 
 
  vars: 
  - var_name: not-mastery 
 
  tasks: 
  - name: set a variable 
    set_fact: 
    task_var_name: "defined variable" 
 
  - name: task with a {{ task_var_name }} 
    debug: 
    msg: "I am mastery task" 
 
  - name: second play with a {{ task_var_name }} 
    hosts: localhost 
    gather_facts: false 
 
  tasks: 
  - name: task with a {{ runtime_var_name }} 
    debug: 
    msg: "I am another mastery task" 

At first glance, one might expect at least var_name and task_var_name to render correctly. We can clearly see task_var_name being defined before its use. However, armed with our knowledge that playbooks are parsed in their entirety before execution, we know better:

As we can see, the only variable name that is properly rendered is var_name, as it was defined as a static play variable.