Creating custom facts

While Facter's built-in facts are useful, it's actually quite easy to add your own facts. For example, if you have machines in different data centers or hosting providers, you could add a custom fact for this so that Puppet can determine whether any local settings need to be applied (for example, local DNS servers or network routes).

How to do it...

Here's an example of a simple custom fact:

Create the directory modules/facts/lib/facter and then create the file modules/facts/lib/facter/hello.rb with the following contents:
```
Facter.add(:hello) do
  setcode do
    "Hello, world"
  end
end
```

Modify your site.pp file as follows:

node 'cookbook' {
  notify { $::hello: }
}

Run Puppet:

[root@cookbook ~]# puppet agent -t
Notice: /File[/var/lib/puppet/lib/facter/hello.rb]/ensure: defined content as '{md5}f66d5e290459388c5ffb3694dd22388b'
Info: Loading facts
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416205745'
Notice: Hello, world
Notice: /Stage[main]/Main/Node[cookbook]/Notify[Hello, world]/message: defined 'message' as 'Hello, world'
Notice: Finished catalog run in 0.53 seconds

How it works...

Facter facts are defined in Ruby files that are distributed with facter. Puppet can add additional facts to facter by creating files within the lib/facter subdirectory of a module. These files are then transferred to client nodes as we saw earlier with the puppetlabs-stdlib module. To have the command-line facter use these puppet facts, append the -p option to facter as shown in the following command line:

[root@cookbook ~]# facter hello

[root@cookbook ~]# facter -p hello
Hello, world

Tip

If you are using an older version of Puppet (older than 3.0), you will need to enable pluginsync in your puppet.conf file as shown in the following command line:

[main]
pluginsync = true

Facts can contain any Ruby code, and the last value evaluated inside the setcode do ... end block will be the value returned by the fact. For example, you could make a more useful fact that returns the number of users currently logged in to the system:

Facter.add(:users) do
  setcode do
    %x{/usr/bin/who |wc -l}.chomp
  end
end

To reference the fact in your manifests, just use its name like a built-in fact:

notify { "${::users} users logged in": }
Notice:  2 users logged in

You can add custom facts to any Puppet module. When creating facts that will be used by multiple modules, it may make sense to place them in a facts module. In most cases, the custom fact is related to a specific module and should be placed in that module.

There's more...

The name of the Ruby file that holds the fact definition is irrelevant. You can name this file whatever you wish; the name of the fact comes from the Facter.add() function call. You may also call this function several times within a single Ruby file to define multiple facts as necessary. For instance, you could grep the /proc/meminfo file and return several facts based on memory information as shown in the meminfo.rb file in the following code snippet:

File.open('/proc/meminfo') do |f|
  f.each_line { |line|
  if (line[/^Active:/])
    Facter.add(:memory_active) do
      setcode do line.split(':')[1].to_i
      end
    end
  end
  if (line[/^Inactive:/])
    Facter.add(:memory_inactive) do
      setcode do line.split(':')[1].to_i
      end
    end
  end
  }
end

After synchronizing this file to a node, the memory_active and memory_inactive facts would be available as follows:

[root@cookbook ~]# facter -p |grep memory_
memory_active => 63780
memory_inactive => 58188

You can extend the use of facts to build a completely nodeless Puppet configuration; in other words, Puppet can decide what resources to apply to a machine, based solely on the results of facts. Jordan Sissel has written about this approach at http://www.semicomplete.com/blog/geekery/puppet-nodeless-configuration.html.

You can find out more about custom facts, including how to make sure that OS-specific facts work only on the relevant systems, and how to weigh facts so that they're evaluated in a specific order at the puppetlabs website:

http://docs.puppetlabs.com/guides/custom_facts.html

How it works...

Facter facts are defined in Ruby files that are distributed with facter. Puppet can add additional facts to facter by creating files within the lib/facter subdirectory of a module. These files are then transferred to client nodes as we saw earlier with the puppetlabs-stdlib module. To have the command-line facter use these puppet facts, append the -p option to facter as shown in the following command line:

[root@cookbook ~]# facter hello

[root@cookbook ~]# facter -p hello
Hello, world

Tip

If you are using an older version of Puppet (older than 3.0), you will need to enable pluginsync in your puppet.conf file as shown in the following command line:

[main]
pluginsync = true

Facts can contain any Ruby code, and the last value evaluated inside the setcode do ... end block will be the value returned by the fact. For example, you could make a more useful fact that returns the number of users currently logged in to the system:

Facter.add(:users) do
  setcode do
    %x{/usr/bin/who |wc -l}.chomp
  end
end

To reference the fact in your manifests, just use its name like a built-in fact:

notify { "${::users} users logged in": }
Notice:  2 users logged in

You can add custom facts to any Puppet module. When creating facts that will be used by multiple modules, it may make sense to place them in a facts module. In most cases, the custom fact is related to a specific module and should be placed in that module.

There's more...

The name of the Ruby file that holds the fact definition is irrelevant. You can name this file whatever you wish; the name of the fact comes from the Facter.add() function call. You may also call this function several times within a single Ruby file to define multiple facts as necessary. For instance, you could grep the /proc/meminfo file and return several facts based on memory information as shown in the meminfo.rb file in the following code snippet:

File.open('/proc/meminfo') do |f|
  f.each_line { |line|
  if (line[/^Active:/])
    Facter.add(:memory_active) do
      setcode do line.split(':')[1].to_i
      end
    end
  end
  if (line[/^Inactive:/])
    Facter.add(:memory_inactive) do
      setcode do line.split(':')[1].to_i
      end
    end
  end
  }
end

After synchronizing this file to a node, the memory_active and memory_inactive facts would be available as follows:

[root@cookbook ~]# facter -p |grep memory_
memory_active => 63780
memory_inactive => 58188

You can extend the use of facts to build a completely nodeless Puppet configuration; in other words, Puppet can decide what resources to apply to a machine, based solely on the results of facts. Jordan Sissel has written about this approach at http://www.semicomplete.com/blog/geekery/puppet-nodeless-configuration.html.

You can find out more about custom facts, including how to make sure that OS-specific facts work only on the relevant systems, and how to weigh facts so that they're evaluated in a specific order at the puppetlabs website:

http://docs.puppetlabs.com/guides/custom_facts.html

There's more...

The name of the Ruby file that holds the fact definition is irrelevant. You can name this file whatever you wish; the name of the fact comes from the Facter.add() function call. You may also call this function several times within a single Ruby file to define multiple facts as necessary. For instance, you could grep the /proc/meminfo file and return several facts based on memory information as shown in the meminfo.rb file in the following code snippet:

File.open('/proc/meminfo') do |f|
  f.each_line { |line|
  if (line[/^Active:/])
    Facter.add(:memory_active) do
      setcode do line.split(':')[1].to_i
      end
    end
  end
  if (line[/^Inactive:/])
    Facter.add(:memory_inactive) do
      setcode do line.split(':')[1].to_i
      end
    end
  end
  }
end

After synchronizing this file to a node, the memory_active and memory_inactive facts would be available as follows:

[root@cookbook ~]# facter -p |grep memory_
memory_active => 63780
memory_inactive => 58188

You can extend the use of facts to build a completely nodeless Puppet configuration; in other words, Puppet can decide what resources to apply to a machine, based solely on the results of facts. Jordan Sissel has written about this approach at http://www.semicomplete.com/blog/geekery/puppet-nodeless-configuration.html.

You can find out more about custom facts, including how to make sure that OS-specific facts work only on the relevant systems, and how to weigh facts so that they're evaluated in a specific order at the puppetlabs website:

http://docs.puppetlabs.com/guides/custom_facts.html

Adding external facts

The Creating custom facts recipe describes how to add extra facts written in Ruby. You can also create facts from simple text files or scripts with external facts instead.

External facts live in the /etc/facter/facts.d directory and have a simple key=value format like this:

message="Hello, world"

Getting ready

Here's what you need to do to prepare your system to add external facts:

You'll need Facter Version 1.7 or higher to use external facts, so look up the value of facterversion or use facter -v:
```
[root@cookbook ~]# facter facterversion
2.3.0
[root@cookbook ~]# facter -v
2.3.0
```
You'll also need to create the external facts directory, using the following command:
```
[root@cookbook ~]# mkdir -p /etc/facter/facts.d
```

How to do it...

In this example, we'll create a simple external fact that returns a message, as shown in the Creating custom facts recipe:

Create the file /etc/facter/facts.d/local.txt with the following contents:
```
model=ED-209
```
Run the following command:
```
[root@cookbook ~]# facter model
ED-209
```
Well, that was easy! You can add more facts to the same file, or other files, of course, as follows:
```
model=ED-209
builder=OCP
directives=4
```
However, what if you need to compute a fact in some way, for example, the number of logged-in users? You can create executable facts to do this.
Create the file /etc/facter/facts.d/users.sh with the following contents:
```
#!/bin/sh
echo users=`who |wc -l`
```

Make this file executable with the following command:

[root@cookbook ~]# chmod a+x /etc/facter/facts.d/users.sh

Now check the users value with the following command:
```
[root@cookbook ~]# facter users
2
```

How it works...

In this example, we'll create an external fact by creating files on the node. We'll also show how to override a previously defined fact.

Current versions of Facter will look into /etc/facter/facts.d for files of type .txt, .json, or .yaml. If facter finds a text file, it will parse the file for key=value pairs and add the key as a new fact:
```
[root@cookbook ~]# facter model
ED-209
```
If the file is a YAML or JSON file, then facter will parse the file for key=value pairs in the respective format. For YAML, for instance:
```
---
registry: NCC-68814
class: Andromeda
shipname: USS Prokofiev
```

The resulting output will be as follows:

[root@cookbook ~]# facter registry class shipname
class => Andromeda
registry => NCC-68814
shipname => USS Prokofiev

In the case of executable files, Facter will assume that their output is a list of key=value pairs. It will execute all the files in the facts.d directory and add their output to the internal fact hash.
Tip
In Windows, batch files or PowerShell scripts may be used in the same way that executable scripts are used in Linux.
In the users example, Facter will execute the users.sh script, which results in the following output:
```
users=2
```
It will then search this output for users and return the matching value:
```
[root@cookbook ~]# facter users
2
```
If there are
multiple matches for the key you specified, Facter determines which fact to return based on a weight property. In my version of facter, the weight of external facts is 10,000 (defined in facter/util/directory_loader.rb as EXTERNAL_FACT_WEIGHT). This high value is to ensure that the facts you define can override the supplied facts. For example:
```
[root@cookbook ~]# facter architecture
x86_64
[root@cookbook ~]# echo "architecture=ppc64">>/etc/facter/facts.d/myfacts.txt
[root@cookbook ~]# facter architecture
ppc64
```

There's more...

Since all external facts have a weight of 10,000, the order in which they are parsed within the /etc/facter/facts.d directory sets their precedence (with the last one encountered having the highest precedence). To create a fact that will be favored over another, you'll need to have it created in a file that comes last alphabetically:

[root@cookbook ~]# facter architecture
ppc64
[root@cookbook ~]# echo "architecture=r10000" >>/etc/facter/facts.d/z-architecture.txt
[root@cookbook ~]# facter architecture
r10000

Debugging external facts

If you're having trouble getting Facter to recognize your external facts, run Facter in debug mode to see what's happening:

ubuntu@cookbook:~/puppet$ facter -d robin
Fact file /etc/facter/facts.d/myfacts.json was parsed but returned an empty data set

The X JSON file was parsed but returned an empty data set error, which means Facter didn't find any key=value pairs in the file or (in the case of an executable fact) in its output.

Note

Note that if you have external facts present, Facter parses or runs all the facts in the /etc/facter/facts.d directory every time you query Facter. If some of these scripts take a long time to run, that can significantly slow down anything that uses Facter (run Facter with the --iming switch to troubleshoot this). Unless a particular fact needs to be recomputed every time it's queried, consider replacing it with a cron job that computes it every so often and writes the result to a text file in the Facter directory.

Using external facts in Puppet

Any external facts you create will be available to both Facter and Puppet. To reference external facts in your Puppet manifests, just use the fact name in the same way you would for a built-in or custom fact:

notify { "There are $::users people logged in right now.": }

Unless you are specifically attempting to override a defined fact, you should avoid using the name of a predefined fact.

How to do it...

In this example, we'll create a simple external fact that returns a message, as shown in the Creating custom facts recipe:

Create the file /etc/facter/facts.d/local.txt with the following contents:
```
model=ED-209
```
Run the following command:
```
[root@cookbook ~]# facter model
ED-209
```
Well, that was easy! You can add more facts to the same file, or other files, of course, as follows:
```
model=ED-209
builder=OCP
directives=4
```
However, what if you need to compute a fact in some way, for example, the number of logged-in users? You can create executable facts to do this.
Create the file /etc/facter/facts.d/users.sh with the following contents:
```
#!/bin/sh
echo users=`who |wc -l`
```

Make this file executable with the following command:

[root@cookbook ~]# chmod a+x /etc/facter/facts.d/users.sh

Now check the users value with the following command:
```
[root@cookbook ~]# facter users
2
```

How it works...

In this example, we'll create an external fact by creating files on the node. We'll also show how to override a previously defined fact.

Current versions of Facter will look into /etc/facter/facts.d for files of type .txt, .json, or .yaml. If facter finds a text file, it will parse the file for key=value pairs and add the key as a new fact:
```
[root@cookbook ~]# facter model
ED-209
```
If the file is a YAML or JSON file, then facter will parse the file for key=value pairs in the respective format. For YAML, for instance:
```
---
registry: NCC-68814
class: Andromeda
shipname: USS Prokofiev
```

The resulting output will be as follows:

[root@cookbook ~]# facter registry class shipname
class => Andromeda
registry => NCC-68814
shipname => USS Prokofiev

In the case of executable files, Facter will assume that their output is a list of key=value pairs. It will execute all the files in the facts.d directory and add their output to the internal fact hash.
Tip
In Windows, batch files or PowerShell scripts may be used in the same way that executable scripts are used in Linux.
In the users example, Facter will execute the users.sh script, which results in the following output:
```
users=2
```
It will then search this output for users and return the matching value:
```
[root@cookbook ~]# facter users
2
```
If there are
multiple matches for the key you specified, Facter determines which fact to return based on a weight property. In my version of facter, the weight of external facts is 10,000 (defined in facter/util/directory_loader.rb as EXTERNAL_FACT_WEIGHT). This high value is to ensure that the facts you define can override the supplied facts. For example:
```
[root@cookbook ~]# facter architecture
x86_64
[root@cookbook ~]# echo "architecture=ppc64">>/etc/facter/facts.d/myfacts.txt
[root@cookbook ~]# facter architecture
ppc64
```

There's more...

Since all external facts have a weight of 10,000, the order in which they are parsed within the /etc/facter/facts.d directory sets their precedence (with the last one encountered having the highest precedence). To create a fact that will be favored over another, you'll need to have it created in a file that comes last alphabetically:

[root@cookbook ~]# facter architecture
ppc64
[root@cookbook ~]# echo "architecture=r10000" >>/etc/facter/facts.d/z-architecture.txt
[root@cookbook ~]# facter architecture
r10000

Debugging external facts

If you're having trouble getting Facter to recognize your external facts, run Facter in debug mode to see what's happening:

ubuntu@cookbook:~/puppet$ facter -d robin
Fact file /etc/facter/facts.d/myfacts.json was parsed but returned an empty data set

The X JSON file was parsed but returned an empty data set error, which means Facter didn't find any key=value pairs in the file or (in the case of an executable fact) in its output.

Note

Note that if you have external facts present, Facter parses or runs all the facts in the /etc/facter/facts.d directory every time you query Facter. If some of these scripts take a long time to run, that can significantly slow down anything that uses Facter (run Facter with the --iming switch to troubleshoot this). Unless a particular fact needs to be recomputed every time it's queried, consider replacing it with a cron job that computes it every so often and writes the result to a text file in the Facter directory.

Using external facts in Puppet

Any external facts you create will be available to both Facter and Puppet. To reference external facts in your Puppet manifests, just use the fact name in the same way you would for a built-in or custom fact:

notify { "There are $::users people logged in right now.": }

Unless you are specifically attempting to override a defined fact, you should avoid using the name of a predefined fact.

How it works...

In this example, we'll create an external fact by creating files on the node. We'll also show how to override a previously defined fact.

Current versions of Facter will look into /etc/facter/facts.d for files of type .txt, .json, or .yaml. If facter finds a text file, it will parse the file for key=value pairs and add the key as a new fact:
```
[root@cookbook ~]# facter model
ED-209
```
If the file is a YAML or JSON file, then facter will parse the file for key=value pairs in the respective format. For YAML, for instance:
```
---
registry: NCC-68814
class: Andromeda
shipname: USS Prokofiev
```

The resulting output will be as follows:

[root@cookbook ~]# facter registry class shipname
class => Andromeda
registry => NCC-68814
shipname => USS Prokofiev

In the case of executable files, Facter will assume that their output is a list of key=value pairs. It will execute all the files in the facts.d directory and add their output to the internal fact hash.
Tip
In Windows, batch files or PowerShell scripts may be used in the same way that executable scripts are used in Linux.
In the users example, Facter will execute the users.sh script, which results in the following output:
```
users=2
```
It will then search this output for users and return the matching value:
```
[root@cookbook ~]# facter users
2
```
If there are
multiple matches for the key you specified, Facter determines which fact to return based on a weight property. In my version of facter, the weight of external facts is 10,000 (defined in facter/util/directory_loader.rb as EXTERNAL_FACT_WEIGHT). This high value is to ensure that the facts you define can override the supplied facts. For example:
```
[root@cookbook ~]# facter architecture
x86_64
[root@cookbook ~]# echo "architecture=ppc64">>/etc/facter/facts.d/myfacts.txt
[root@cookbook ~]# facter architecture
ppc64
```

There's more...

Since all external facts have a weight of 10,000, the order in which they are parsed within the /etc/facter/facts.d directory sets their precedence (with the last one encountered having the highest precedence). To create a fact that will be favored over another, you'll need to have it created in a file that comes last alphabetically:

[root@cookbook ~]# facter architecture
ppc64
[root@cookbook ~]# echo "architecture=r10000" >>/etc/facter/facts.d/z-architecture.txt
[root@cookbook ~]# facter architecture
r10000

Debugging external facts

If you're having trouble getting Facter to recognize your external facts, run Facter in debug mode to see what's happening:

ubuntu@cookbook:~/puppet$ facter -d robin
Fact file /etc/facter/facts.d/myfacts.json was parsed but returned an empty data set

The X JSON file was parsed but returned an empty data set error, which means Facter didn't find any key=value pairs in the file or (in the case of an executable fact) in its output.

Note

Note that if you have external facts present, Facter parses or runs all the facts in the /etc/facter/facts.d directory every time you query Facter. If some of these scripts take a long time to run, that can significantly slow down anything that uses Facter (run Facter with the --iming switch to troubleshoot this). Unless a particular fact needs to be recomputed every time it's queried, consider replacing it with a cron job that computes it every so often and writes the result to a text file in the Facter directory.

Using external facts in Puppet

Any external facts you create will be available to both Facter and Puppet. To reference external facts in your Puppet manifests, just use the fact name in the same way you would for a built-in or custom fact:

notify { "There are $::users people logged in right now.": }

Unless you are specifically attempting to override a defined fact, you should avoid using the name of a predefined fact.

There's more...

Since all external facts have a weight of 10,000, the order in which they are parsed within the /etc/facter/facts.d directory sets their precedence (with the last one encountered having the highest precedence). To create a fact that will be favored over another, you'll need to have it created in a file that comes last alphabetically:

[root@cookbook ~]# facter architecture
ppc64
[root@cookbook ~]# echo "architecture=r10000" >>/etc/facter/facts.d/z-architecture.txt
[root@cookbook ~]# facter architecture
r10000

Debugging external facts

If you're having trouble getting Facter to recognize your external facts, run Facter in debug mode to see what's happening:

ubuntu@cookbook:~/puppet$ facter -d robin
Fact file /etc/facter/facts.d/myfacts.json was parsed but returned an empty data set

The X JSON file was parsed but returned an empty data set error, which means Facter didn't find any key=value pairs in the file or (in the case of an executable fact) in its output.

Note

Note that if you have external facts present, Facter parses or runs all the facts in the /etc/facter/facts.d directory every time you query Facter. If some of these scripts take a long time to run, that can significantly slow down anything that uses Facter (run Facter with the --iming switch to troubleshoot this). Unless a particular fact needs to be recomputed every time it's queried, consider replacing it with a cron job that computes it every so often and writes the result to a text file in the Facter directory.

Using external facts in Puppet

Any external facts you create will be available to both Facter and Puppet. To reference external facts in your Puppet manifests, just use the fact name in the same way you would for a built-in or custom fact:

notify { "There are $::users people logged in right now.": }

Unless you are specifically attempting to override a defined fact, you should avoid using the name of a predefined fact.

Note

Note that if you have external facts present, Facter parses or runs all the facts in the /etc/facter/facts.d directory every time you query Facter. If some of these scripts take a long time to run, that can significantly slow down anything that uses Facter (run Facter with the --iming switch to troubleshoot this). Unless a particular fact needs to be recomputed every time it's queried, consider replacing it with a cron job that computes it every so often and writes the result to a text file in the Facter directory.

Using external facts in Puppet

Any external facts you create will be available to both Facter and Puppet. To reference external facts in your Puppet manifests, just use the fact name in the same way you would for a built-in or custom fact:

notify { "There are $::users people logged in right now.": }

Unless you are specifically attempting to override a defined fact, you should avoid using the name of a predefined fact.

Setting facts as environment variables

Another handy way to get information into Puppet and Facter is to pass it using environment variables. Any environment variable whose name starts with FACTER_ will be interpreted as a fact. For example, ask facter the value of hello using the following command:

[root@cookbook ~]# facter -p hello
Hello, world

Now override the value with an environment variable and ask again:

[root@cookbook ~]# FACTER_hello='Howdy!' facter -p hello
Howdy!

It works just as well with Puppet, so let's run through an example.

How to do it...

In this example we'll set a fact using an environment variable:

Keep the node definition for cookbook the same as our last example:
```
node cookbook {
  notify {"$::hello": }
}
```

Run the following command:

[root@cookbook ~]# FACTER_hello="Hallo Welt" puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416212026'
Notice: Hallo Welt
Notice: /Stage[main]/Main/Node[cookbook]/Notify[Hallo Welt]/message: defined 'message' as 'Hallo Welt'
Notice: Finished catalog run in 0.27 seconds

set a fact using an environment variable:

Keep the node definition for cookbook the same as our last example:
```
node cookbook {
  notify {"$::hello": }
}
```

Run the following command:

[root@cookbook ~]# FACTER_hello="Hallo Welt" puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416212026'
Notice: Hallo Welt
Notice: /Stage[main]/Main/Node[cookbook]/Notify[Hallo Welt]/message: defined 'message' as 'Hallo Welt'
Notice: Finished catalog run in 0.27 seconds

Generating manifests with the Puppet resource command

If you have a server that is already configured as it needs to be, or nearly so, you can capture that configuration as a Puppet manifest. The Puppet resource command generates Puppet manifests from the existing configuration of a system. For example, you can have puppet resource generate a manifest that creates all the users found on the system. This is very useful to take a snapshot of a working system and get its configuration quickly into Puppet.

How to do it...

Here are some examples of using puppet resource to get data from a running system:

To generate the manifest for a particular user, run the following command:

[root@cookbook ~]# puppet resource user thomas
user { 'thomas':
  ensure           => 'present',
  comment          => 'thomas Admin User',
  gid              => '1001',
  groups           => ['bin', 'wheel'],
  home             => '/home/thomas',
  password         => '!!',
  password_max_age => '99999',
  password_min_age => '0',
  shell            => '/bin/bash',
  uid              => '1001',
}

For a particular service, run the following command:

[root@cookbook ~]# puppet resource service sshd
service { 'sshd':
  ensure => 'running',
  enable => 'true',
}

For a package, run the following command:

[root@cookbook ~]# puppet resource package kernel
package { 'kernel':
  ensure => '2.6.32-431.23.3.el6',
}

There's more...

You can use puppet resource to examine each of the resource types available in Puppet. In the preceding examples, we generated a manifest for a specific instance of the resource type, but you can also use puppet resource to dump all instances of the resource:

[root@cookbook ~]# puppet resource service
service { 'abrt-ccpp':
  ensure => 'running',
  enable => 'true',
}
service { 'abrt-oops':
  ensure => 'running',
  enable => 'true',
}
service { 'abrtd':
  ensure => 'running',
  enable => 'true',
}
service { 'acpid':
  ensure => 'running',
  enable => 'true',
}
service { 'atd':
  ensure => 'running',
  enable => 'true',
}
service { 'auditd':
  ensure => 'running',
  enable => 'true',
}

This will output the state of each service on the system; this is because each service is an enumerable resource. When you try the same command with a resource that is not enumerable, you get an error message:

[root@cookbook ~]# puppet resource file
Error: Could not run: Listing all file instances is not supported.  Please specify a file or directory, e.g. puppet resource file /etc

Asking Puppet to describe each file on the system will not work; that's something best left to an audit tool such as tripwire (a system designed to look for changes on every file on the system, http://www.tripwire.com).

particular service, run the following command:

[root@cookbook ~]# puppet resource service sshd
service { 'sshd':
  ensure => 'running',
  enable => 'true',
}

For a package, run the following command:

[root@cookbook ~]# puppet resource package kernel
package { 'kernel':
  ensure => '2.6.32-431.23.3.el6',
}

There's more...

You can use puppet resource to examine each of the resource types available in Puppet. In the preceding examples, we generated a manifest for a specific instance of the resource type, but you can also use puppet resource to dump all instances of the resource:

[root@cookbook ~]# puppet resource service
service { 'abrt-ccpp':
  ensure => 'running',
  enable => 'true',
}
service { 'abrt-oops':
  ensure => 'running',
  enable => 'true',
}
service { 'abrtd':
  ensure => 'running',
  enable => 'true',
}
service { 'acpid':
  ensure => 'running',
  enable => 'true',
}
service { 'atd':
  ensure => 'running',
  enable => 'true',
}
service { 'auditd':
  ensure => 'running',
  enable => 'true',
}

This will output the state of each service on the system; this is because each service is an enumerable resource. When you try the same command with a resource that is not enumerable, you get an error message:

[root@cookbook ~]# puppet resource file
Error: Could not run: Listing all file instances is not supported.  Please specify a file or directory, e.g. puppet resource file /etc

Asking Puppet to describe each file on the system will not work; that's something best left to an audit tool such as tripwire (a system designed to look for changes on every file on the system, http://www.tripwire.com).

resource types available in Puppet. In the preceding examples, we generated a manifest for a specific instance of the resource type, but you can also use puppet resource to dump all instances of the resource:

[root@cookbook ~]# puppet resource service
service { 'abrt-ccpp':
  ensure => 'running',
  enable => 'true',
}
service { 'abrt-oops':
  ensure => 'running',
  enable => 'true',
}
service { 'abrtd':
  ensure => 'running',
  enable => 'true',
}
service { 'acpid':
  ensure => 'running',
  enable => 'true',
}
service { 'atd':
  ensure => 'running',
  enable => 'true',
}
service { 'auditd':
  ensure => 'running',
  enable => 'true',
}

This will output the state of each service on the system; this is because each service is an enumerable resource. When you try the same command with a resource that is not enumerable, you get an error message:

[root@cookbook ~]# puppet resource file
Error: Could not run: Listing all file instances is not supported.  Please specify a file or directory, e.g. puppet resource file /etc

Asking Puppet to describe each file on the system will not work; that's something best left to an audit tool such as tripwire (a system designed to look for changes on every file on the system, http://www.tripwire.com).

Generating manifests with other tools

If you want to quickly capture the complete configuration of a running system as a Puppet manifest, there are a couple of tools available to help. In this example, we'll look at Blueprint, which is designed to examine a machine and dump its state as Puppet code.

Getting ready

Here's what you need to do to prepare your system to use Blueprint.

Run the following command to install Blueprint; we'll use puppet resource here to change the state of the python-pip package:

[root@cookbook ~]# puppet resource package python-pip ensure=installed
Notice: /Package[python-pip]/ensure: created
package { 'python-pip':
  ensure => '1.3.1-4.el6',
}
[root@cookbook ~]# pip install blueprint
Downloading/unpacking blueprint
  Downloading blueprint-3.4.2.tar.gz (59kB): 59kB downloaded
  Running setup.py egg_info for package blueprint
Installing collected packages: blueprint
  Running setup.py install for blueprint
    changing mode of build/scripts-2.6/blueprint from 644 to 755
...
Successfully installed blueprint
Cleaning up...

Tip

You may need to install Git on your cookbook node if it is not already installed.

How to do it...

These steps will show you how to run Blueprint:

Run the following commands:

[root@cookbook ~]# mkdir blueprint && cd blueprint
[root@cookbook blueprint]# blueprint create -P blueprint_test
# [blueprint] searching for APT packages to exclude
# [blueprint] searching for Yum packages to exclude
# [blueprint] caching excluded Yum packages
# [blueprint] parsing blueprintignore(5) rules
# [blueprint] searching for npm packages
# [blueprint] searching for configuration files
# [blueprint] searching for APT packages
# [blueprint] searching for PEAR/PECL packages
# [blueprint] searching for Python packages
# [blueprint] searching for Ruby gems
# [blueprint] searching for software built from source
# [blueprint] searching for Yum packages
# [blueprint] searching for service dependencies
blueprint_test/manifests/init.pp

Read the blueprint_test/manifests/init.pp file to see the generated code:

#
# Automatically generated by blueprint(7).  Edit at your own risk.
#
class blueprint_test {
  Exec {
    path => '/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin',
  }
  Class['sources'] -> Class['files'] -> Class['packages']
    class files {
      file {
        '/etc':
          ensure => directory;
        '/etc/aliases.db':
content => template('blueprint_test/etc/aliases.db'),
          ensure  => file,
group   => root,
          mode    => 0644,
          owner   => root;
'/etc/audit':
          ensure => directory;
'/etc/audit/audit.rules':
          content => template('blueprint_test/etc/audit/audit.rules'),
          ensure  => file,
          group   => root,
          mode    => 0640,
          owner   => root;
        '/etc/blkid':
          ensure => directory;
'/etc/cron.hourly':
          ensure => directory;
'/etc/cron.hourly/run-backup':
          content => template('blueprint_test/etc/cron.hourly/run-backup'),
          ensure  => file,
          group   => root,
          mode    => 0755,
owner   => root;
'/etc/crypttab':
          content => template('blueprint_test/etc/crypttab'),
          ensure  => file,
          group   => root,
          mode    => 0644,
          owner   => root;

There's more...

Blueprint just takes a snapshot of the system as it stands; it makes no intelligent decisions, and Blueprint captures all the files on the system and all the packages. It will generate a configuration much larger than you may actually require. For instance, when configuring a server, you may specify that you want the Apache package installed. The dependencies for the Apache package will be installed automatically and you need to specify them. When generating the configuration with a tool such as Blueprint, you will capture all those dependencies and lock the versions that are installed on your system currently. Looking at our generated Blueprint code, we can see that this is the case:

class yum {
  package {
    'GeoIP':
      ensure => '1.5.1-5.el6.x86_64';
    'PyXML':
      ensure => '0.8.4-19.el6.x86_64';
    'SDL':
      ensure => '1.2.14-3.el6.x86_64';
    'apr':
      ensure => '1.3.9-5.el6_2.x86_64';
    'apr-util':
      ensure => '1.3.9-3.el6_0.1.x86_64';

If you were creating this manifest yourself, you would likely specify ensure => installed instead of a specific version.

Packages install default versions of files. Blueprint has no notion of this and will add all the files to the manifest, even those that have not changed. By default, Blueprint will indiscriminately capture all the files in /etc as file resources.

Blueprint and similar tools have a very small use case generally, but may help you to get familiar with the Puppet syntax and give you some ideas on how to specify your own manifests. I would not recommend blindly using this tool to create a system, however.

There's no shortcut to good configuration management, those who hope to save time and effort by cutting and pasting someone else's code as a whole (as with public modules) are likely to find that it saves neither.

Tip

You may need to install Git on your cookbook node if it is not already installed.

How to do it...

These steps will show you how to run Blueprint:

Run the following commands:

[root@cookbook ~]# mkdir blueprint && cd blueprint
[root@cookbook blueprint]# blueprint create -P blueprint_test
# [blueprint] searching for APT packages to exclude
# [blueprint] searching for Yum packages to exclude
# [blueprint] caching excluded Yum packages
# [blueprint] parsing blueprintignore(5) rules
# [blueprint] searching for npm packages
# [blueprint] searching for configuration files
# [blueprint] searching for APT packages
# [blueprint] searching for PEAR/PECL packages
# [blueprint] searching for Python packages
# [blueprint] searching for Ruby gems
# [blueprint] searching for software built from source
# [blueprint] searching for Yum packages
# [blueprint] searching for service dependencies
blueprint_test/manifests/init.pp

Read the blueprint_test/manifests/init.pp file to see the generated code:

#
# Automatically generated by blueprint(7).  Edit at your own risk.
#
class blueprint_test {
  Exec {
    path => '/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin',
  }
  Class['sources'] -> Class['files'] -> Class['packages']
    class files {
      file {
        '/etc':
          ensure => directory;
        '/etc/aliases.db':
content => template('blueprint_test/etc/aliases.db'),
          ensure  => file,
group   => root,
          mode    => 0644,
          owner   => root;
'/etc/audit':
          ensure => directory;
'/etc/audit/audit.rules':
          content => template('blueprint_test/etc/audit/audit.rules'),
          ensure  => file,
          group   => root,
          mode    => 0640,
          owner   => root;
        '/etc/blkid':
          ensure => directory;
'/etc/cron.hourly':
          ensure => directory;
'/etc/cron.hourly/run-backup':
          content => template('blueprint_test/etc/cron.hourly/run-backup'),
          ensure  => file,
          group   => root,
          mode    => 0755,
owner   => root;
'/etc/crypttab':
          content => template('blueprint_test/etc/crypttab'),
          ensure  => file,
          group   => root,
          mode    => 0644,
          owner   => root;

There's more...

Blueprint just takes a snapshot of the system as it stands; it makes no intelligent decisions, and Blueprint captures all the files on the system and all the packages. It will generate a configuration much larger than you may actually require. For instance, when configuring a server, you may specify that you want the Apache package installed. The dependencies for the Apache package will be installed automatically and you need to specify them. When generating the configuration with a tool such as Blueprint, you will capture all those dependencies and lock the versions that are installed on your system currently. Looking at our generated Blueprint code, we can see that this is the case:

class yum {
  package {
    'GeoIP':
      ensure => '1.5.1-5.el6.x86_64';
    'PyXML':
      ensure => '0.8.4-19.el6.x86_64';
    'SDL':
      ensure => '1.2.14-3.el6.x86_64';
    'apr':
      ensure => '1.3.9-5.el6_2.x86_64';
    'apr-util':
      ensure => '1.3.9-3.el6_0.1.x86_64';

If you were creating this manifest yourself, you would likely specify ensure => installed instead of a specific version.

Packages install default versions of files. Blueprint has no notion of this and will add all the files to the manifest, even those that have not changed. By default, Blueprint will indiscriminately capture all the files in /etc as file resources.

Blueprint and similar tools have a very small use case generally, but may help you to get familiar with the Puppet syntax and give you some ideas on how to specify your own manifests. I would not recommend blindly using this tool to create a system, however.

There's no shortcut to good configuration management, those who hope to save time and effort by cutting and pasting someone else's code as a whole (as with public modules) are likely to find that it saves neither.

show you how to run Blueprint:

Run the following commands:

[root@cookbook ~]# mkdir blueprint && cd blueprint
[root@cookbook blueprint]# blueprint create -P blueprint_test
# [blueprint] searching for APT packages to exclude
# [blueprint] searching for Yum packages to exclude
# [blueprint] caching excluded Yum packages
# [blueprint] parsing blueprintignore(5) rules
# [blueprint] searching for npm packages
# [blueprint] searching for configuration files
# [blueprint] searching for APT packages
# [blueprint] searching for PEAR/PECL packages
# [blueprint] searching for Python packages
# [blueprint] searching for Ruby gems
# [blueprint] searching for software built from source
# [blueprint] searching for Yum packages
# [blueprint] searching for service dependencies
blueprint_test/manifests/init.pp

Read the blueprint_test/manifests/init.pp file to see the generated code:

#
# Automatically generated by blueprint(7).  Edit at your own risk.
#
class blueprint_test {
  Exec {
    path => '/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin',
  }
  Class['sources'] -> Class['files'] -> Class['packages']
    class files {
      file {
        '/etc':
          ensure => directory;
        '/etc/aliases.db':
content => template('blueprint_test/etc/aliases.db'),
          ensure  => file,
group   => root,
          mode    => 0644,
          owner   => root;
'/etc/audit':
          ensure => directory;
'/etc/audit/audit.rules':
          content => template('blueprint_test/etc/audit/audit.rules'),
          ensure  => file,
          group   => root,
          mode    => 0640,
          owner   => root;
        '/etc/blkid':
          ensure => directory;
'/etc/cron.hourly':
          ensure => directory;
'/etc/cron.hourly/run-backup':
          content => template('blueprint_test/etc/cron.hourly/run-backup'),
          ensure  => file,
          group   => root,
          mode    => 0755,
owner   => root;
'/etc/crypttab':
          content => template('blueprint_test/etc/crypttab'),
          ensure  => file,
          group   => root,
          mode    => 0644,
          owner   => root;

There's more...

Blueprint just takes a snapshot of the system as it stands; it makes no intelligent decisions, and Blueprint captures all the files on the system and all the packages. It will generate a configuration much larger than you may actually require. For instance, when configuring a server, you may specify that you want the Apache package installed. The dependencies for the Apache package will be installed automatically and you need to specify them. When generating the configuration with a tool such as Blueprint, you will capture all those dependencies and lock the versions that are installed on your system currently. Looking at our generated Blueprint code, we can see that this is the case:

class yum {
  package {
    'GeoIP':
      ensure => '1.5.1-5.el6.x86_64';
    'PyXML':
      ensure => '0.8.4-19.el6.x86_64';
    'SDL':
      ensure => '1.2.14-3.el6.x86_64';
    'apr':
      ensure => '1.3.9-5.el6_2.x86_64';
    'apr-util':
      ensure => '1.3.9-3.el6_0.1.x86_64';

If you were creating this manifest yourself, you would likely specify ensure => installed instead of a specific version.

Packages install default versions of files. Blueprint has no notion of this and will add all the files to the manifest, even those that have not changed. By default, Blueprint will indiscriminately capture all the files in /etc as file resources.

Blueprint and similar tools have a very small use case generally, but may help you to get familiar with the Puppet syntax and give you some ideas on how to specify your own manifests. I would not recommend blindly using this tool to create a system, however.

There's no shortcut to good configuration management, those who hope to save time and effort by cutting and pasting someone else's code as a whole (as with public modules) are likely to find that it saves neither.

just takes a snapshot of the system as it stands; it makes no intelligent decisions, and Blueprint captures all the files on the system and all the packages. It will generate a configuration much larger than you may actually require. For instance, when configuring a server, you may specify that you want the Apache package installed. The dependencies for the Apache package will be installed automatically and you need to specify them. When generating the configuration with a tool such as Blueprint, you will capture all those dependencies and lock the versions that are installed on your system currently. Looking at our generated Blueprint code, we can see that this is the case:

class yum {
  package {
    'GeoIP':
      ensure => '1.5.1-5.el6.x86_64';
    'PyXML':
      ensure => '0.8.4-19.el6.x86_64';
    'SDL':
      ensure => '1.2.14-3.el6.x86_64';
    'apr':
      ensure => '1.3.9-5.el6_2.x86_64';
    'apr-util':
      ensure => '1.3.9-3.el6_0.1.x86_64';

If you were creating this manifest yourself, you would likely specify ensure => installed instead of a specific version.

Packages install default versions of files. Blueprint has no notion of this and will add all the files to the manifest, even those that have not changed. By default, Blueprint will indiscriminately capture all the files in /etc as file resources.

Blueprint and similar tools have a very small use case generally, but may help you to get familiar with the Puppet syntax and give you some ideas on how to specify your own manifests. I would not recommend blindly using this tool to create a system, however.

There's no shortcut to good configuration management, those who hope to save time and effort by cutting and pasting someone else's code as a whole (as with public modules) are likely to find that it saves neither.

Using an external node classifier

When Puppet runs on a node, it needs to know which classes should be applied to that node. For example, if it is a web server node, it might need to include an apache class. The normal way to map nodes to classes is in the Puppet manifest itself, for example, in your site.pp file:

node 'web1' {
  include apache
}

Alternatively, you can use an External Node Classifier (ENC) to do this job. An ENC is any executable program that can accept the fully-qualified domain name (FQDN) as the first command-line argument ($1). The script is expected to return a list of classes, parameters, and an optional environment to apply to the node. The output is expected to be in the standard YAML format. When using an ENC, you should keep in mind that the classes applied through the standard site.pp manifest are merged with those provided by the ENC.

Note

Parameters returned by the ENC are available as top-scope variables to the node.

An ENC could be a simple shell script, for example, or a wrapper around a more complicated program or API that can decide how to map nodes to classes. The ENC provided by Puppet enterprise and The Foreman (http://theforeman.org/) are both simple scripts, which connect to the web API of their respective systems.

In this example, we'll build the most simple of ENCs, a shell script that simply prints a list of classes to include. We'll start by including an enc class, which defines notify that will print a top-scope variable $enc.

Getting ready

We'll start by creating our enc class to include with the enc script:

Run the following command:

t@mylaptop ~/puppet $ mkdir -p modules/enc/manifests

Create the file modules/enc/manifests/init.pp with the following contents:
```
class enc {
  notify {"We defined this from $enc": }
}
```

How to do it...

Here's how to build a simple external node classifier. We'll perform all these steps on our Puppet master server. If you are running masterless, then do these steps on a node:

Create the file /etc/puppet/cookbook.sh with the following contents:
```
#!/bin/bash
cat <<EOF
---
classes:
enc:
parameters:
  enc: $0
EOF
```

Run the following command:

root@puppet:/etc/puppet# chmod a+x cookbook.sh

Modify your /etc/puppet/puppet.conf file as follows:

[main]
  node_terminus = exec
  external_nodes = /etc/puppet/cookbook.sh

Restart Apache (restart the master) to make the change effective.
Ensure your site.pp file has the following empty definition for the default node:
```
node default {}
```

Run Puppet:

[root@cookbook ~]# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416376937'
Notice: We defined this from /etc/puppet/cookbook.sh
Notice: /Stage[main]/Enc/Notify[We defined this from /etc/puppet/cookbook.sh]/message: defined 'message' as 'We defined this from /etc/puppet/cookbook.sh'
Notice: Finished catalog run in 0.17 seconds

How it works...

When an ENC is set in puppet.conf, Puppet will call the specified program with the node's fqdn (technically, the certname variable) as the first command-line argument. In our example script, this argument is ignored, and it just outputs a fixed list of classes (actually, just one class).

Obviously this script is not terribly useful; a more sophisticated script might check a database to find the class list, or look up the node in a hash, or an external text file or database (often an organization's configuration management database, CMDB). Hopefully, this example is enough to get you started with writing your own external node classifier. Remember that you can write your script in any language you prefer.

There's more...

An ENC can supply a whole list of classes to be included in the node, in the following (YAML) format:

---
classes:
  CLASS1:
  CLASS2:
  CLASS3:

For classes that take parameters, you can use this format:

---
classes:
  mysql:
    package: percona-server-server-5.5
    socket:  /var/run/mysqld/mysqld.sock
    port:    3306

You can also produce top-scope variables using an ENC with this format:

---
parameters:
  message: 'Anyone home MyFly?'

Variables that you set in this way will be available in your manifest using the normal syntax for a top-scope variable, for example $::message.

How to do it...

Here's how to build a simple external node classifier. We'll perform all these steps on our Puppet master server. If you are running masterless, then do these steps on a node:

Create the file /etc/puppet/cookbook.sh with the following contents:
```
#!/bin/bash
cat <<EOF
---
classes:
enc:
parameters:
  enc: $0
EOF
```

Run the following command:

root@puppet:/etc/puppet# chmod a+x cookbook.sh

Modify your /etc/puppet/puppet.conf file as follows:

[main]
  node_terminus = exec
  external_nodes = /etc/puppet/cookbook.sh

Restart Apache (restart the master) to make the change effective.
Ensure your site.pp file has the following empty definition for the default node:
```
node default {}
```

Run Puppet:

[root@cookbook ~]# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416376937'
Notice: We defined this from /etc/puppet/cookbook.sh
Notice: /Stage[main]/Enc/Notify[We defined this from /etc/puppet/cookbook.sh]/message: defined 'message' as 'We defined this from /etc/puppet/cookbook.sh'
Notice: Finished catalog run in 0.17 seconds

How it works...

When an ENC is set in puppet.conf, Puppet will call the specified program with the node's fqdn (technically, the certname variable) as the first command-line argument. In our example script, this argument is ignored, and it just outputs a fixed list of classes (actually, just one class).

Obviously this script is not terribly useful; a more sophisticated script might check a database to find the class list, or look up the node in a hash, or an external text file or database (often an organization's configuration management database, CMDB). Hopefully, this example is enough to get you started with writing your own external node classifier. Remember that you can write your script in any language you prefer.

There's more...

An ENC can supply a whole list of classes to be included in the node, in the following (YAML) format:

---
classes:
  CLASS1:
  CLASS2:
  CLASS3:

For classes that take parameters, you can use this format:

---
classes:
  mysql:
    package: percona-server-server-5.5
    socket:  /var/run/mysqld/mysqld.sock
    port:    3306

You can also produce top-scope variables using an ENC with this format:

---
parameters:
  message: 'Anyone home MyFly?'

Variables that you set in this way will be available in your manifest using the normal syntax for a top-scope variable, for example $::message.

How it works...

When an ENC is set in puppet.conf, Puppet will call the specified program with the node's fqdn (technically, the certname variable) as the first command-line argument. In our example script, this argument is ignored, and it just outputs a fixed list of classes (actually, just one class).

Obviously this script is not terribly useful; a more sophisticated script might check a database to find the class list, or look up the node in a hash, or an external text file or database (often an organization's configuration management database, CMDB). Hopefully, this example is enough to get you started with writing your own external node classifier. Remember that you can write your script in any language you prefer.

There's more...

An ENC can supply a whole list of classes to be included in the node, in the following (YAML) format:

---
classes:
  CLASS1:
  CLASS2:
  CLASS3:

For classes that take parameters, you can use this format:

---
classes:
  mysql:
    package: percona-server-server-5.5
    socket:  /var/run/mysqld/mysqld.sock
    port:    3306

You can also produce top-scope variables using an ENC with this format:

---
parameters:
  message: 'Anyone home MyFly?'

Variables that you set in this way will be available in your manifest using the normal syntax for a top-scope variable, for example $::message.

There's more...

An ENC can supply a whole list of classes to be included in the node, in the following (YAML) format:

---
classes:
  CLASS1:
  CLASS2:
  CLASS3:

For classes that take parameters, you can use this format:

---
classes:
  mysql:
    package: percona-server-server-5.5
    socket:  /var/run/mysqld/mysqld.sock
    port:    3306

You can also produce top-scope variables using an ENC with this format:

---
parameters:
  message: 'Anyone home MyFly?'

Variables that you set in this way will be available in your manifest using the normal syntax for a top-scope variable, for example $::message.

Creating your own resource types

As you know, Puppet has a bunch of useful built-in resource types: packages, files, users, and so on. Usually, you can do everything you need to do by using either combinations of these built-in resources, or define, which you can use more or less in the same way as a resource (see Chapter 3, Writing Better Manifests for information on definitions).

In the early days of Puppet, creating your own resource type was more common as the list of core resources was shorter than it is today. Before you consider creating your own resource type, I suggest searching the Forge for alternative solutions. Even if you can find a project that only partially solves your problem, you will be better served by extending and helping out that project, rather than trying to create your own. However, if you need to create your own resource type, Puppet makes it quite easy. The native types are written in Ruby, and you will need a basic familiarity with Ruby in order to create your own.

Let's refresh our memory on the distinction between types and providers. A type describes a resource and the parameters it can have (for example, the package type). A provider tells Puppet how to implement a resource type for a particular platform or situation (for example, the apt/dpkg providers implement the package type for Debian-like systems).

A single type (package) can have many providers (APT, YUM, Fink, and so on). If you don't specify a provider when declaring a resource, Puppet will choose the most appropriate one given the environment.

We'll use Ruby in this section; if you are not familiar with Ruby try visiting http://www.ruby-doc.org/docs/Tutorial/ or http://www.codecademy.com/tracks/ruby/.

How to do it...

In this section, we'll see how to create a custom type that we can use to manage Git repositories, and in the next section, we'll write a provider to implement this type.

Create the file modules/cookbook/lib/puppet/type/gitrepo.rb with the following contents:

Puppet::Type.newtype(:gitrepo) do
  ensurable

  newparam(:source) do
    isnamevar
  end

  newparam(:path)
end

How it works...

Custom types can live in any module, in a lib/puppet/type subdirectory and in a file named for the type (in our example, that's modules/cookbook/lib/puppet/type/gitrepo.rb).

The first line of gitrepo.rb tells Puppet to register a new type named gitrepo:

Puppet::Type.newtype(:gitrepo) do

The ensurable line automatically gives the type an ensure property, such as Puppet's built-in resources:

ensurable

We'll now give the type some parameters. For the moment, all we need is a source parameter for the Git source URL, and a path parameter to tell Puppet where the repo should be created in the filesystem:

newparam(:source) do
  isnamevar
end

The isnamevar declaration tells Puppet that the source parameter is the type's namevar. So when you declare an instance of this resource, whatever name you give, it will be the value of source, for example:

gitrepo { 'git://github.com/puppetlabs/puppet.git':
  path => '/home/ubuntu/dev/puppet',
}

Finally, we tell Puppet that the type accepts the path parameter:

newparam(:path)

There's more...

When deciding whether or not you should create a custom type, you should ask a few questions about the resource you are trying to describe such as:

Is the resource enumerable? Can you easily obtain a list of all the instances of the resource on the system?
Is the resource atomic? Can you ensure that only one copy of the resource exists on the system (this is particularly important when you want to use ensure=>absent on the resource)?
Is there any other resource that describes this resource? In such a case, a defined type based on the existing resource would, in most cases, be a simpler solution.

Documentation

Our example is deliberately simple, but when you move on to developing real custom types for your production environment, you should add documentation strings to describe what the type and its parameters do, for example:

Puppet::Type.newtype(:gitrepo) do
  @doc = "Manages Git repos"

  ensurable

  newparam(:source) do
    desc "Git source URL for the repo"
    isnamevar
  end

  newparam(:path) do
    desc "Path where the repo should be created"
  end
end

Validation

You can use parameter validation to generate useful error messages when someone tries to pass bad values to the resource. For example, you could validate that the directory where the repo is to be created actually exists:

newparam(:path) do
  validate do |value|
    basepath = File.dirname(value)
    unless File.directory?(basepath)
      raise ArgumentError , "The path %s doesn't exist" % basepath
    end
  end
end

You can also specify the list of allowed values that the parameter can take:

newparam(:breakfast) do
  newvalues(:bacon, :eggs, :sausages)
end

How it works...

Custom types can live in any module, in a lib/puppet/type subdirectory and in a file named for the type (in our example, that's modules/cookbook/lib/puppet/type/gitrepo.rb).

The first line of gitrepo.rb tells Puppet to register a new type named gitrepo:

Puppet::Type.newtype(:gitrepo) do

The ensurable line automatically gives the type an ensure property, such as Puppet's built-in resources:

ensurable

We'll now give the type some parameters. For the moment, all we need is a source parameter for the Git source URL, and a path parameter to tell Puppet where the repo should be created in the filesystem:

newparam(:source) do
  isnamevar
end

The isnamevar declaration tells Puppet that the source parameter is the type's namevar. So when you declare an instance of this resource, whatever name you give, it will be the value of source, for example:

gitrepo { 'git://github.com/puppetlabs/puppet.git':
  path => '/home/ubuntu/dev/puppet',
}

Finally, we tell Puppet that the type accepts the path parameter:

newparam(:path)

There's more...

When deciding whether or not you should create a custom type, you should ask a few questions about the resource you are trying to describe such as:

Is the resource enumerable? Can you easily obtain a list of all the instances of the resource on the system?
Is the resource atomic? Can you ensure that only one copy of the resource exists on the system (this is particularly important when you want to use ensure=>absent on the resource)?
Is there any other resource that describes this resource? In such a case, a defined type based on the existing resource would, in most cases, be a simpler solution.

Documentation

Our example is deliberately simple, but when you move on to developing real custom types for your production environment, you should add documentation strings to describe what the type and its parameters do, for example:

Puppet::Type.newtype(:gitrepo) do
  @doc = "Manages Git repos"

  ensurable

  newparam(:source) do
    desc "Git source URL for the repo"
    isnamevar
  end

  newparam(:path) do
    desc "Path where the repo should be created"
  end
end

Validation

You can use parameter validation to generate useful error messages when someone tries to pass bad values to the resource. For example, you could validate that the directory where the repo is to be created actually exists:

newparam(:path) do
  validate do |value|
    basepath = File.dirname(value)
    unless File.directory?(basepath)
      raise ArgumentError , "The path %s doesn't exist" % basepath
    end
  end
end

You can also specify the list of allowed values that the parameter can take:

newparam(:breakfast) do
  newvalues(:bacon, :eggs, :sausages)
end

live in any module, in a lib/puppet/type subdirectory and in a file named for the type (in our example, that's modules/cookbook/lib/puppet/type/gitrepo.rb).

The first line of gitrepo.rb tells Puppet to register a new type named gitrepo:

Puppet::Type.newtype(:gitrepo) do

The ensurable line automatically gives the type an ensure property, such as Puppet's built-in resources:

ensurable

We'll now give the type some parameters. For the moment, all we need is a source parameter for the Git source URL, and a path parameter to tell Puppet where the repo should be created in the filesystem:

newparam(:source) do
  isnamevar
end

The isnamevar declaration tells Puppet that the source parameter is the type's namevar. So when you declare an instance of this resource, whatever name you give, it will be the value of source, for example:

gitrepo { 'git://github.com/puppetlabs/puppet.git':
  path => '/home/ubuntu/dev/puppet',
}

Finally, we tell Puppet that the type accepts the path parameter:

newparam(:path)

There's more...

When deciding whether or not you should create a custom type, you should ask a few questions about the resource you are trying to describe such as:

Is the resource enumerable? Can you easily obtain a list of all the instances of the resource on the system?
Is the resource atomic? Can you ensure that only one copy of the resource exists on the system (this is particularly important when you want to use ensure=>absent on the resource)?
Is there any other resource that describes this resource? In such a case, a defined type based on the existing resource would, in most cases, be a simpler solution.

Documentation

Our example is deliberately simple, but when you move on to developing real custom types for your production environment, you should add documentation strings to describe what the type and its parameters do, for example:

Puppet::Type.newtype(:gitrepo) do
  @doc = "Manages Git repos"

  ensurable

  newparam(:source) do
    desc "Git source URL for the repo"
    isnamevar
  end

  newparam(:path) do
    desc "Path where the repo should be created"
  end
end

Validation

You can use parameter validation to generate useful error messages when someone tries to pass bad values to the resource. For example, you could validate that the directory where the repo is to be created actually exists:

newparam(:path) do
  validate do |value|
    basepath = File.dirname(value)
    unless File.directory?(basepath)
      raise ArgumentError , "The path %s doesn't exist" % basepath
    end
  end
end

You can also specify the list of allowed values that the parameter can take:

newparam(:breakfast) do
  newvalues(:bacon, :eggs, :sausages)
end

resource you are trying to describe such as:

Is the resource enumerable? Can you easily obtain a list of all the instances of the resource on the system?
Is the resource atomic? Can you ensure that only one copy of the resource exists on the system (this is particularly important when you want to use ensure=>absent on the resource)?
Is there any other resource that describes this resource? In such a case, a defined type based on the existing resource would, in most cases, be a simpler solution.

Documentation

Our example is deliberately simple, but when you move on to developing real custom types for your production environment, you should add documentation strings to describe what the type and its parameters do, for example:

Puppet::Type.newtype(:gitrepo) do
  @doc = "Manages Git repos"

  ensurable

  newparam(:source) do
    desc "Git source URL for the repo"
    isnamevar
  end

  newparam(:path) do
    desc "Path where the repo should be created"
  end
end

Validation

You can use parameter validation to generate useful error messages when someone tries to pass bad values to the resource. For example, you could validate that the directory where the repo is to be created actually exists:

newparam(:path) do
  validate do |value|
    basepath = File.dirname(value)
    unless File.directory?(basepath)
      raise ArgumentError , "The path %s doesn't exist" % basepath
    end
  end
end

You can also specify the list of allowed values that the parameter can take:

newparam(:breakfast) do
  newvalues(:bacon, :eggs, :sausages)
end

to describe what the type and its parameters do, for example:

Puppet::Type.newtype(:gitrepo) do
  @doc = "Manages Git repos"

  ensurable

  newparam(:source) do
    desc "Git source URL for the repo"
    isnamevar
  end

  newparam(:path) do
    desc "Path where the repo should be created"
  end
end

Validation

You can use parameter validation to generate useful error messages when someone tries to pass bad values to the resource. For example, you could validate that the directory where the repo is to be created actually exists:

newparam(:path) do
  validate do |value|
    basepath = File.dirname(value)
    unless File.directory?(basepath)
      raise ArgumentError , "The path %s doesn't exist" % basepath
    end
  end
end

You can also specify the list of allowed values that the parameter can take:

newparam(:breakfast) do
  newvalues(:bacon, :eggs, :sausages)
end

when someone tries to pass bad values to the resource. For example, you could validate that the directory where the repo is to be created actually exists:

newparam(:path) do
  validate do |value|
    basepath = File.dirname(value)
    unless File.directory?(basepath)
      raise ArgumentError , "The path %s doesn't exist" % basepath
    end
  end
end

You can also specify the list of allowed values that the parameter can take:

newparam(:breakfast) do
  newvalues(:bacon, :eggs, :sausages)
end

Creating your own providers

In the previous section, we created a new custom type called gitrepo and told Puppet that it takes two parameters, source and path. However, so far, we haven't told Puppet how to actually check out the repo; in other words, how to create a specific instance of this type. That's where the provider comes in.

We saw that a type will often have several possible providers. In our example, there is only one sensible way to instantiate a Git repo, so we'll only supply one provider: git. If you were to generalize this type—to just repo, say—it's not hard to imagine creating several different providers depending on the type of repo, for example, git, svn, cvs, and so on.

How to do it...

We'll add the git provider, and create an instance of a gitrepo resource to check that it all works. You'll need Git installed for this to work, but if you're using the Git-based manifest management setup described in Chapter 2, Puppet Infrastructure, we can safely assume that Git is available.

Create the file modules/cookbook/lib/puppet/provider/gitrepo/git.rb with the following contents:

require 'fileutils'

Puppet::Type.type(:gitrepo).provide(:git) do
  commands :git => "git"

  def create
    git "clone", resource[:source], resource[:path]
  end

  def exists?
    File.directory? resource[:path]
  end
end

Modify your site.pp file as follows:

node 'cookbook' {
  gitrepo { 'https://github.com/puppetlabs/puppetlabs-git':
    ensure => present,
    path   => '/tmp/puppet',
  }
}

Run Puppet:

[root@cookbook ~]# puppet agent -t
Notice: /File[/var/lib/puppet/lib/puppet/type/gitrepo.rb]/ensure: defined content as '{md5}6471793fe2b4372d40289ad4b614fe0b'
Notice: /File[/var/lib/puppet/lib/puppet/provider/gitrepo]/ensure: created
Notice: /File[/var/lib/puppet/lib/puppet/provider/gitrepo/git.rb]/ensure: defined content as '{md5}f860388234d3d0bdb3b3ec98bbf5115b'
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416378876'
Notice: /Stage[main]/Main/Node[cookbook]/Gitrepo[https://github.com/puppetlabs/puppetlabs-git]/ensure: created
Notice: Finished catalog run in 2.59 seconds

How it works...

Custom providers can live in any module, in a lib/puppet/provider/TYPE_NAME subdirectory in a file named after the provider. (The provider is the actual program that is run on the system; in our example, the program is Git and the provider is in modules/cookbook/lib/puppet/provider/gitrepo/git.rb. Note that the name of the module is irrelevant.)

After an ntitial require line in git.rb, we tell Puppet to register a new provider for the gitrepo type with the following line:

Puppet::Type.type(:gitrepo).provide(:git) do

When you declare an instance of the gitrepo type in your manifest, Puppet will first of all check whether the instance already exists, by calling the exists? method on the provider. So we need to supply this method, complete with code to check whether an instance of the gitrepo type already exists:

def exists?
  File.directory? resource[:path]
end

This is not the most sophisticated implementation; it simply returns true if a directory exists matching the path parameter of the instance. A better implementation of exists? might check, for example, whether there is a .git subdirectory and that it contains valid Git metadata. But this will do for now.

If exists? returns true, then Puppet will take no further action because the specified resource exists (as far as Puppet knows). If it returns false, Puppet assumes the resource doesn't yet exist, and will try to create it by calling the provider's create method.

Accordingly, we supply some code for the create method that calls the git clone command to create the repo:

def create
  git "clone", resource[:source], resource[:path]
end

The method has access to the instance's parameters, which we need to know where to check out the repo from, and which directory to create it in. We get this by looking at resource[:source] and resource[:path].

There's more...

You can see that custom types and providers in Puppet are very powerful. In fact, they can do anything—at least, anything that Ruby can do. If you are managing some parts of your infrastructure with complicated define statements and exec resources, you may want to consider replacing these with a custom type. However, as stated previously, it's worth looking around to see if someone else has already done this before implementing your own.

Our example was very simple, and there is much more to learn about writing your own types. If you're going to distribute your code for others to use, or even if you aren't, it's a good idea to include tests with it. puppetlabs has a useful page on the interface between custom types and providers:

http://docs.puppetlabs.com/guides/custom_types.html

on implementing providers:

http://docs.puppetlabs.com/guides/provider_development.html

and a complete worked example of developing a custom type and provider, a little more advanced than that presented in this book:

http://docs.puppetlabs.com/guides/complete_resource_example.html

Chapter 2, Puppet Infrastructure, we can safely assume that Git is available.

Create the file modules/cookbook/lib/puppet/provider/gitrepo/git.rb with the following contents:

require 'fileutils'

Puppet::Type.type(:gitrepo).provide(:git) do
  commands :git => "git"

  def create
    git "clone", resource[:source], resource[:path]
  end

  def exists?
    File.directory? resource[:path]
  end
end

Modify your site.pp file as follows:

node 'cookbook' {
  gitrepo { 'https://github.com/puppetlabs/puppetlabs-git':
    ensure => present,
    path   => '/tmp/puppet',
  }
}

Run Puppet:

[root@cookbook ~]# puppet agent -t
Notice: /File[/var/lib/puppet/lib/puppet/type/gitrepo.rb]/ensure: defined content as '{md5}6471793fe2b4372d40289ad4b614fe0b'
Notice: /File[/var/lib/puppet/lib/puppet/provider/gitrepo]/ensure: created
Notice: /File[/var/lib/puppet/lib/puppet/provider/gitrepo/git.rb]/ensure: defined content as '{md5}f860388234d3d0bdb3b3ec98bbf5115b'
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416378876'
Notice: /Stage[main]/Main/Node[cookbook]/Gitrepo[https://github.com/puppetlabs/puppetlabs-git]/ensure: created
Notice: Finished catalog run in 2.59 seconds

How it works...

Custom providers can live in any module, in a lib/puppet/provider/TYPE_NAME subdirectory in a file named after the provider. (The provider is the actual program that is run on the system; in our example, the program is Git and the provider is in modules/cookbook/lib/puppet/provider/gitrepo/git.rb. Note that the name of the module is irrelevant.)

After an ntitial require line in git.rb, we tell Puppet to register a new provider for the gitrepo type with the following line:

Puppet::Type.type(:gitrepo).provide(:git) do

When you declare an instance of the gitrepo type in your manifest, Puppet will first of all check whether the instance already exists, by calling the exists? method on the provider. So we need to supply this method, complete with code to check whether an instance of the gitrepo type already exists:

def exists?
  File.directory? resource[:path]
end

This is not the most sophisticated implementation; it simply returns true if a directory exists matching the path parameter of the instance. A better implementation of exists? might check, for example, whether there is a .git subdirectory and that it contains valid Git metadata. But this will do for now.

If exists? returns true, then Puppet will take no further action because the specified resource exists (as far as Puppet knows). If it returns false, Puppet assumes the resource doesn't yet exist, and will try to create it by calling the provider's create method.

Accordingly, we supply some code for the create method that calls the git clone command to create the repo:

def create
  git "clone", resource[:source], resource[:path]
end

The method has access to the instance's parameters, which we need to know where to check out the repo from, and which directory to create it in. We get this by looking at resource[:source] and resource[:path].

There's more...

You can see that custom types and providers in Puppet are very powerful. In fact, they can do anything—at least, anything that Ruby can do. If you are managing some parts of your infrastructure with complicated define statements and exec resources, you may want to consider replacing these with a custom type. However, as stated previously, it's worth looking around to see if someone else has already done this before implementing your own.

Our example was very simple, and there is much more to learn about writing your own types. If you're going to distribute your code for others to use, or even if you aren't, it's a good idea to include tests with it. puppetlabs has a useful page on the interface between custom types and providers:

http://docs.puppetlabs.com/guides/custom_types.html

on implementing providers:

http://docs.puppetlabs.com/guides/provider_development.html

and a complete worked example of developing a custom type and provider, a little more advanced than that presented in this book:

http://docs.puppetlabs.com/guides/complete_resource_example.html

can live in any module, in a lib/puppet/provider/TYPE_NAME subdirectory in a file named after the provider. (The provider is the actual program that is run on the system; in our example, the program is Git and the provider is in modules/cookbook/lib/puppet/provider/gitrepo/git.rb. Note that the name of the module is irrelevant.)

After an ntitial require line in git.rb, we tell Puppet to register a new provider for the gitrepo type with the following line:

Puppet::Type.type(:gitrepo).provide(:git) do

When you declare an instance of the gitrepo type in your manifest, Puppet will first of all check whether the instance already exists, by calling the exists? method on the provider. So we need to supply this method, complete with code to check whether an instance of the gitrepo type already exists:

def exists?
  File.directory? resource[:path]
end

This is not the most sophisticated implementation; it simply returns true if a directory exists matching the path parameter of the instance. A better implementation of exists? might check, for example, whether there is a .git subdirectory and that it contains valid Git metadata. But this will do for now.

If exists? returns true, then Puppet will take no further action because the specified resource exists (as far as Puppet knows). If it returns false, Puppet assumes the resource doesn't yet exist, and will try to create it by calling the provider's create method.

Accordingly, we supply some code for the create method that calls the git clone command to create the repo:

def create
  git "clone", resource[:source], resource[:path]
end

The method has access to the instance's parameters, which we need to know where to check out the repo from, and which directory to create it in. We get this by looking at resource[:source] and resource[:path].

There's more...

You can see that custom types and providers in Puppet are very powerful. In fact, they can do anything—at least, anything that Ruby can do. If you are managing some parts of your infrastructure with complicated define statements and exec resources, you may want to consider replacing these with a custom type. However, as stated previously, it's worth looking around to see if someone else has already done this before implementing your own.

Our example was very simple, and there is much more to learn about writing your own types. If you're going to distribute your code for others to use, or even if you aren't, it's a good idea to include tests with it. puppetlabs has a useful page on the interface between custom types and providers:

http://docs.puppetlabs.com/guides/custom_types.html

on implementing providers:

http://docs.puppetlabs.com/guides/provider_development.html

and a complete worked example of developing a custom type and provider, a little more advanced than that presented in this book:

http://docs.puppetlabs.com/guides/complete_resource_example.html

and providers:

http://docs.puppetlabs.com/guides/custom_types.html

on implementing providers:

http://docs.puppetlabs.com/guides/provider_development.html

and a complete worked example of developing a custom type and provider, a little more advanced than that presented in this book:

http://docs.puppetlabs.com/guides/complete_resource_example.html

Creating custom functions

If you've read the recipe Using GnuPG to encrypt secrets in Chapter 4, Working with Files and Packages, then you've already seen an example of a custom function (in that example, we created a secret function, which shelled out to GnuPG). Let's look at custom functions in a little more detail now and build an example.

How to do it...

If you've read the recipe Distributing cron jobs efficiently in Chapter 6, Managing Resources and Files, you might remember that we used the inline_template function to set a random time for cron jobs to run, based on the hostname of the node. In this example, we'll take that idea and turn it into a custom function called random_minute:

Create the file modules/cookbook/lib/puppet/parser/functions/random_minute.rb with the following contents:

module Puppet::Parser::Functions
  newfunction(:random_minute, :type => :rvalue) do |args|
    lookupvar('hostname').sum % 60
  end
end

Modify your site.pp file as follows:

node 'cookbook' {
  cron { 'randomised cron job':
    command => '/bin/echo Hello, world >>/tmp/hello.txt',
    hour    => '*',
    minute  => random_minute(),
  }
}

Run Puppet:

[root@cookbook ~]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/random_minute.rb]/ensure: defined content as '{md5}e6ff40165e74677e5837027bb5610744'
Info: Loading facts
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416379652'
Notice: /Stage[main]/Main/Node[cookbook]/Cron[custom fuction example job]/ensure: created
Notice: Finished catalog run in 0.41 seconds

Check crontab with the following command:

[root@cookbook ~]# crontab -l
# HEADER: This file was autogenerated at Wed Nov 19 01:48:11 -0500 2014 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: run-backup
0 15 * * * /usr/local/bin/backup
# Puppet Name: custom fuction example job
15 * * * * /bin/echo Hallo, welt >>/tmp/hallo.txt

How it works...

Custom functions can live in any module, in the lib/puppet/parser/functions subdirectory in a file named after the function (in our example, random_minute.rb).

The function code goes inside a module ... end block like this:

module Puppet::Parser::Functions
  ...
end

We then call newfunction to declare our new function, passing the name (:random_minute) and the type of function (:rvalue):

newfunction(:random_minute, :type => :rvalue) do |args|

The :rvalue bit simply means that this function returns a value.

Finally, the function code itself is as follows:

    lookupvar('hostname').sum % 60

The lookupvar function lets you access facts and variables by name; in this case, hostname to get the name of the node we're running on. We use the Ruby sum method to get the numeric sum of the characters in this string, and then perform integer division modulo 60 to make sure the result is in the range 0..59.

There's more...

You can, of course, do a lot more with custom functions. In fact, anything you can do in Ruby, you can do in a custom function. You also have access to all the facts and variables that are in scope at the point in the Puppet manifest where the function is called, by calling lookupvar as shown in the example. You can also work on arguments, for example, a general purpose hashing function that takes two arguments: the size of the hash table and optionally the thing to hash. Create modules/cookbook/lib/puppet/parser/functions/hashtable.rb with the following contents:

module Puppet::Parser::Functions
  newfunction(:hashtable, :type => :rvalue) do |args|
    if args.length == 2
      hashtable=lookupvar(args[1]).sum
    else
      hashtable=lookupvar('hostname').sum
    end

    if args.length > 0
      size = args[0].to_i
    else
      size = 60
    end
    unless size == 0
      hashtable % size
    else
      0
    end
  end
end

Now we'll create a test for our hashtable function and alter site.pp as follows:

node cookbook {
  $hours = hashtable(24)
  $minutes = hashtable()
  $days = hashtable(30)
  $days_fqdn = hashtable(30,'fqdn')
  $days_ipaddress = hashtable(30,'ipaddress')
  notify {"\n hours=${hours}\n minutes=${minutes}\n days=${days}\n days_fqdn=${days_fqdn}\n days_ipaddress=${days_ipaddress}\n":}
}

Now, run Puppet and observe the values that are returned:

Notice:  hours=15
 minutes=15
 days=15
 days_fqdn=4
 days_ipaddress=2

Our simple definition quickly grew when we added the ability to add arguments. As with all programming, care should be taken when working with arguments to ensure that you do not have any error conditions. In the preceding code, we specifically looked for the situation where the size variable was 0, to avoid a divide by zero error.

To find out more about what you can do with custom functions, see the puppetlabs website:

http://docs.puppetlabs.com/guides/custom_functions.html

recipe Distributing cron jobs efficiently in Chapter 6, Managing Resources and Files, you might remember that we used the inline_template function to set a random time for cron jobs to run, based on the hostname of the node. In this example, we'll take that idea and turn it into a custom function called random_minute:

Create the file modules/cookbook/lib/puppet/parser/functions/random_minute.rb with the following contents:

module Puppet::Parser::Functions
  newfunction(:random_minute, :type => :rvalue) do |args|
    lookupvar('hostname').sum % 60
  end
end

Modify your site.pp file as follows:

node 'cookbook' {
  cron { 'randomised cron job':
    command => '/bin/echo Hello, world >>/tmp/hello.txt',
    hour    => '*',
    minute  => random_minute(),
  }
}

Run Puppet:

[root@cookbook ~]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/random_minute.rb]/ensure: defined content as '{md5}e6ff40165e74677e5837027bb5610744'
Info: Loading facts
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1416379652'
Notice: /Stage[main]/Main/Node[cookbook]/Cron[custom fuction example job]/ensure: created
Notice: Finished catalog run in 0.41 seconds

Check crontab with the following command:

[root@cookbook ~]# crontab -l
# HEADER: This file was autogenerated at Wed Nov 19 01:48:11 -0500 2014 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: run-backup
0 15 * * * /usr/local/bin/backup
# Puppet Name: custom fuction example job
15 * * * * /bin/echo Hallo, welt >>/tmp/hallo.txt

How it works...

Custom functions can live in any module, in the lib/puppet/parser/functions subdirectory in a file named after the function (in our example, random_minute.rb).

The function code goes inside a module ... end block like this:

module Puppet::Parser::Functions
  ...
end

We then call newfunction to declare our new function, passing the name (:random_minute) and the type of function (:rvalue):

newfunction(:random_minute, :type => :rvalue) do |args|

The :rvalue bit simply means that this function returns a value.

Finally, the function code itself is as follows:

    lookupvar('hostname').sum % 60

The lookupvar function lets you access facts and variables by name; in this case, hostname to get the name of the node we're running on. We use the Ruby sum method to get the numeric sum of the characters in this string, and then perform integer division modulo 60 to make sure the result is in the range 0..59.

There's more...

You can, of course, do a lot more with custom functions. In fact, anything you can do in Ruby, you can do in a custom function. You also have access to all the facts and variables that are in scope at the point in the Puppet manifest where the function is called, by calling lookupvar as shown in the example. You can also work on arguments, for example, a general purpose hashing function that takes two arguments: the size of the hash table and optionally the thing to hash. Create modules/cookbook/lib/puppet/parser/functions/hashtable.rb with the following contents:

module Puppet::Parser::Functions
  newfunction(:hashtable, :type => :rvalue) do |args|
    if args.length == 2
      hashtable=lookupvar(args[1]).sum
    else
      hashtable=lookupvar('hostname').sum
    end

    if args.length > 0
      size = args[0].to_i
    else
      size = 60
    end
    unless size == 0
      hashtable % size
    else
      0
    end
  end
end

Now we'll create a test for our hashtable function and alter site.pp as follows:

node cookbook {
  $hours = hashtable(24)
  $minutes = hashtable()
  $days = hashtable(30)
  $days_fqdn = hashtable(30,'fqdn')
  $days_ipaddress = hashtable(30,'ipaddress')
  notify {"\n hours=${hours}\n minutes=${minutes}\n days=${days}\n days_fqdn=${days_fqdn}\n days_ipaddress=${days_ipaddress}\n":}
}

Now, run Puppet and observe the values that are returned:

Notice:  hours=15
 minutes=15
 days=15
 days_fqdn=4
 days_ipaddress=2

Our simple definition quickly grew when we added the ability to add arguments. As with all programming, care should be taken when working with arguments to ensure that you do not have any error conditions. In the preceding code, we specifically looked for the situation where the size variable was 0, to avoid a divide by zero error.

To find out more about what you can do with custom functions, see the puppetlabs website:

http://docs.puppetlabs.com/guides/custom_functions.html

functions can live in any module, in the lib/puppet/parser/functions subdirectory in a file named after the function (in our example, random_minute.rb).

The function code goes inside a module ... end block like this:

module Puppet::Parser::Functions
  ...
end

We then call newfunction to declare our new function, passing the name (:random_minute) and the type of function (:rvalue):

newfunction(:random_minute, :type => :rvalue) do |args|

The :rvalue bit simply means that this function returns a value.

Finally, the function code itself is as follows:

    lookupvar('hostname').sum % 60

The lookupvar function lets you access facts and variables by name; in this case, hostname to get the name of the node we're running on. We use the Ruby sum method to get the numeric sum of the characters in this string, and then perform integer division modulo 60 to make sure the result is in the range 0..59.

There's more...

You can, of course, do a lot more with custom functions. In fact, anything you can do in Ruby, you can do in a custom function. You also have access to all the facts and variables that are in scope at the point in the Puppet manifest where the function is called, by calling lookupvar as shown in the example. You can also work on arguments, for example, a general purpose hashing function that takes two arguments: the size of the hash table and optionally the thing to hash. Create modules/cookbook/lib/puppet/parser/functions/hashtable.rb with the following contents:

module Puppet::Parser::Functions
  newfunction(:hashtable, :type => :rvalue) do |args|
    if args.length == 2
      hashtable=lookupvar(args[1]).sum
    else
      hashtable=lookupvar('hostname').sum
    end

    if args.length > 0
      size = args[0].to_i
    else
      size = 60
    end
    unless size == 0
      hashtable % size
    else
      0
    end
  end
end

Now we'll create a test for our hashtable function and alter site.pp as follows:

node cookbook {
  $hours = hashtable(24)
  $minutes = hashtable()
  $days = hashtable(30)
  $days_fqdn = hashtable(30,'fqdn')
  $days_ipaddress = hashtable(30,'ipaddress')
  notify {"\n hours=${hours}\n minutes=${minutes}\n days=${days}\n days_fqdn=${days_fqdn}\n days_ipaddress=${days_ipaddress}\n":}
}

Now, run Puppet and observe the values that are returned:

Notice:  hours=15
 minutes=15
 days=15
 days_fqdn=4
 days_ipaddress=2

Our simple definition quickly grew when we added the ability to add arguments. As with all programming, care should be taken when working with arguments to ensure that you do not have any error conditions. In the preceding code, we specifically looked for the situation where the size variable was 0, to avoid a divide by zero error.

To find out more about what you can do with custom functions, see the puppetlabs website:

http://docs.puppetlabs.com/guides/custom_functions.html

test for our hashtable function and alter site.pp as follows:

node cookbook {
  $hours = hashtable(24)
  $minutes = hashtable()
  $days = hashtable(30)
  $days_fqdn = hashtable(30,'fqdn')
  $days_ipaddress = hashtable(30,'ipaddress')
  notify {"\n hours=${hours}\n minutes=${minutes}\n days=${days}\n days_fqdn=${days_fqdn}\n days_ipaddress=${days_ipaddress}\n":}
}

Now, run Puppet and observe the values that are returned:

Notice:  hours=15
 minutes=15
 days=15
 days_fqdn=4
 days_ipaddress=2

Our simple definition quickly grew when we added the ability to add arguments. As with all programming, care should be taken when working with arguments to ensure that you do not have any error conditions. In the preceding code, we specifically looked for the situation where the size variable was 0, to avoid a divide by zero error.

To find out more about what you can do with custom functions, see the puppetlabs website:

http://docs.puppetlabs.com/guides/custom_functions.html

Testing your puppet manifests with rspec-puppet

It would be great if we could verify that our Puppet manifests satisfy certain expectations without even having to run Puppet. The rspec-puppet tool is a nifty tool to do this. Based on RSpec, a testing framework for Ruby programs, rspec-puppet lets you write test cases for your Puppet manifests that are especially useful to catch regressions (bugs introduced when fixing another bug), and refactoring problems (bugs introduced when reorganizing your code).

Getting ready

Here's what you'll need to do to install rspec-puppet.

Run the following commands:

t@mylaptop~ $ sudo puppet resource package rspec-puppet ensure=installed provider=gem
Notice: /Package[rspec-puppet]/ensure: created
package { 'rspec-puppet':
  ensure => ['1.0.1'],
}
t@mylaptop ~ $ sudo puppet resource package puppetlabs_spec_helper ensure=installed provider=gem
Notice: /Package[puppetlabs_spec_helper]/ensure: created
package { 'puppetlabs_spec_helper':
  ensure => ['0.8.2'],
}

How to do it...

Let's create an example class, thing, and write some tests for it.

Define the thing class:

class thing {
  service {'thing':
    ensure  => 'running',
    enable  => true,
    require => Package['thing'],
  }
  package {'thing':
    ensure => 'installed'
  }
  file {'/etc/thing.conf':
    content => 'fubar\n',
    mode    => 0644,
    require => Package['thing'],
    notify  => Service['thing'],
  }
}

Run the following commands:

t@mylaptop ~/puppet]$cd modules/thing
t@mylaptop~/puppet/modules/thing $ rspec-puppet-init
 + spec/
 + spec/classes/
 + spec/defines/
 + spec/functions/
 + spec/hosts/
 + spec/fixtures/
 + spec/fixtures/manifests/
 + spec/fixtures/modules/
 + spec/fixtures/modules/heartbeat/
 + spec/fixtures/manifests/site.pp
 + spec/fixtures/modules/heartbeat/manifests
 + spec/fixtures/modules/heartbeat/templates
 + spec/spec_helper.rb
 + Rakefile

Create the file spec/classes/thing_spec.rb with the following contents:

require 'spec_helper'

describe 'thing' do
  it { should create_class('thing') }
  it { should contain_package('thing') }
  it { should contain_service('thing').with(
    'ensure' => 'running'
  ) }
  it { should contain_file('/etc/things.conf') }
end

Run the following commands:

t@mylaptop ~/.puppet/modules/thing $ rspec
...F

Failures:

  1) thing should contain File[/etc/things.conf]
     Failure/Error: it { should contain_file('/etc/things.conf') }
       expected that the catalogue would contain File[/etc/things.conf]
     # ./spec/classes/thing_spec.rb:9:in `block (2 levels) in <top (required)>'

Finished in 1.66 seconds
4 examples, 1 failure

Failed examples:

rspec ./spec/classes/thing_spec.rb:9 # thing should contain File[/etc/things.conf]

How it works...

The rspec-puppet-init command creates a framework of directories for you to put your specs (test programs) in. At the moment, we're just interested in the spec/classes directory. This is where you'll put your class specs, one per class, named after the class it tests, for example, thing_spec.rb.

The spec code itself begins with the following statement, which sets up the RSpec environment to run the specs:

require 'spec_helper'

Then, a describe block follows:

describe 'thing' do
  ..
end

The describe identifies the class we're going to test (thing) and wraps the list of assertions about the class inside a do .. end block.

Assertions are our stated expectations of the thing class. For example, the first assertion is the following:

  it { should create_class('thing') }

The create_class assertion is used to ensure that the named class is actually created. The next line:

  it { should contain_package('thing') }

The contain_package assertion means what it says: the class should contain a package resource named thing.

Next, we test for the existence of the thing service:

it { should contain_service('thing').with(
  'ensure' => 'running'
) }

The preceding code actually contains two assertions. First, that the class contains a thing service:

contain_service('thing')

Second, that the service has an ensure attribute with the value running:

with(
  'ensure' => 'running'
)

You can specify any attributes and values you want using the with method, as a comma-separated list. For example, the following code asserts several attributes of a file resource:

it { should contain_file('/tmp/hello.txt').with(
  'content' => "Hello, world\n",
  'owner'   => 'ubuntu',
  'group'   => 'ubuntu',
  'mode'    => '0644'
) }

In our thing example, we need to only test that the file thing.conf is present, using the following code:

it { should contain_file('/etc/thing.conf') }

When you run the rake spec command, rspec-puppet will compile the relevant Puppet classes, run all the specs it finds, and display the results:

...F
Failures:
  1) thing should contain File[/etc/things.conf]
     Failure/Error: it { should contain_file('/etc/things.conf') }
       expected that the catalogue would contain File[/etc/things.conf]
     # ./spec/classes/thing_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 1.66 seconds
4 examples, 1 failure

As you can see, we defined the file in our test as /etc/things.conf but the file in the manifests is /etc/thing.conf, so the test fails. Edit thing_spec.rb and change /etc/things.conf to /etc/thing.conf:

  it { should contain_file('/etc/thing.conf') }

Now run rspec again:

t@mylaptop ~/.puppet/modules/thing $ rspec
....
Finished in 1.6 seconds
4 examples, 0 failures

There's more...

There are many conditions you can verify with rspec. Any resource type can be verified with contain_<resource type>(title). In addition to verifying your classes will apply correctly, you can also test functions and definitions by using the appropriate subdirectories within the spec directory (classes, defines, or functions).

You can find more information about rspec-puppet, including complete documentation for the assertions available and a tutorial, at http://rspec-puppet.com/.

When you want to start testing how your code applies to nodes, you'll need to look at another tool, beaker. Beaker works with various virtualization platforms to create temporary virtual machines to which Puppet code is applied. The results are then used for acceptance testing of the Puppet code. This method of testing and developing at the same time is known as Test-driven development (TDD). More information about beaker is available on the GitHub site at https://github.com/puppetlabs/beaker.

How to do it...

Let's create an example class, thing, and write some tests for it.

Define the thing class:

class thing {
  service {'thing':
    ensure  => 'running',
    enable  => true,
    require => Package['thing'],
  }
  package {'thing':
    ensure => 'installed'
  }
  file {'/etc/thing.conf':
    content => 'fubar\n',
    mode    => 0644,
    require => Package['thing'],
    notify  => Service['thing'],
  }
}

Run the following commands:

t@mylaptop ~/puppet]$cd modules/thing
t@mylaptop~/puppet/modules/thing $ rspec-puppet-init
 + spec/
 + spec/classes/
 + spec/defines/
 + spec/functions/
 + spec/hosts/
 + spec/fixtures/
 + spec/fixtures/manifests/
 + spec/fixtures/modules/
 + spec/fixtures/modules/heartbeat/
 + spec/fixtures/manifests/site.pp
 + spec/fixtures/modules/heartbeat/manifests
 + spec/fixtures/modules/heartbeat/templates
 + spec/spec_helper.rb
 + Rakefile

Create the file spec/classes/thing_spec.rb with the following contents:

require 'spec_helper'

describe 'thing' do
  it { should create_class('thing') }
  it { should contain_package('thing') }
  it { should contain_service('thing').with(
    'ensure' => 'running'
  ) }
  it { should contain_file('/etc/things.conf') }
end

Run the following commands:

t@mylaptop ~/.puppet/modules/thing $ rspec
...F

Failures:

  1) thing should contain File[/etc/things.conf]
     Failure/Error: it { should contain_file('/etc/things.conf') }
       expected that the catalogue would contain File[/etc/things.conf]
     # ./spec/classes/thing_spec.rb:9:in `block (2 levels) in <top (required)>'

Finished in 1.66 seconds
4 examples, 1 failure

Failed examples:

rspec ./spec/classes/thing_spec.rb:9 # thing should contain File[/etc/things.conf]

How it works...

The rspec-puppet-init command creates a framework of directories for you to put your specs (test programs) in. At the moment, we're just interested in the spec/classes directory. This is where you'll put your class specs, one per class, named after the class it tests, for example, thing_spec.rb.

The spec code itself begins with the following statement, which sets up the RSpec environment to run the specs:

require 'spec_helper'

Then, a describe block follows:

describe 'thing' do
  ..
end

The describe identifies the class we're going to test (thing) and wraps the list of assertions about the class inside a do .. end block.

Assertions are our stated expectations of the thing class. For example, the first assertion is the following:

  it { should create_class('thing') }

The create_class assertion is used to ensure that the named class is actually created. The next line:

  it { should contain_package('thing') }

The contain_package assertion means what it says: the class should contain a package resource named thing.

Next, we test for the existence of the thing service:

it { should contain_service('thing').with(
  'ensure' => 'running'
) }

The preceding code actually contains two assertions. First, that the class contains a thing service:

contain_service('thing')

Second, that the service has an ensure attribute with the value running:

with(
  'ensure' => 'running'
)

You can specify any attributes and values you want using the with method, as a comma-separated list. For example, the following code asserts several attributes of a file resource:

it { should contain_file('/tmp/hello.txt').with(
  'content' => "Hello, world\n",
  'owner'   => 'ubuntu',
  'group'   => 'ubuntu',
  'mode'    => '0644'
) }

In our thing example, we need to only test that the file thing.conf is present, using the following code:

it { should contain_file('/etc/thing.conf') }

When you run the rake spec command, rspec-puppet will compile the relevant Puppet classes, run all the specs it finds, and display the results:

...F
Failures:
  1) thing should contain File[/etc/things.conf]
     Failure/Error: it { should contain_file('/etc/things.conf') }
       expected that the catalogue would contain File[/etc/things.conf]
     # ./spec/classes/thing_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 1.66 seconds
4 examples, 1 failure

As you can see, we defined the file in our test as /etc/things.conf but the file in the manifests is /etc/thing.conf, so the test fails. Edit thing_spec.rb and change /etc/things.conf to /etc/thing.conf:

  it { should contain_file('/etc/thing.conf') }

Now run rspec again:

t@mylaptop ~/.puppet/modules/thing $ rspec
....
Finished in 1.6 seconds
4 examples, 0 failures

There's more...

There are many conditions you can verify with rspec. Any resource type can be verified with contain_<resource type>(title). In addition to verifying your classes will apply correctly, you can also test functions and definitions by using the appropriate subdirectories within the spec directory (classes, defines, or functions).

You can find more information about rspec-puppet, including complete documentation for the assertions available and a tutorial, at http://rspec-puppet.com/.

When you want to start testing how your code applies to nodes, you'll need to look at another tool, beaker. Beaker works with various virtualization platforms to create temporary virtual machines to which Puppet code is applied. The results are then used for acceptance testing of the Puppet code. This method of testing and developing at the same time is known as Test-driven development (TDD). More information about beaker is available on the GitHub site at https://github.com/puppetlabs/beaker.

How it works...

The rspec-puppet-init command creates a framework of directories for you to put your specs (test programs) in. At the moment, we're just interested in the spec/classes directory. This is where you'll put your class specs, one per class, named after the class it tests, for example, thing_spec.rb.

The spec code itself begins with the following statement, which sets up the RSpec environment to run the specs:

require 'spec_helper'

Then, a describe block follows:

describe 'thing' do
  ..
end

The describe identifies the class we're going to test (thing) and wraps the list of assertions about the class inside a do .. end block.

Assertions are our stated expectations of the thing class. For example, the first assertion is the following:

  it { should create_class('thing') }

The create_class assertion is used to ensure that the named class is actually created. The next line:

  it { should contain_package('thing') }

The contain_package assertion means what it says: the class should contain a package resource named thing.

Next, we test for the existence of the thing service:

it { should contain_service('thing').with(
  'ensure' => 'running'
) }

The preceding code actually contains two assertions. First, that the class contains a thing service:

contain_service('thing')

Second, that the service has an ensure attribute with the value running:

with(
  'ensure' => 'running'
)

You can specify any attributes and values you want using the with method, as a comma-separated list. For example, the following code asserts several attributes of a file resource:

it { should contain_file('/tmp/hello.txt').with(
  'content' => "Hello, world\n",
  'owner'   => 'ubuntu',
  'group'   => 'ubuntu',
  'mode'    => '0644'
) }

In our thing example, we need to only test that the file thing.conf is present, using the following code:

it { should contain_file('/etc/thing.conf') }

When you run the rake spec command, rspec-puppet will compile the relevant Puppet classes, run all the specs it finds, and display the results:

...F
Failures:
  1) thing should contain File[/etc/things.conf]
     Failure/Error: it { should contain_file('/etc/things.conf') }
       expected that the catalogue would contain File[/etc/things.conf]
     # ./spec/classes/thing_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 1.66 seconds
4 examples, 1 failure

As you can see, we defined the file in our test as /etc/things.conf but the file in the manifests is /etc/thing.conf, so the test fails. Edit thing_spec.rb and change /etc/things.conf to /etc/thing.conf:

  it { should contain_file('/etc/thing.conf') }

Now run rspec again:

t@mylaptop ~/.puppet/modules/thing $ rspec
....
Finished in 1.6 seconds
4 examples, 0 failures

There's more...

There are many conditions you can verify with rspec. Any resource type can be verified with contain_<resource type>(title). In addition to verifying your classes will apply correctly, you can also test functions and definitions by using the appropriate subdirectories within the spec directory (classes, defines, or functions).

You can find more information about rspec-puppet, including complete documentation for the assertions available and a tutorial, at http://rspec-puppet.com/.

When you want to start testing how your code applies to nodes, you'll need to look at another tool, beaker. Beaker works with various virtualization platforms to create temporary virtual machines to which Puppet code is applied. The results are then used for acceptance testing of the Puppet code. This method of testing and developing at the same time is known as Test-driven development (TDD). More information about beaker is available on the GitHub site at https://github.com/puppetlabs/beaker.

There's more...

There are many conditions you can verify with rspec. Any resource type can be verified with contain_<resource type>(title). In addition to verifying your classes will apply correctly, you can also test functions and definitions by using the appropriate subdirectories within the spec directory (classes, defines, or functions).

You can find more information about rspec-puppet, including complete documentation for the assertions available and a tutorial, at http://rspec-puppet.com/.

When you want to start testing how your code applies to nodes, you'll need to look at another tool, beaker. Beaker works with various virtualization platforms to create temporary virtual machines to which Puppet code is applied. The results are then used for acceptance testing of the Puppet code. This method of testing and developing at the same time is known as Test-driven development (TDD). More information about beaker is available on the GitHub site at https://github.com/puppetlabs/beaker.

Using librarian-puppet

When you begin to include modules from the forge in your Puppet infrastructure, keeping track of which versions you installed and ensuring consistency between all your testing areas can become a bit of a problem. Luckily, the tools we will discuss in the next two sections can bring order to your system. We will first begin with librarian-puppet, which uses a special configuration file named Puppetfile to specify the source location of your various modules.

Getting ready

We'll install librarian-puppet to work through the example.

Install librarian-puppet on your Puppet master, using Puppet of course:

root@puppet:~# puppet resource package librarian-puppet ensure=installed provider=gem
Notice: /Package[librarian-puppet]/ensure: created
package { 'librarian-puppet':
  ensure => ['2.0.0'],
}

Tip

If you are working in a masterless environment, install librarian-puppet on the machine from which you will be managing your code. Your gem install may fail if the Ruby development packages are not available on your master; install the ruby-dev package to fix this issue (use Puppet to do it).

How to do it...

We'll use librarian-puppet to download and install a module in this example:

Create a working directory for yourself; librarian-puppet will overwrite your modules directory by default, so we'll work in a temporary location for now:
```
root@puppet:~# mkdir librarian
root@puppet:~# cd librarian
```

Create a new Puppetfile with the following contents:

#!/usr/bin/env ruby
#^syntax detection

forge "https://forgeapi.puppetlabs.com"

# A module from the Puppet Forge
mod 'puppetlabs-stdlib'

Note

Alternatively, you can use librarian-puppet init to create an example Puppetfile and edit it to match our example:

root@puppet:~/librarian# librarian-puppet init
      create  Puppetfile

Now, run librarian-puppet to download and install the puppetlabs-stdlib module in the modules directory:

root@puppet:~/librarian# librarian-puppet install
root@puppet:~/librarian # ls
modules  Puppetfile  Puppetfile.lock
root@puppet:~/librarian # ls modules
stdlib

How it works...

The first line of the Puppetfile makes the Puppetfile appear to be a Ruby source file. These are completely optional but coerces editors into treating the file as though it was written in Ruby (which it is):

#!/usr/bin/env ruby

We next define where the Puppet Forge is located; you may specify an internal Forge here if you have a local mirror:

forge "https://forgeapi.puppetlabs.com"

Now, we added a line to include the puppetlabs-stdlib module:

mod 'puppetlabs-stdlib'

With the Puppetfile in place, we ran librarian-puppet and it downloaded the module from the URL given in the Forge line. As the module was downloaded, librarian-puppet created a Puppetfile.lock file, which includes the location used as source and the version number for the downloaded module:

FORGE
  remote: https://forgeapi.puppetlabs.com
  specs:
    puppetlabs-stdlib (4.4.0)

DEPENDENCIES
  puppetlabs-stdlib (>= 0)

There's more...

The Puppetfile allows you to pull in modules from sources other than the forge. You may use a local Git url or even a GitHub url to download modules that are not on the Forge. More information on librarian-puppet can be found on the GitHub website at https://github.com/rodjek/librarian-puppet.

Note that librarian-puppet will create the modules directory and remove any modules you placed in there by default. Most installations using librarian-puppet opt to place their local modules in a /local subdirectory (/dist or /companyname are also used).

In the next section, we'll talk about r10k, which goes one step further than librarian and manages your entire environment directory.

Tip

If you are working in a masterless environment, install librarian-puppet on the machine from which you will be managing your code. Your gem install may fail if the Ruby development packages are not available on your master; install the ruby-dev package to fix this issue (use Puppet to do it).

How to do it...

We'll use librarian-puppet to download and install a module in this example:

Create a working directory for yourself; librarian-puppet will overwrite your modules directory by default, so we'll work in a temporary location for now:
```
root@puppet:~# mkdir librarian
root@puppet:~# cd librarian
```

Create a new Puppetfile with the following contents:

#!/usr/bin/env ruby
#^syntax detection

forge "https://forgeapi.puppetlabs.com"

# A module from the Puppet Forge
mod 'puppetlabs-stdlib'

Note

Alternatively, you can use librarian-puppet init to create an example Puppetfile and edit it to match our example:

root@puppet:~/librarian# librarian-puppet init
      create  Puppetfile

Now, run librarian-puppet to download and install the puppetlabs-stdlib module in the modules directory:

root@puppet:~/librarian# librarian-puppet install
root@puppet:~/librarian # ls
modules  Puppetfile  Puppetfile.lock
root@puppet:~/librarian # ls modules
stdlib

How it works...

The first line of the Puppetfile makes the Puppetfile appear to be a Ruby source file. These are completely optional but coerces editors into treating the file as though it was written in Ruby (which it is):

#!/usr/bin/env ruby

We next define where the Puppet Forge is located; you may specify an internal Forge here if you have a local mirror:

forge "https://forgeapi.puppetlabs.com"

Now, we added a line to include the puppetlabs-stdlib module:

mod 'puppetlabs-stdlib'

With the Puppetfile in place, we ran librarian-puppet and it downloaded the module from the URL given in the Forge line. As the module was downloaded, librarian-puppet created a Puppetfile.lock file, which includes the location used as source and the version number for the downloaded module:

FORGE
  remote: https://forgeapi.puppetlabs.com
  specs:
    puppetlabs-stdlib (4.4.0)

DEPENDENCIES
  puppetlabs-stdlib (>= 0)

There's more...

The Puppetfile allows you to pull in modules from sources other than the forge. You may use a local Git url or even a GitHub url to download modules that are not on the Forge. More information on librarian-puppet can be found on the GitHub website at https://github.com/rodjek/librarian-puppet.

Note that librarian-puppet will create the modules directory and remove any modules you placed in there by default. Most installations using librarian-puppet opt to place their local modules in a /local subdirectory (/dist or /companyname are also used).

In the next section, we'll talk about r10k, which goes one step further than librarian and manages your entire environment directory.

Note
Alternatively, you can use librarian-puppet init to create an example Puppetfile and edit it to match our example:
```
root@puppet:~/librarian# librarian-puppet init
      create  Puppetfile
```

Now, run librarian-puppet to download and install the puppetlabs-stdlib module in the modules directory:

root@puppet:~/librarian# librarian-puppet install
root@puppet:~/librarian # ls
modules  Puppetfile  Puppetfile.lock
root@puppet:~/librarian # ls modules
stdlib

How it works...

The first line of the Puppetfile makes the Puppetfile appear to be a Ruby source file. These are completely optional but coerces editors into treating the file as though it was written in Ruby (which it is):

#!/usr/bin/env ruby

We next define where the Puppet Forge is located; you may specify an internal Forge here if you have a local mirror:

forge "https://forgeapi.puppetlabs.com"

Now, we added a line to include the puppetlabs-stdlib module:

mod 'puppetlabs-stdlib'

With the Puppetfile in place, we ran librarian-puppet and it downloaded the module from the URL given in the Forge line. As the module was downloaded, librarian-puppet created a Puppetfile.lock file, which includes the location used as source and the version number for the downloaded module:

FORGE
  remote: https://forgeapi.puppetlabs.com
  specs:
    puppetlabs-stdlib (4.4.0)

DEPENDENCIES
  puppetlabs-stdlib (>= 0)

There's more...

The Puppetfile allows you to pull in modules from sources other than the forge. You may use a local Git url or even a GitHub url to download modules that are not on the Forge. More information on librarian-puppet can be found on the GitHub website at https://github.com/rodjek/librarian-puppet.

Note that librarian-puppet will create the modules directory and remove any modules you placed in there by default. Most installations using librarian-puppet opt to place their local modules in a /local subdirectory (/dist or /companyname are also used).

In the next section, we'll talk about r10k, which goes one step further than librarian and manages your entire environment directory.

the Puppetfile makes the Puppetfile appear to be a Ruby source file. These are completely optional but coerces editors into treating the file as though it was written in Ruby (which it is):

#!/usr/bin/env ruby

We next define where the Puppet Forge is located; you may specify an internal Forge here if you have a local mirror:

forge "https://forgeapi.puppetlabs.com"

Now, we added a line to include the puppetlabs-stdlib module:

mod 'puppetlabs-stdlib'

With the Puppetfile in place, we ran librarian-puppet and it downloaded the module from the URL given in the Forge line. As the module was downloaded, librarian-puppet created a Puppetfile.lock file, which includes the location used as source and the version number for the downloaded module:

FORGE
  remote: https://forgeapi.puppetlabs.com
  specs:
    puppetlabs-stdlib (4.4.0)

DEPENDENCIES
  puppetlabs-stdlib (>= 0)

There's more...

The Puppetfile allows you to pull in modules from sources other than the forge. You may use a local Git url or even a GitHub url to download modules that are not on the Forge. More information on librarian-puppet can be found on the GitHub website at https://github.com/rodjek/librarian-puppet.

Note that librarian-puppet will create the modules directory and remove any modules you placed in there by default. Most installations using librarian-puppet opt to place their local modules in a /local subdirectory (/dist or /companyname are also used).

In the next section, we'll talk about r10k, which goes one step further than librarian and manages your entire environment directory.

you to pull in modules from sources other than the forge. You may use a local Git url or even a GitHub url to download modules that are not on the Forge. More information on librarian-puppet can be found on the GitHub website at https://github.com/rodjek/librarian-puppet.

Note that librarian-puppet will create the modules directory and remove any modules you placed in there by default. Most installations using librarian-puppet opt to place their local modules in a /local subdirectory (/dist or /companyname are also used).

In the next section, we'll talk about r10k, which goes one step further than librarian and manages your entire environment directory.

Using r10k

The Puppetfile is a very good format to describe which modules you wish to include in your environment. Building upon the Puppetfile is another tool, r10k. r10k is a total environment management tool. You can use r10k to clone a local Git repository into your environmentpath and then place the modules specified in your Puppetfile into that directory. The local Git repository is known as the master repository; it is where r10k expects to find your Puppetfile. r10k also understands Puppet environments and will clone Git branches into subdirectories of your environmentpath, simplifying the deployment of multiple environments. What makes r10k particularly useful is its use of a local cache directory to speed up deployments. Using a configuration file, r10k.yaml, you can specify where to store this cache and also where your master repository is held.

Getting ready

We'll install r10k on our controlling machine (usually the master). This is where we will control all the modules downloaded and installed.

Install r10k on your puppet master, or on whichever machine you wish to manage your environmentpath directory:

root@puppet:~# puppet resource package r10k ensure=installed provider=gem
Notice: /Package[r10k]/ensure: created
package { 'r10k':
  ensure => ['1.3.5'],
}

Make a new copy of your Git repository (optional, do this on your Git server):

[git@git repos]$ git clone --bare puppet.git puppet-r10k.git
Initialized empty Git repository in /home/git/repos/puppet-r10k.git/

Check out the new Git repository (on your local machine) and move the existing modules directory to a new location. We'll use /local in this example:

t@mylaptop ~ $ git clone [email protected]:repos/puppet-r10k.git
Cloning into 'puppet-r10k'...
remote: Counting objects: 2660, done.
remote: Compressing objects: 100% (2136/2136), done.
remote: Total 2660 (delta 913), reused 1049 (delta 238)
Receiving objects: 100% (2660/2660), 738.20 KiB | 0 bytes/s, done.
Resolving deltas: 100% (913/913), done.
Checking connectivity... done.
t@mylaptop ~ $ cd puppet-r10k/
t@mylaptop ~/puppet-r10k $ git checkout production
Branch production set up to track remote branch production from origin.
Switched to a new branch 'production'
t@mylaptop ~/puppet-r10k $ git mv modules local
t@mylaptop ~/puppet-r10k $ git commit -m "moving modules in preparation for r10k"
[master c96d0dc] moving modules in preparation for r10k
 9 files changed, 0 insertions(+), 0 deletions(-)
 rename {modules => local}/base (100%)
 rename {modules => local}/puppet/files/papply.sh (100%)
 rename {modules => local}/puppet/files/pull-updates.sh (100%)
 rename {modules => local}/puppet/manifests/init.pp (100%)

How to do it...

We'll create a Puppetfile to control r10k and install modules on our master.

Create a Puppetfile into the new Git repository with the following contents:

forge "http://forge.puppetlabs.com"
mod 'puppetlabs/puppetdb', '3.0.0'
mod 'puppetlabs/stdlib', '3.2.0'
mod 'puppetlabs/concat'
mod 'puppetlabs/firewall'

Add the Puppetfile to your new repository:

t@mylaptop ~/puppet-r10k $ git add Puppetfile
t@mylaptop ~/puppet-r10k $ git commit -m "adding Puppetfile"
[production d42481f] adding Puppetfile
 1 file changed, 7 insertions(+)
 create mode 100644 Puppetfile
t@mylaptop ~/puppet-r10k $ git push
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 589 bytes | 0 bytes/s, done.
Total 5 (delta 2), reused 0 (delta 0)
To [email protected]:repos/puppet-r10k.git
   cf8dfb9..d42481f  production -> production

Back to your master, create /etc/r10k.yaml with the following contents:

---
:cachedir: '/var/cache/r10k'
:sources:
 :plops:
  remote: '[email protected]:repos/puppet-r10k.git'
  basedir: '/etc/puppet/environments'

Run r10k to have the /etc/puppet/environments directory populated (hint: create a backup of your /etc/puppet/environments directory first):
```
root@puppet:~# r10k deploy environment -p
```

Verify that your /etc/puppet/environments directory has a production subdirectory. Within that directory, the /local directory will exist and the modules directory will have all the modules listed in the Puppetfile:

root@puppet:/etc/puppet/environments# tree -L 2
.
├── master
│   ├── manifests
│   ├── modules
│   └── README
└── production
    ├── environment.conf
    ├── local
    ├── manifests
    ├── modules
    ├── Puppetfile
    └── README

How it works...

We started by creating a copy of our Git repository; this was only done to preserve the earlier work and is not required. The important thing to remember with r10k and librarian-puppet is that they both assume they are in control of the /modules subdirectory. We need to move our modules out of the way and create a new location for the modules.

In the r10k.yaml file, we specified the location of our new repository. When we ran r10k, it first downloaded this repository into its local cache. Once the Git repository is downloaded locally, r10k will go through each branch and look for a Puppetfile within the branch. For each branch/Puppetfile combination, the modules specified within are downloaded first to the local cache directory (cachedir) and then into the basedir, which was given in r10k.yaml.

There's more...

You can automate the deployment of your environments using r10k. The command we used to run r10k and populate our environments directory can be easily placed inside a Git hook to automatically update your environment. There is also a marionette collective (mcollective) plugin (https://github.com/acidprime/r10k), which can be used to have r10k run on an arbitrary set of servers.

Using either of these tools will help keep your site consistent, even if you are not taking advantage of the various modules available on the Forge.

on our controlling machine (usually the master). This is where we will control all the modules downloaded and installed.

Install r10k on your puppet master, or on whichever machine you wish to manage your environmentpath directory:

root@puppet:~# puppet resource package r10k ensure=installed provider=gem
Notice: /Package[r10k]/ensure: created
package { 'r10k':
  ensure => ['1.3.5'],
}

Make a new copy of your Git repository (optional, do this on your Git server):

[git@git repos]$ git clone --bare puppet.git puppet-r10k.git
Initialized empty Git repository in /home/git/repos/puppet-r10k.git/

Check out the new Git repository (on your local machine) and move the existing modules directory to a new location. We'll use /local in this example:

t@mylaptop ~ $ git clone [email protected]:repos/puppet-r10k.git
Cloning into 'puppet-r10k'...
remote: Counting objects: 2660, done.
remote: Compressing objects: 100% (2136/2136), done.
remote: Total 2660 (delta 913), reused 1049 (delta 238)
Receiving objects: 100% (2660/2660), 738.20 KiB | 0 bytes/s, done.
Resolving deltas: 100% (913/913), done.
Checking connectivity... done.
t@mylaptop ~ $ cd puppet-r10k/
t@mylaptop ~/puppet-r10k $ git checkout production
Branch production set up to track remote branch production from origin.
Switched to a new branch 'production'
t@mylaptop ~/puppet-r10k $ git mv modules local
t@mylaptop ~/puppet-r10k $ git commit -m "moving modules in preparation for r10k"
[master c96d0dc] moving modules in preparation for r10k
 9 files changed, 0 insertions(+), 0 deletions(-)
 rename {modules => local}/base (100%)
 rename {modules => local}/puppet/files/papply.sh (100%)
 rename {modules => local}/puppet/files/pull-updates.sh (100%)
 rename {modules => local}/puppet/manifests/init.pp (100%)

How to do it...

We'll create a Puppetfile to control r10k and install modules on our master.

Create a Puppetfile into the new Git repository with the following contents:

forge "http://forge.puppetlabs.com"
mod 'puppetlabs/puppetdb', '3.0.0'
mod 'puppetlabs/stdlib', '3.2.0'
mod 'puppetlabs/concat'
mod 'puppetlabs/firewall'

Add the Puppetfile to your new repository:

t@mylaptop ~/puppet-r10k $ git add Puppetfile
t@mylaptop ~/puppet-r10k $ git commit -m "adding Puppetfile"
[production d42481f] adding Puppetfile
 1 file changed, 7 insertions(+)
 create mode 100644 Puppetfile
t@mylaptop ~/puppet-r10k $ git push
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 589 bytes | 0 bytes/s, done.
Total 5 (delta 2), reused 0 (delta 0)
To [email protected]:repos/puppet-r10k.git
   cf8dfb9..d42481f  production -> production

Back to your master, create /etc/r10k.yaml with the following contents:

---
:cachedir: '/var/cache/r10k'
:sources:
 :plops:
  remote: '[email protected]:repos/puppet-r10k.git'
  basedir: '/etc/puppet/environments'

Run r10k to have the /etc/puppet/environments directory populated (hint: create a backup of your /etc/puppet/environments directory first):
```
root@puppet:~# r10k deploy environment -p
```

Verify that your /etc/puppet/environments directory has a production subdirectory. Within that directory, the /local directory will exist and the modules directory will have all the modules listed in the Puppetfile:

root@puppet:/etc/puppet/environments# tree -L 2
.
├── master
│   ├── manifests
│   ├── modules
│   └── README
└── production
    ├── environment.conf
    ├── local
    ├── manifests
    ├── modules
    ├── Puppetfile
    └── README

How it works...

We started by creating a copy of our Git repository; this was only done to preserve the earlier work and is not required. The important thing to remember with r10k and librarian-puppet is that they both assume they are in control of the /modules subdirectory. We need to move our modules out of the way and create a new location for the modules.

In the r10k.yaml file, we specified the location of our new repository. When we ran r10k, it first downloaded this repository into its local cache. Once the Git repository is downloaded locally, r10k will go through each branch and look for a Puppetfile within the branch. For each branch/Puppetfile combination, the modules specified within are downloaded first to the local cache directory (cachedir) and then into the basedir, which was given in r10k.yaml.

There's more...

You can automate the deployment of your environments using r10k. The command we used to run r10k and populate our environments directory can be easily placed inside a Git hook to automatically update your environment. There is also a marionette collective (mcollective) plugin (https://github.com/acidprime/r10k), which can be used to have r10k run on an arbitrary set of servers.

Using either of these tools will help keep your site consistent, even if you are not taking advantage of the various modules available on the Forge.

your master, create /etc/r10k.yaml with the following contents:

---
:cachedir: '/var/cache/r10k'
:sources:
 :plops:
  remote: '[email protected]:repos/puppet-r10k.git'
  basedir: '/etc/puppet/environments'

Run r10k to have the /etc/puppet/environments directory populated (hint: create a backup of your /etc/puppet/environments directory first):
```
root@puppet:~# r10k deploy environment -p
```

Verify that your /etc/puppet/environments directory has a production subdirectory. Within that directory, the /local directory will exist and the modules directory will have all the modules listed in the Puppetfile:

root@puppet:/etc/puppet/environments# tree -L 2
.
├── master
│   ├── manifests
│   ├── modules
│   └── README
└── production
    ├── environment.conf
    ├── local
    ├── manifests
    ├── modules
    ├── Puppetfile
    └── README

How it works...

We started by creating a copy of our Git repository; this was only done to preserve the earlier work and is not required. The important thing to remember with r10k and librarian-puppet is that they both assume they are in control of the /modules subdirectory. We need to move our modules out of the way and create a new location for the modules.

In the r10k.yaml file, we specified the location of our new repository. When we ran r10k, it first downloaded this repository into its local cache. Once the Git repository is downloaded locally, r10k will go through each branch and look for a Puppetfile within the branch. For each branch/Puppetfile combination, the modules specified within are downloaded first to the local cache directory (cachedir) and then into the basedir, which was given in r10k.yaml.

There's more...

You can automate the deployment of your environments using r10k. The command we used to run r10k and populate our environments directory can be easily placed inside a Git hook to automatically update your environment. There is also a marionette collective (mcollective) plugin (https://github.com/acidprime/r10k), which can be used to have r10k run on an arbitrary set of servers.

Using either of these tools will help keep your site consistent, even if you are not taking advantage of the various modules available on the Forge.

creating a copy of our Git repository; this was only done to preserve the earlier work and is not required. The important thing to remember with r10k and librarian-puppet is that they both assume they are in control of the /modules subdirectory. We need to move our modules out of the way and create a new location for the modules.

In the r10k.yaml file, we specified the location of our new repository. When we ran r10k, it first downloaded this repository into its local cache. Once the Git repository is downloaded locally, r10k will go through each branch and look for a Puppetfile within the branch. For each branch/Puppetfile combination, the modules specified within are downloaded first to the local cache directory (cachedir) and then into the basedir, which was given in r10k.yaml.

There's more...

You can automate the deployment of your environments using r10k. The command we used to run r10k and populate our environments directory can be easily placed inside a Git hook to automatically update your environment. There is also a marionette collective (mcollective) plugin (https://github.com/acidprime/r10k), which can be used to have r10k run on an arbitrary set of servers.

Using either of these tools will help keep your site consistent, even if you are not taking advantage of the various modules available on the Forge.

(https://github.com/acidprime/r10k), which can be used to have r10k run on an arbitrary set of servers.

Using either of these tools will help keep your site consistent, even if you are not taking advantage of the various modules available on the Forge.

	"By all means leave the road when you wish. That is precisely the use of a road: to reach individually chosen points of departure."
	--Robert Bringhurst, The Elements of Typographic Style

You're reading from DevOps: Puppet, Docker, and Kubernetes Practical recipes to make the most of DevOps with powerful tools

Table of Contents (6) Chapters

Chapter 9. External Tools and the Puppet Ecosystem

Introduction

Creating custom facts

How to do it...

How it works...

Tip

There's more...

See also

How it works...

Tip

There's more...

See also

Tip

There's more...

See also

See also

Adding external facts

Getting ready

How to do it...

How it works...

Tip

There's more...

Debugging external facts

Note

Using external facts in Puppet

See also

How to do it...

How it works...

Tip

There's more...

Debugging external facts

Note

Using external facts in Puppet

See also

How it works...

Tip

There's more...

Debugging external facts

Note

Using external facts in Puppet

See also

Tip

There's more...

Debugging external facts

Note

Using external facts in Puppet

See also

Debugging external facts

Note

Using external facts in Puppet

See also

Note

Using external facts in Puppet

Setting facts as environment variables

How to do it...

Generating manifests with the Puppet resource command

How to do it...

There's more...

There's more...

Generating manifests with other tools

Getting ready

Tip

How to do it...

There's more...

Tip

How to do it...

There's more...

There's more...

Using an external node classifier

Note

Getting ready

How to do it...

How it works...

There's more...

See also

How to do it...

How it works...

There's more...