Creating a decentralized Puppet architecture

Some systems work best when they're decentralized. The Mafia is a good example, although, of course, there is no Mafia.

A common way to use Puppet is to run a Puppet Master server, which Puppet clients can then connect to and receive their manifests. However, you don't need a Puppet Master to use Puppet. You can run the puppet apply command directly on a manifest file to have Puppet apply it:

ubuntu@cookbook:~/puppet$ puppet apply manifests/site.pp
Notice: Finished catalog run in 0.08 seconds

In other words, if you can arrange to distribute a suitable manifest file to a client machine, you can have Puppet execute it directly without the need for a central Puppet Master. This removes the performance bottleneck of a single master server, and also eliminates a single point of failure. It also avoids having to sign and exchange the SSL certificates when provisioning a new client machine.

There are many ways you could deliver the manifest file to the client, but Git (or any version control system) does most of the work for you. You can edit your manifests in a local working copy, commit them to Git, and push them to a central repo, and from there they can be automatically distributed to the client machines.

Getting ready

If your Puppet manifests aren't already in Git, follow the steps in Managing your manifests with Git.

You'll need a second machine to check out a copy of your Puppet repo. If you're using EC2 instances, create another instance, and call it something like cookbook2.

How to do it...

Follow these steps:

1. Check out your GitHub repo on the new machine:

   ubuntu@cookbook2:~$ git clone 
     [email protected]:bitfield/cookbook.git puppet
   Cloning into 'puppet'...
   remote: Counting objects: 8, done.
   remote: Compressing objects: 100% (5/5), done.
   remote: Total 8 (delta 0), reused 5 (delta 0)
   Receiving objects: 100% (8/8), done.
   

2. Modify your manifests/nodes.pp file, as follows:

   node 'cookbook', 'cookbook2' {
     file { '/tmp/hello':
       content => "Hello, world\n",
     }
   }

3. Run the following command:

   ubuntu@cookbook2:~/puppet$ sudo puppet apply manifests/site.pp
   Notice: /Stage[main]//Node[cookbook2]/File[/tmp/hello]/ensure:
   defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'
   Notice: Finished catalog run in 0.05 seconds
   

How it works...

We've created a new working copy of the Puppet repo on the new machine:

ubuntu@cookbook2:~$ git clone [email protected]:bitfield/cookbook.git puppet

However, before we can run Puppet, we have to create a node declaration for the cookbook2 node:

node 'cookbook', 'cookbook2' {
  ...
}

Now, we apply the manifest:

ubuntu@cookbook2:~/puppet$ sudo puppet apply manifests/site.pp

Puppet finds the node declaration for cookbook2 and applies the same manifest that we used before on cookbook:

Notice: /Stage[main]//Node[cookbook2]/File[/tmp/hello]/ensure:
  defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'

There's more...

Having scaled your Puppet infrastructure from one machine to the other, you can now extend it to as many as you like! All you need to do is check out the Git repo on a machine, and run puppet apply.

This is a great way to add Puppet management to your existing machines without lots of complicated setup, or using an extra machine to serve as a Puppet Master. Many of my clients have switched to using a Git-based infrastructure because it's simpler, easier to scale, and easier to maintain.

A refinement which you might like to consider is having each machine automatically pull changes from GitHub and apply them with Puppet. Then all you need to do is push a change to GitHub, and it will roll out to all your Puppet-managed machines within a certain time. We'll see how to do this in the following sections.