Test-Driven Chef Cookbooks with Test Kitchen

By: July 18, 2013

You can find information on the interwebs about setting up test-kitchenberkshelf,  and minitest-chef-handler but nothing that seems to bring them all together in a quick tutorial. At least if there is, I haven't found it.

Using the information I distilled from a couple blog posts and the github repos related to the tools mentioned above, I put together some steps for initializing a cookbook with test-kitchen and berkshelf to allow creating and executing tests on a virtualbox vm provisioned using chef and vagrant. Let's get started with the setup.

Gathering the Ingredients

The following software is required to run this tutorial:

  • Virtualbox: Software that allows virtualized environments to run on your system.
  • vagrant: Software that provides an easy to use interface with various virtual environment providers.
  • chef: Configuration management software used to provision infrastructure via cookbooks(collections of scripts).
  • berkshelf: Used to organize the cookbooks required by the infrastructure.
  • test-kitchen: An application that tests cookbooks by running tests across a user defined set of platforms.

Install vagrant and Virtualbox for your system using the directions provided on their websites.

Ruby version 1.9.1 or greater is required. You can check your version by running ruby --version  on the commandline. If your version is not sufficient, check the Download Ruby page for ways to get a newer version.

Next, install the cheftest-kitchen and berkshelf gems. You may need to use sudo if your user doesn't have permissions to install gems.

Preparing the Workspace

So now that we have all the tools needed to test, lets create and test a cookbook on ubuntu 12.04. Start with creating a repo using your favorite version control tool. In this example we use git.

Layout the cookbook using the knife command provided by chef. This command will layout the files and directories contained in a cookbook.

In the new cookbook directory, run the kitchen init command to initialize your cookbook to support test-kitchen. You may need to use sudo kitchen init if your user cannot install gems.

Now, since we only want to test on ubuntu 12.04, lets edit test-kitchen's config contained in the .kitchen.yml file. Open the .kitchen.yml file and remove all the platforms except for ubuntu-12.04. The platforms section should look similar to this:

At this point, test-kitchen is setup to run on the ubuntu-12.04 platform. Running kitchen test  will spin up a vagrant vm running ubuntu 12.04, install chef using the omnibus chef packages from opscode and attempt to converge your empty cookbook. Now to create some tests.

Mixing the Ingredients

This is not meant to be an example of how to properly test a cookbook but more for providing an example of setting up a cookbook to test using minitest-chef-handler which runs checks on your system at the end of a chef run.

Minitest-chef-handler requires some setup before we can start testing. First, create a cookbook in the  test/cookbooks directory that will contain all of our tests:

Next, create and edit the Berksfile in the root directory of the salmon cookbook. The  Berksfile contains the dependencies of the cookbook being developed. Add the following to provide test-kitchen with the needed dependencies to test your cookbook.

Next, include the recipe you want to test in the corresponding recipe in the test cookbook. Add the following to test/cookbooks/salmon_test/recipes/default.rb:

Lastly, setup your .kitchen.yml to load the new test cookbook into the test suite by changing the suites section to the following:

Baking the Masterpiece

Now in proper TDD fashion, lets write a failing test and fix it. Create the test/cookbooks/salmon_test/files/default/tests/minitest/ directory and put the following code into the default_test.rb file in the directory.

The test in this file above looks for a file called  capers in the /home directory that is expected to be created by the default recipe of the salmon cookbook.

Run the tests with kitchen test. They should fail:

You may notice in the output that the virtual machine was not destroyed. When tests fail, test-kitchen doesn't destroy the virtual machine so it can be examined to determine the cause of the failure. To login to the vm, just run kitchen login default-ubuntu-1204.

Now let's fix the test by adding the following to the default recipe in recipes/default.rb of the cookbook under test:

Run the tests again with kitchen test. They should pass this time and the virtual machine will be destroyed at the end:

Eating the Cake

When I first started looking at test-kitchen, I thought it was the tool that ties everything together. But the further down the chef testing rabbit hole I went, I found that test-kitchen is but one of many testing tools used to properly test chef cookbooks. Since a cookbook is software, it requires testing just like any other software project. A mix of lint checking, syntax checking, unit-testing and integration tests are required. Having chef converge the cookbooks on a system is a test in itself but in order to minimize regressions, a proper unit-test suite should be run. See Unit Testing Chef Cookbooks by Seth Vargo for some great information on that topic.

Tools for testing the areas mentioned above already exist. The foodcritic utility handles lint checking. The knife utility syntax checks the cookbook using knife cookbook testChefSpec and minitest-chef-handler provide the frameworks for creating both unit and integration tests. Test-kitchen handles the execution of integration tests across different platforms.

I'm thankful for the enormous amounts of work put into creating these tools and the awesome community surrounding chef. There's still a lot of work to be done in documenting and providing working examples of using these tools for proper chef testing.  Hopefully, this post alleviates some pain and helps someone else get started testing their cookbooks.

  • tolleiv

    Thanks for the good summary. Using this type of test-workflow means ~7-10mins per test-run (based on Vagrant) ... that's quite hard when you really want to work in a TDD style. Do you have any recommendations on how to shorted this? (Especially during development)

  • tolleiv

    Just to answer my own question - kitchen converge helps a lot to minimize testing time.

  • Nickolay Yurin

    Try use docker instead of vagrant, it's much faster and quite simple to setup.