DevOps tools: Packer

opsworks-packer-logo

Packer is a pretty comfortable tool for environment construction. The main purpose of Packer could be splitted onto few steps. At first – build clean guest machine from OS image with system tweaks, required for development. Later – add provisioning scripts and collect everything to the full image that could be launched in any moment. If you set build parameters properly, process is automated and doesn’t need your participance. Other feature of Packer is ability to build images for plenty of VM platforms. And, surely, Vagrant compatability. Also Packer is cross platforming. You can build images in any system – Linux, Windows or Mac OS X.

Packer configuration included in JSON file with special instructions. I’d like to describe Packer use through my personal experience – Debian 8 build with LEMP environment.

1. Image internal settings.


The first part of Packer image configuration. You can make modify your config much easier with variables. Just type into special directive all variables you need and use it below. Later you won’t have any troubles to alter build parameters.

"variables": {
       "debian_version": "8.2.0",
	"ssh": "vagrant"
    }

The main image configuration is forming into “builders” directive. Some parameters is required, some is not and set by default. You have to describe what VM platform you will use after building. This example has a VirtualBox, but you can choose everything and build it in the one Packer workflow. You can find list of supported platforms in the Packer documentation. Also there will be type of OS – special word that`s possible for your platform to understand. My guest OS is Debian x64. Don`t forget to type the name of your GuestAdditions image if you use VirtualBox. At last set hostname of your machine.

What’s also important for us?

  • Hardware settings (CPU cores, RAM size and disk size);
  • Path to OS installer (it could be local path or Web-source);
  • Path to preconfiguration file;
  • SSH settings for user authentication at post-installation processes. If you build a vagrant box – setup vagrant user and password. I did it through variable.
 "builders": [
    {
      "type": "virtualbox-iso",
      “guest_os_type": "Debian_64",
      "guest_additions_path": "VBoxGuestAdditions.iso",
      "virtualbox_version_file": ".vbox_version",
      "vm_name": "debian8",
	  
	  "vboxmanage": [
        [ "modifyvm", "{{.Name}}", "--memory", "1024" ],
        [ "modifyvm", "{{.Name}}", "--cpus", "2" ]
      ],

      "disk_size": 15000,
      
	"iso_checksum": "762eb3dfc22f85faf659001ebf270b4f",
      	"iso_checksum_type": "md5",
       "iso_url": "D:/iso_exe/debian-{{user `debian_version`}}-amd64-netinst.iso",
	  
	"http_directory": "http",
      	"ssh_username": "{{user `ssh`}}",
      	"ssh_password": "{{user `ssh`}}",
      	"ssh_port": 22,
      	"ssh_wait_timeout": "10000s"

2. OS installation instructions.


clip_image006_thumb1

The backbone of Packer automatization. Packer is perfect for experts, cause they can install OS through auto-installation file. If you have no idea how to install Linux distro automatically, I’d suggest you to learn this field. To tell the truth, now I don’t know it too. This example based on ready configuration by deimosfr Github repository.

Boot commands included in special subdirectory of “builders” template. You can set the boot wait time. This is an interval between VM launch and boot shell launch. Also you have to declare shutdown command that will be run after installation. Inside “boot command” directive you can type any key you need to go through boot command typing. Special parameter “wait” is necessary for hardware synchronization. Sometimes, when you need one keyboard interrupt to type one letter, Packer does more. Consequently boot instructions becomes corrupted. “Wait” command helps to fix it.

One boot command usually should execute preconfiguration file which contains installation instructions. In Debian-based systems this is a preseed file with particular syntax. RHEL systems have “kickstarter”. You can learn how to make this file with documentation of your OS.

         "boot_wait": "10s",
     	 "shutdown_command": "echo 'vagrant'|sudo -S /sbin/shutdown -hP now",
	  
	"boot_command": [
        "",
        "install ",
		"preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed_jessie.cfg ",
        "debian-installer=en_US ",
        "auto ",
        "locale=en_US ",
        "kbd-chooser/method=us ",
        "keyboard-configuration/xkb-keymap=us ",
        "netcfg/get_hostname={{ .Name }} ",
        "netcfg/get_domain=vagrantup.com ",
        "fb=false ",
        "debconf/frontend=noninteractive ",
        "console-setup/ask_detect=false ",
        "console-keymaps-at/keymap=us ",
        ""
		]
	}

3. Shell scripts.


bash

After guest OS backbone formation you can configure this machine by provisioners. You have a wide choice of tools in Packer. There are Bash scripts, CMS (Ansible, Chef, Puppet, Salt) or Powershell for Windows builds. Now we’ll focus on shell-scripts and Ansible playbooks.

Below is a description of necessary scripts for environment customization:

  • apt

Updates every system and gets some more for your requirements.

  • ansible

Runs pip as a Python package manager and installs Ansible. It could be done through the apt too, but now apt has an obsolete Ansible version. Pip is more suitable way, because some Ansible features are supported only in newest versions.

  • networking

Configures network settings for VirtualBox and Vagrant. At first it removes persistent-net-rules. After rules deeting your image won`t keep static interfaces name any longer. At last it deletes internal dhcp settings and set “pre-up sleep 2” parameter for correct “vagrant ssh” execution.

  • sudoers

Here is just a settings of /etc/sudoers. You have to provide all rights for user to run Vagrant box properly. Also it excludes tty, because it`s not necessary to have it in Vagrant.

  • vagrant

Here the vagrant user is set (if it’s not set yet) with home directory. After that new SSH public key will be generated and owned by vagrant user with all required access rights. If you skip this issue, vagrant up won`t work for your box. At the end of script I`ve added some SSH configuration parameter just in case.

  • vbaddguest

If you focused on VirtualBox environment, your box will be buggy without GuestAdditions installation. This scripts takes ISO-file you pointed before in json, mounts it in /tmp, executes .run file and removes everything used to install at last.

  • cleanup

Makes environment cleanup of settings you did. This script removes all existed Linux-images and command history for security.

"provisioners": [
    {
      "type": "shell",
      "execute_command": "echo 'vagrant' | {{.Vars}} sudo -E -S sh '{{.Path}}'",
      "scripts": [
        "scripts/apt.sh",
	"scripts/ansible.sh",
        "scripts/networking.sh",
        "scripts/sudoers.sh",
	"scripts/vagrant.sh",
        "scripts/vbaddguest.sh",
        "scripts/cleanup.sh"
      ]
    }

4. Ansible playbooks.


4xbn0u92odosejn9izlr0g-ansible2014_logo_black_tagline

Ansible is a flexible tool to build a project environment by software installation and configuration. It`s still a provisioner, but another one. Describe a local path to your playbooks in JSON config. Also there should be path to inventory file (you need to put there only localhost), path to every executed role and variables files directory. Be careful to make it all. When any shell script will be stopped with error/exception, Packer will stop to build your image. After that you have to try again.

{
      "type": "ansible-local",
      "playbook_file": "ansible/group2.yml",
      "inventory_file": "ansible/hosts",
      "role_paths": [
      "ansible/roles/package_install",
      "ansible/roles/php",
      "ansible/roles/nginx",
      "ansible/roles/mysql"
      ],
      “group_vars”: “ansible/group_vars”,
      “host_vars”: “ansible/host_vars”
    }

5. Final Vagrant box assembly.


If you want to build Packer artefacts as a Vagrant box, you can use post-processors template. Generally you can make the post-processor personally. Fortunately, Vagrant has its own post-processor. All you need -declare it and type the output name of box. Pay close attention: Packer has built-in environmental variables, it`s pretty comfortable to use it. Full list of this variables is here.

"post-processors": [
    {
      "type": "vagrant",
      "output": "debian{{user `debian_version`}}_{{.Provider}}.box"
    }
	]

6. Build an image through the command line.


The last step is an execution itself. It reads all you pointed on JSON. Get a command line, move to your project directory, check your file syntax and build it! Here is the steps.

cd [path/to/packer_builds]
packer validate test.json
packer build test.json

Don’t forget to check that Packer binary files layout should be inside of directory included in PATH variable. The picture during the building looks like that.

2.jpg

Here is the full json-configuration of Packer.

{
    "variables": {
        "debian_version": "8.2.0",
		"ssh": "vagrant"
    },
	
  "builders": [
    {
	  "type": "virtualbox-iso",
	  "guest_os_type": "Debian_64",
      "guest_additions_path": "VBoxGuestAdditions.iso",
      "virtualbox_version_file": ".vbox_version",
      "vm_name": "debian8",
	  
	  "vboxmanage": [
        [ "modifyvm", "{{.Name}}", "--memory", "1024" ],
        [ "modifyvm", "{{.Name}}", "--cpus", "2" ]
      ],
      "disk_size": 15000,
      
	  "iso_checksum": "762eb3dfc22f85faf659001ebf270b4f",
      "iso_checksum_type": "md5",
      "iso_url": "D:/iso_exe/debian-{{user `debian_version`}}-amd64-netinst.iso",
	  
	  "http_directory": "http",
      "ssh_username": "{{user `ssh`}}",
      "ssh_password": "{{user `ssh`}}",
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
	  
	  "boot_wait": "10s",
      "shutdown_command": "echo 'vagrant'|sudo -S /sbin/shutdown -hP now",
	  
	  "boot_command": [
        "",
        "install ",
		"preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed_jessie.cfg ",
        "debian-installer=en_US ",
        "auto ",
        "locale=en_US ",
        "kbd-chooser/method=us ",
        "keyboard-configuration/xkb-keymap=us ",
        "netcfg/get_hostname={{ .Name }} ",
        "netcfg/get_domain=vagrantup.com ",
        "fb=false ",
        "debconf/frontend=noninteractive ",
        "console-setup/ask_detect=false ",
        "console-keymaps-at/keymap=us ",
        ""
		]
	}
	],
	
	  "provisioners": [
    {
      "type": "shell",
      "execute_command": "echo 'vagrant' | {{.Vars}} sudo -E -S sh '{{.Path}}'",
      "scripts": [
        "scripts/apt.sh",
		"scripts/ansible.sh",
        "scripts/networking.sh",
        "scripts/sudoers.sh",
		"scripts/vagrant.sh",
        "scripts/vbaddguest.sh",
        "scripts/cleanup.sh"
      ]
    },
	{
      "type": "ansible-local",
	  "playbook_file": "ansible/group2.yml",
	  "inventory_file": "ansible/hosts",
      "role_paths": [
      "ansible/roles/package_install",
	  "ansible/roles/php",
      "ansible/roles/nginx",
	  "ansible/roles/mysql"
      ]
    }
  ],
  
      "post-processors": [
    {
      "type": "vagrant",
      "output": "debian{{user `debian_version`}}_{{.Provider}}.box"
    }

If you have any more questions about structure of Packer-project, you can take a look on my GitHub repository. It has all Shell scripts and Ansible roles described above.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s