DevOps tools: Ansible

4xbn0u92odosejn9izlr0g-ansible2014_logo_black_tagline1

One of miscellaneous DevOps basics – processes automation. The less you will concentrate on trivial tasks, the more successful your product will be done. And development drudgery turns into one more canvas for creativity? If it’s not perfect, what is?

The most useful tool for infrastructure maintenance is a configuration management system (CMS). It helps you to keep main settings of services in one well-built place with opportunity to use it again. Every CMS has its own special syntax for infrastructure management. In this post I’d like to pay close attention on Ansible, but also there is plenty of options. It could be Chef, Puppet, SaltStack. I’m pretty sure in the future there will be many forks for convenience reason.

Ansible based on Python and contains many Python-based manipulation tools. There is a YAML syntax Jinja2 template engine, Paramiko SSH implementation etc.. The main Ansible benefit – you don’t have any client-side requirements except OS compatibility! For nodes configuration all you have to do – install Ansible in one node and copy SSH keys to remote hosts.

So, let’s skip the worship rite. Ansible documentation has much more details. I’d like to answer on your question “Why should I use it” in practice. Below will be described a simple LAMP installation: Apache, MySQL, PHP.

1. Hosts file.


One of the most important questions of CMS execution – what shouldl be configured. Hosts file is an inventory with data about desired nodes for playbook execution. It’s thought you should already link all nodes by SSH key. By default hosts file located in /etc/ansible/hosts, but you can use it everywhere. Just point a file layout in executive command. Here is a simple example of file.

[webservers]
149.7.102.141 ansible_connection=ssh ansible_user=root 
81.177.142.44 ansible_connection=ssh ansible_port=722 ansible_user=root

You can see group name inside of brackets. It’s comfortable to rotate nodes during the configuration. If you want to check connection – just ping all nodes described in file.

ansible all -i hosts -m ping

It should be done well if you have a correct SSH connection through key.

2. System update role.


In most cases you just need to run update command in node package system. But how is it doing in multitudinous quantity of nodes? Ansible role can get it! Just use 2 simple commands.

- name: update packages for apt-based systems[1]
  apt: update_cache=yes cache_valid_time=3600
  when: ansible_distribution == "Debian" or ansible_distribution == "Ubuntu"

- name: get epel repo for rpm-based systems
  yum: name=epel-release state=latest
 when: ansible_distribution == "CentOS" or ansible_distribution == "Red Hat Enterprise Linux"

- name: update packages
  yum: name=* state=latest
when: ansible_distribution == "CentOS" or ansible_distribution == "Red Hat Enterprise Linux"

Apt and yum is embedded Ansible modules which have responsibility to configure your hosts. And pay attention on when command: you can run some roles with OS system choice. It’s convenient if you have some tasks played correctly only on special kind of Linux distro. Like apt configuration on Ubuntu, or yum on CentOS.

3. PHP role.


Here I’d like to open the best way to organize your roles. Ansible contributors provide the next role instance splitted on directories. Thanks to documentation creators.

roles/
    common/               # this hierarchy represents a "role"
        tasks/            #
            main.yml      #  <-- tasks file can include smaller files if warranted
        handlers/         #
            main.yml      #  <-- handlers file
        templates/        #  <-- files for use with the template resource
            ntp.conf.j2   #  <------- templates end in .j2
        files/            #
            bar.txt       #  <-- files for use with the copy resource
            foo.sh        #  <-- script files for use with the script resource
        vars/             #
            main.yml      #  <-- variables associated with this role
        defaults/         #
            main.yml      #  <-- default lower priority variables for this role
        meta/             #
            main.yml      #  <-- role dependencies

In fact meta part is not really required if you’re not suppose to publish your role in Ansible Galaxy. Galaxy is a big warehouse of roles intended on special kind of configuration. Defaults and vars make almost the same kind of work, so use either one.

Let’s observe this structure on PHP installation. Except templates mechanism that will be explained later.

Tasks module.

The pinnacle of roles and playbooks generally. Task file main.yml is a range of commands you suppose to do for configured machines. For easier modification you can split tasks on groups for each file and include it in the main.yml. Example:

main.yml

 - include: php_install.yml
 - include: php_configure.yml
 - include: php_stuff.yml

php_install.yml

- name: install packages [Debian/Ubuntu]
   apt: name={{ item }} state=present
   with_items:
     - "php5-fpm"
     - "php5-mysql"
     - "php5-gd"
     - "php5-imagick"
     - "php5-pgsql"
   when: ansible_distribution == "Debian" or ansible_distribution == "Ubuntu"
   tags: install

 - name: get rid from php5-mysql old package [CentOS/RHEL]
   yum: name=php-mysql state=absent
   when: ansible_distribution == "CentOS" or ansible_distribution == "Red Hat Enterprise Linux"
   tags: remove

 - name: install packages [CentOS/RHEL]
   yum: name={{ item }} state=present
   with_items:
     - "php"
     - "php-fpm"
     - "php-common"
     - "php-mysqlnd"
     - "php-gd"
     - "php-xml"
     - "php-mbstring"
   when: ansible_distribution == "CentOS" or ansible_distribution == "Red Hat Enterprise Linux"
   tags: install

php_configure.yml

- name: change php configuration [Debian/Ubuntu]
  lineinfile: dest={{ item.dest }} regexp={{ item.regexp }} line={{ item.line }}
  with_items:
    - { dest: '/etc/php5/fpm/php.ini', regexp: '^cgi.fix_pathinfo=', line: 'cgi.fix_pathinfo=0' }
  when: ansible_distribution == "Debian" or ansible_distribution == "Ubuntu"
  notify: restart php
  tags: configure

php_stuff.yml

- name: make a site directory
  file: path={{ site_directory }} owner={{ remote_user }} group={{ remote_group }} mode=0755 state=directory
   tags: configure

 - name: check phpinfo in site directory
   copy: src=info.php dest={{ site_directory }}/index.php owner={{ remote_user }} group={{ remote_group }} mode=0755
   notify: restart php
   tags: test

Note. Inside of double brackets – variables. Variables make your roles flexible for either situation. Default values of variables included in defaults part. Tags helps to include or exclude a special kind of tasks in playbook running.

Handlers module.

It’s a list of subtasks that could be run few times. It’s useful if you need to run some tasks repeatedly. For example, reboot your service or check configuration. Handlers execution described at tasks files.

- name: restart php
  service: name={{ php_daemon }} state=restarted

Files module.

It’s just a list of files you want to push to your node. For PHP instance there could be info.php file declared on one of tasks.

Defaults module.

Here is the list of variables described in tasks/handlers/templates. main.yml includes default values in case if configuration standard is satisfied you.

   ---
   php_daemon: php5-fpm
   site_directory: /var/www/html
   remote_user: root
   remote_group: root

4. Apache role.


Almost the same structure but also there could be templates of configuration files. Template files should have a .j2 extension for Jinja2 processing. Here is a template example.

ServerRoot {{ apache_root }}
Listen 80
Include conf.modules.d/*.conf
User {{ apache_user }}
Group {{ apache_user }}
ServerAdmin {{ apache_admin_mail }}
ServerName {{ domain_name }}:80
DocumentRoot "{{ site_directory }}"
<Directory "{{ site_directory }}">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

5. MySQL role.


By structure MySQL role has the same configuration. If I would insert the full YAML code of playbook – this post would be tremendous and useless. You can take a look at full bunch of tasks at my GitHub repo.

6. Variables file.


Usually variables file is stored in host_vars directory inside of playbook directory. In variables file you can override all default role settings for your goals. Example for CentOS configuration which has a special name of demons and configs.

---
apache_daemon: httpd
domain_name: localhost
site_directory: /var/www/html
local_directory: /root/documents
remote_user: root
remote_group: root
mysql_daemon: mysql
remote_mysql_pass: password
site_directory: /var/www/html
mysql_port: 3306
php_daemon: php-fpm

7. Collect it all in one playbook…


Skeleton of Ansible configuration. Here you pass all information to Ansible required for forth configuration management. There should be name of roles, hosts group, remote user and variables layout. Example of LAMP playbook.

---
 - hosts: webservers
   remote_user: "{{ remote_user }}"
   sudo: yes
   gather_facts: yes
   vars_files:
     - host_vars/all.yml

   roles:
     - iptables
     - php
     - mysql
     - apache

8. …and run it!


Just one command!

ansible-playbook -i hosts lamp.yml

If you want to miss some roles or just include a couple – use special flags.

ansible-playbook -i hosts lamp.yml --tags "install,configure"
ansible-playbook -i hosts lamp.yml --skip-tags "stuff"

Surely Ansible has few shortcomings like every kind of software. At the moment I made this post (2.0 version) they have a little bunch of this:

  • Parallelism absence.

If you need to run one playbook on few nodes you need to wait each one before switching to the next node. Ansible is not a hyper-threading software and cannot provide several SSH connections. I know, it’s not easy to organize this feature. The most likely, it should be fixed by Paramiko external utility.  But it’s a fact and it’s not convenient for big organizations intended on plenty of nodes.

  • Interaction with another DevOps tools.

Sometimes Ansible playbooks execution is not stroke dependent on operating system and software environment. Here is an example. I tried to automate Yii framework structure building at the Vagrant+VirtualBox+Windows bundle. But it’s failed in 1.8.1 Vagrant version. Because Ansible couldn’t move or change any file in remote host. This issue referred with VirtualBox synced folders mechanism. It’s not quite possible to work with files in folder by Ansible. Probably it’s a lack of Vagrant that’s going to be fixed after version upgrade. But fact in the next: Ansible is not ideal to blend with another software.

  • Inability to create several files on node.

When you will run shell script supposed to create a lot of files – you’ll get nothing! Of course, in “shell” module you can mark files expected to create. But you can’t keep an eye on hundreds of files from bash commands. In a nutshell: Ansible doesn’t have a correct build module which could be a simple “make” analog. I asked Ansible developers about this module implementation. Got an answer that it could be realized in the future. But now we’re still looking for a scp alternative of installation from shell. You can take a look at another SCM’s. Hope that Ruby-based Chef or Puppet are doing this kind of job well.

That’s all I want to say about Ansible in a nutshell. Hope you’ve realised the awesome possibility to simple build of your infrastructure.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s