2014. augusztus 10., vasárnap

Ready with Ansible

In my last post I wrote about finishing my work on  Puppet module and moving to Ansible. You can reason out why I'm writing these sentences - yes, I'm done (as far as a role in Ansible can be done).


This time I choosed a different approach to develop something new. In the case of Salt and Puppet I mapped their configuration syntax to syslog-ng's sysntax. I wanted to give as much freedom as possible to the users. In this particular case with Ansible, I choosed a different approach.


Ansible is a really powerful tool in the hand of DevOps, sysadmins.  It's easy to use and can easily set up your infrastructure from scratch. It's learning curve is definitely flat, after Puppet I almost say it's horizontal :) It uses YAML files for describing your system (like Salt), and SSH to make changes. It doesn't need an agent to be preinstalled just a Python interpreter and some basic Python modules so it can also be used to set up network devices, like switches and routers. You can use it with public key authentication, but it can also ask for passwords.


You can do ad-hoc things with Ansible, or organize the steps into so called playbooks. For example, you can update the cache of apt, install a cool software then configure it. When you do this a lot of times, maybe you want simplify the process and just say: This computer must act as this _role_. This is the key here: organize the steps into roles, that’s what I did, too.


I didn’t want to write a “yet another modul”, which can emit syslog-ng configuration from itself (to be precise, for Ansible and Salt no one existed, by now). I wanted to write some high level stuff, a role, which can be assigned to computers. IMHO most users don’t want to configure syslog-ng itself, they just want it to run and see the logs or would like to create a central log server possibly with TLS and forward all their logs to there.


I have good news for them, the role is ready for use in Ansible-galaxy. Just install it, and use:
- hosts: servers
  vars:
    syslog_ng_mode: server
    syslog_ng_server_sources:
      - "127.0.0.1":
          proto: udp
          port: 1524

    syslog_ng_server_dest_dir: /var/logg
    syslog_ng_server_file_macro: $YEAR.$MONTH.$DAY/$HOST.log

  remote_user: root
  roles:
    - {role: ansible-syslog-ng }


By using this role, you get syslog-ng  installed with the default settings on your machines, this is called local mode. In this mode syslog-ng doesn’t receive logs from network, just local sources, like internal() or system(). You can change this behavior by setting syslog_ng_mode to server, client or manual.


In server mode syslog-ng creates network sources (currently TCP, UDP and TCP-TLS) and stores the logs under a specific directory which can be templated. You can use predefined filters as well in each source:


syslog_ng_server_sources:
 - "candrop.example.com":
     proto: udp
     port: 1234
     filters:
       - f_error
       - f_kern
 - "secure.example.com":
     proto: tls
     port: 10514
     ca_dir: /opt/syslog-ng/etc/syslog-ng/ca.d
     key_file: /opt/syslog-ng/etc/syslog-ng/key.d/client.key
     cert_file: /opt/syslog-ng/etc/syslog-ng/cert.d/client_cert.pem
 - "tcp.example.com":
     proto: udp
     port: 1234

The counterside of server mode is the client mode. Syslog-ng will be configured to send its logs to one or more log servers, via UDP, TCP or TCP-TLS and you can also use filters.

The last mode - manual - is for syslog-ng hackers. If you are not satisfied with local, client or server modes, you can get a “vanilla” syslog-ng.conf. More better, you can use Jinja templates in it and have access to all variables.


There are more configuration options, these are just scratching the surface. You can add includes, set where to place logs, force Ansible to check the generated syntax before reloading syslog-ng, and so on. You can find the code and documentation here [1].


I developed and tested this module on an Ubuntu Trusty, but Travis runs all tests on Precise. So the supported platforms currently are Ubuntu or Debian systems. The 3.3, 3.4 and 3.5 syslog-ng versions are supported, (but unfortunately) their configuration files are not the same.


Finally I hope that these four modes covers the most use cases and will be used by a lot of Ansible users.

2014. augusztus 4., hétfő

Puppet is done and I'm moving to Ansible

Last time, when I wrote about my status, I was a little disappointed with Puppet and Ruby. I just couldn't get them to do what I wanted from them. Two new languages, two new challanges. But time was on my side, and step by step I managed to finish my project :-)


The development of my Puppet module reached its target, and it's ready for use. There will be some modifications (simplifying the configuration syntax), but they will be backward compatible.


You can generate syslog-ng sources, destinations, templates, etc. with this module. After I solved a relative require issue, the Travis builds are in nice green colors. The problem was, I couldn't use my own Ruby library placed next to a Puppet custom function below 1.9.2 Ruby version. The require './mylib' syntax didn't work at all, so I had to use a bit uglier, but working solution [1].


I gave up a separate develop branch and changed to a master branch with pull requests. On my machine it was simple to just commit to the develop branch, then rebase master onto it, but it had some drawbacks. First of all, I cannot run spec tests and build module packages on the same machine. This caused a lot of small commits (the spec tests succeded, but IRL the module didn't work) and an ugly git history. Then I created a local git repo (git clone --bare) and rebased my changes into one commit. That sounds good, but GitHub offers a nicer model. The syslog-ng project uses a master branch, and the contributors can make pull requests against that. The cool thing is, with each pull request Travis runs the tests against the merged code and evaluates the results. No more "build failed" on the master branch and the git history is clean, that's enough for me.


I got used to Ruby, I might even say I’ve grown like it. It's a great language and its block thing is awesome. But there are some weird stuff I can't get used to with my Python history.


I also created a Dockerfile, which prepares a development environment which can be used with my module. I hope it will help people get in touch more easily with this project. It's possible to run the spec tests and build a package, too :)


Last, but not least the documentation is also ready to be read and used. You can find the source code here [2]. The next challenge is Ansible, on which I have already started to work.



2014. július 23., szerda

Status report - started to work on Puppet

It's time to give my readers a follow up, how my project is going. In my last post I wrote about the pull requests I sent to Salt. They got merged, but the review process showed up some mistakes. My contribution was moved to salt-contrib, where I could make some changes on them. I've got some extra style fixes  but they broke my unit tests :( Now, my code is ready again to merge into salt's master branch.

In the meantime I began to take care of the Puppet part. Unfortunately, it's harder than I thought it will be. If I want to handle it, I have to learn not just Ruby but Puppet's own DSL and also that weird spec testing. I like challenges, so I began to get over some examples and documentation.

The struggling began

The first problems appeared when I tried to run Puppet from source. The documented installation process didn't work, I had to install some gems manually based on the exceptions I got. There were some problems with rake as well. After a while I was able to generate a boilerplate module, but the generated spec test failed... I also found a bug in puppet lint. I had some misconception about including a class and declaring it with parameters. Maybe it was because the boilerplate spec test gave me strange include errors. I began to push my codes into GitHub [1] and set up Travis to do some auto tests. In my happiness I added more and more versions to test with Travis. Then I realized, there are incompatibilities between Puppet versions... Looking through the documentation I also found some strange things, my favorite is here [2]. The status of this bug report is accepted, because there is no activity on it.

When I saw the light

It turned out, that there is some nasty problem with my development environment. I was curious and tested my code on a Trusty with Puppet installed by apt. My always failing spec unittests got the power and succeeded. In a moment I cloned my Puppet test VM and set up as my new development environment on it. I don't have a fancy Atom editor on it, but Vim is my best friend without a GUI (maybe with a GUI, too). I added some extra entries to the hosts file (and set chattr +i on it, to not be overridden at reboot), installed git and now it's ready to rock :)

Maybe I was a little critical with Puppet. Its learning curve is definitely steeper than Salt's and IMHO the documentation is also less brilliant. But the declarative language seems to be a good idea, just that old school pull mechanism wouldn't be.

Here are some examples I think it will work when I'm finished:

syslog_ng::source {"s_gsoc":
 type => file,
 options => {
   file => /var/log/apache.log,
   follow_freq => 1,
   flags => [no_parse, validate_utf8]
 }
}

syslog_ng::destination {"d_gsoc":
 type => file,
 options => {
   file => /var/log/apache_out.log
 }
}

syslog_ng::log {"some name":
 sources => s_gsoc,
 destinations => d_gsoc
}

Tomorrow I will write a letter to the Puppet developers how I would like to implement my ideas. I hope they will help me :)



2014. július 8., kedd

Pull requests sent

After I finished my work on Salt, I wrote to the salt-users list, and they gave me some good piece of advice. Today I sent two pull requests to Salt.

The first one contains a fix to Salt's ps module [1]. It uses the psutil module, which breaks the compatibility with it's earlier versions. I added some wrapper functions, which are able hide the version differences. My syslog-ng module uses this ps module, and I had problems with it, so I had to fix it :)

The second pull request contains the prime cut of my work, the syslog-ng state and execution module is in this pull request. It can be auto merged, but the Jenkins build failed, but that's not my fault.

Tomorrow I will met with my mentor to talk about the further progress.

2014. június 20., péntek

Work on Salt is in final state

I was out for a while, it's exam time in my university. Next to the studying I worked on my GSoC task but didn't write blog about it. Now, I have done with my exams so I can make up leeway.


I worked on a Salt state module and IMHO it's  development is reaching its final state. I'm done with config generation from YAML, you can find a howto here:



There are a lot of examples I hope you will try them. I don't want to repeat myself, under the above link you can find information how to use the new syslog-ng state modul.
At first it was difficult to me to generate perfect syntax configurations. There are a lot of statement types (log, channel, source, options, etc.), some of them have names, some not. In a statement you can use already defined statements (I call them references) like source(s_local) or you can define them inline. The statements have options and the options have parameters. Some options have a type (like file("input.txt"), its type is file), but in some places you can omit them. For example you can write this:

Now you can see, you can't represent the file type as a dictionary. It should be a list, because its first item is just a string, there is no key for it. You must use different separator characters in different levels of the configuration tree. You can omit the colons in options, but in their parameters you can't. These are not very difficult problems but they are annoying until you get used to them. I built a recursive algorithm which tracks the level in the configuration and uses the appropriate separators, opening and enclosing characters.  You can find the source code here:



Syslog-ng uses words yes and no in some places, so you have to use them in YAML also. PyYAML will load these variables as booleans if you don't enclose them in quotation marks:



If you have any questions or thoughts please leave a comment!



2014. május 20., kedd

Collecting use cases

In my first post I wrote about myself and my attendance in GSoC 2014. I've already taken a look at Salt and wrote a proof of concept code. You can find it here:


Recently I met with my mentor and got my first task: collect use cases.  The configuration management is a huge area and there is not just one software on the market: Salt, Chef, Puppet, Ansible (these are the most popular ones).

You can find below the summed up use cases that we managed to make up:

  1. A lot of computers send their logs to a central log server,
  2. syslog-ng listens on more than one interface and forwards the logs to a central log server,
  3. you want to do some application layer filtering on log messages,
  4. you may want to store application specific configuration files under a conf.d directory,
  5. you might have template configuration files which can be used in more than one place (e.g. data centers),
  6. grids: you got N new computers then you want them to apply the configuration of the "old" machines.
In the case of Salt, the requirements of a general modul are:
  1. It should be some sort of high level stuff,
  2. you may say: "I want a new destination, which is ...",
  3. the users should have the opportunity to define the desired state ("I want a new destination to be created...")
  4. it'd be nice to generate syslog-ng configuration from YAML files,
  5. you might say: "Here is a file, take this as your config"
  6. the infrastructure management is very significant. The syslog-ng configuration must be generatable (by using meta data).
I have some ideas about a Jinja templated solution, but I need more information about Salt to implement them. If you have any suggestions, feel free to share it with me!

2014. május 19., hétfő

About me

My name is Tibi, I am a 23 year old student living in Hungary. I study Computer Science at BUTE, currently working on my MSc degree. 

I live in Budapest but came here from a little village so I brought along the love of nature and animals. In my spare time I read books or listen to music. I lead a little group of volunteer students (~60 people), who aren't content with our education and want more knowledge about IT. We operate the network of our dormitory, which consists of more than 1000 Gigabit endpoints.  Our server park provides services to the whole faculty and moreover we have teams working on IT security, mobile application- and game development.

I like open source software, so I applied to Google Summer of Code 2014. After a few weeks I was informed that the syslog-ng project accepted my proposal, so I'll work on it this summer.

I created this blog to inform people about the progress of my GSoC 2014 work. My task is not so exact as the other ones, but I will try to explain: it's about configuration management. I'd like to make easier the administration of syslog-ng in cloud environments. You can find my proposal here: 


I'm planning to write posts every week about my progress. If you have any questions, feel free to write comments :)