At my part time job, my junior has been fleshing out a Jenkins (formerly known as Hudson) installation.

In the long term, what we have in mind is providing a proper testing suite for the developers and testers in the company to use. But to get to grips with the software and what it's capable of doing, he came up with a great idea: transition our routine cron tasks into Jenkins 'builds'.

The back story: to date, our filesystem and other backups of all our servers (not many, totalling between 30 to 40) have been running on crons. All output from these crons are emailed to us for review later. This seemed like a good idea, but I can assure you, reading 40 emails of 'success' every morning gets very old very quickly. And worse: the regular success of these tasks can easily cause a, er, less thorough review of the 40 emails gradually, leading to a sysadmin missing something important due to habitual 'she'll be right' glances.

Moving to Jenkins changes our approach: our backup tasks are already adequately coded to exit on any errors that we'd want to catch. Jenkins can schedule and execute jobs on 'slave' nodes, even simple shell scripts, and is intelligent enough to fail the 'build' should it detect a non-zero exit status.

What does this mean? It means that our nightly crons have been deleted entirely, favouring Jenkins' ability to schedule the tasks itself. Successful builds are still reported, but via the IRC plugin provided by Jenkins, to our IRC channel.

Finally, failed builds are notified in IRC as well, by e-mail to us, and also to the Notifo plugin, meaning our iPhones ping us to a problem, just like any Nagios alert.

The productivity increase of this should not be laughed at: now we can glance at what we *know* are successful builds (and review the full output for those of us like me, who can't help but be anal about reviewing task output regularly..), get pinged only for *real* problems, and have a full reviewable history of the tasks in Jenkins to identify any patterns.

One other thing I had to do a while ago was review the staggered cron sequences across all servers. This involved reviewing every single crontab across every server. With Jenkins in control instead of cron, this ordered list of scheduled tasks is pre-made, as well as sortable by success/failure.

Why am I yammering about Jenkins, on such a drab subject as crons and scheduled tasks? The reason is, we've more or less completed this task, and since then, Green Bee Digital have also moved scheduled tasks to http://jenkins.greenbeedigital.com.au:8080 as well. Today, I realised I could transition other tasks into Jenkins: specifically, backup testing tools (Felicity), and more significantly, scheduled tests of Aegir installations using Libcloud (see frigg).

Indeed, as of this evening, I have a build project that:

* Provisions a new VPS at Rackspace Cloud
* Installs Aegir
* Builds a Drupal 6, Drupal 7 and OpenAtrium platform using Drush Make
* Installs a site (with respective install profiles) on all three platforms

If anything fails here, an alert will come through to me and to #aegir on Freenode. This will help us to test the Aegir install (especially to ensure our compatibility with Drush and Drush Make releases), and to identify any regressions in our own project.

So what? In doing so, I've produced a tool that could allow you to spin up a temporary virtual server, drag in your codebase from Drush Make, and do a test install of a site on it. This could become a routine start-to-finish test of your application, features, modules etc, all at the click of a button (or less, by scheduling it!)

The ultimate aim, of course, is to make the tool so useful that you might forget the obligatory stupid open source project name.

More information (especially the code) to follow in a future blog post :)