(Added title to blog post: March 8, 2013)
Why autobuild in the first place?
Your software has been building fine, manually, for a while so one might be tempted to think that autobuilds are superflous. Why check if your software builds and why test it on all the platforms you target automatically when you so easily can do it manually?
In Searduino’s case most of the code is written in C (and C++) using POSIX threads so it should be possible to say that if it build son platofrm A it should build equally well on platofrm B, shouldn’t it? The Java and Python extension could potentially be tested on one platform alone. I don’t think so – but I wouldn’t argue to strong for this case.
I would say you always gain from building your software on various platforms. And a side effect could be nightly dist files and even binary releases.
Searduino is a bit tricky – we don’t claim it is tricky in a unique way nor one of the trickier projects. Anyhow … Searduino provides libraries cross compiled for the various Arduino boards so we’re getting some more interesting cases where things can go wrong. Several times we’ve ran into problems with various combinations of the versions of the cross compiler (avr-gcc), libc (avr-libc), Arduino software and Searduino itself. Add to that the operating systems we target (GNU/Linux (Debian and Ubuntu – Fedora developers are more than welcome!!!), Mac and Windows (using cygwin)). We need to make sure it is easy to build the software – on as many platforms as possible and in an automated fashion. The old build server crashed recently, so we’ve been setting up a new. We’ve had Ubuntu (amd64), built natively, for some days and we needed to add Ubuntu (32 bit). We will continue with other OS:s and GNU/Linux distributions soon.
We don’t have a 32 bit computer lying around and even if we did we don’t want it to be powered on all day. So we decided, as most developers do I assume, to run the builds and test on a virtualised environment. Enter VMM.
How to test the Arduino libraries, built for the Arduino boards? Should we test on actual boards – how do we check if the code is working? Is it enough to build and upload with no errors? This something we will deal with another day. But this blog post is more about VMM so we’re leaving that for now.
Autobuilding using VMM
With VMM it’s easy to start, stop and execute commands on virtualised machines (currently VirtualBox and qemu). On the github site there’s a manual guiding you how to set VMM up.
Here’s what we did to get VMM to autobuild Searduino.
- Install VMM
- Install Ubuntu (32bit) in Virtualbox and call the client/guest Searduino-32.
- Added a user, autobuilder, to the client/guest Searduino-32. This user should run the actual builds.
- Add your (local) ssh keys to the autobuilder (.ssh/autohorized_keys) user and to the root account (this is needed to close the host as graceful as possible).
- Create a basic configuration for VMM called .ats/ats.conf
# ATS Settings
# ATSVM Settings
SSH_TEST_OPTIONS=” -o connectTimeout=3″
- Create a client configuration file on the host (not on the guest/client) called .ats/clients/Searduino-32.conf
VM_NAME="Searduino-32" VM_TYPE="VirtualBox" VM_IP_ADDRESS=localhost VM_USER=autobuilder VM_SUPERUSER=root SSH_PORT=auto SSH_SHUTDOWN_COMMAND="shutdown -h now"
… and now we should be ready to write our script to do the actual test.
Stripped down version of Searduino’s autobuild script:
# start the client
ats-client –start-client-headless Searduino-32
# make sure the directory is empty
ats-client –client-exec Searduino-32 “rm -fr searduino”
# clone the Searduino git repo
ats-client –client-exec Searduino-32 “git clone git://git.sv.gnu.org/searduino.git”
# Build Searduino – the command line is nasty. Blame Searduino for this
ats-client –client-exec Searduino-32 “export BUILD_DOC=false ; export CXXFLAGS=\”-I/usr/lib/jvm/java-6-openjdk/include/\” ; export CFLAGS=\”-I/usr/lib/jvm/java-6-openjdk/include/\” ; cd searduino && bin/build-and-test.sh”
# Build a binary dist
ats-client –client-exec Searduino-32 “export BUILD_DOC=false ; export CXXFLAGS=\”-I/usr/lib/jvm/java-6-openjdk/include/\” ; export CFLAGS=\”-I/usr/lib/jvm/java-6-openjdk/include/\” ; cd searduino && bin/build-bin-dist.sh”
# Close down the client
ats-client –stop-client Searduino-32
Schedule the builds the way you want it. We simply put the script in /etc/cron.daily/
…. you may, which Searduino does, have to transfer some files from the client. This will be explained later – and we’re currently developing better support for this in VMM. A more complete example (a step by step guide) is currently being written. Meanwhile, check out this script https://github.com/tis-innovation-park/vmm/blob/master/examples/searduino.tmpl.
Results from the execution of the Searduino script:
Builds logs: http://126.96.36.199/searduino-build/build-logs/20130301/i686/
Binary dist: http://188.8.131.52/searduino-build/bin-dist/
The configuration needed for using VMM is limeted to some 15-20 lines. Writing a script like the above is only some 10 lines worth of work. So using VMM to do autobuilds is easy.
We’re currently working hard on getting VMM to be useful to as many people as possible.
- It would be great if you use it
- It would be great if you want to join us.
/Henrik and Shaun
BTW, ats will be renamed to vmm shortly