While working for a PHP Developer shop, we used to have a number of different tools for continuous integration:
Using all these different tools made things very confusing, and our sysadmin had to learn all these different technologies to try and keep a CI single system together. Some of the main issues with this configuration were:
- Xinc – could be tricky to configure, it looks like a PHP application but doesn't behave like one, it has poor documentation, and to have Git support you have to migrate to the latest version
- PhpUnderControl – did a really good job for us over the years, but the project looks half dead and project XML configuration can be quite tricky
- Selenium Server – to be able to run tests in a web browser (Linux) we installed full X Server, which is not the best solution from resources point of view: it needs at least 2GB of RAM
To consolidate these systems and their configuration we opted for Jenkins CI which is a continuation of the old Hudson project, available under Creative Commons License, aimed at making CI as easy as possible to implement. The beauty of Jenkins (apart from its single tool structure that contains all CI elements) is that it is a single tool with a pretty, easy GUI that enables easy configuration of a project so that it can be tested.
Another massive benefit is that Jenkins is one of those brilliant tools (like Git) that is so customisable that you can bend it to work in whatever way you need. In this post we shall be looking at how Loft implemented Jenkins into our workflow practices.So, let's get started...
While planning the server, we listed what services we would need from it:
- Git and SVN abilities - for getting the projects out of our repositories
- PHPUnit
- Xdebug
- Selenium Server
- JUnit - for reports
(Not part of the plan but added for fun:)
- Clover reports - code coverage
Once we had the plan the next step was the build...
Jenkins installation
First, we needed a server to set up as our Continuous Integration server (henceforth referred to as CI server). This was to be a Linux Box as it is far easier to work with than a Windows box, as well as more secure and cheaper. A disadvantage is that you can't run tests in Internet Explorer on this box.
Luckily for us, Jenkins has it's own repo which can be installed on almost any distribution of Linux that has it's own package manager (Such as Yum or Apt). For RHEL and similar you can install the repository with:
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum install jenkins
And for Debian/Ubuntu:
wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins
This will do all the hard work of installing Jenkins to the server that we are working on, so all we have to do (as root) is start the service:
/etc/init.d/jenkins start
This will start Jenkins and the Jetty (Java-based webserver) that it runs on. Like us, you may decide that the Jetty is not fit for your purpose and might look for something like NginX or Apache to handle a proxy pass to the Jetty which will give you some extra security to hide your Jenkins Interface behind.
Most of Jenkins is very configurable from the management GUI. From this web page you can download and install new modules to the server which will allow you to use different tools to work out more information about your code and project. These plugins.... do just that: they plug into the project configuration area, which enables the user to add more build stages to the project and extra tests.
Selenium Server requirements
For functional testing we use Selenium Tests with the Selenium RC server to control requests to a web server which acts as a real user going through the site, clicking on links and filling in forms, all the while taking pictures of problems and assessing that all the correct information is displayed on the page. One problem with these tests is that they require a GUI or some form of desktop to use the web browser. In previous set ups of CI, Loft tended to use a Full Blown Desktop (Gnome or KDE) which made our sysadmin's life trickier.
In this case we decided to install a different type of GUI on to the server: one that could still be viewed over an encrypted VNC connection and give access to the web browser that Selenium is running, but would also be light enough that it would not impact on the CI potential of the server.
To remedy this we used a combination of:
- xvfb - as the X server
- x11vnc - acting as the vnc server with the -shared -forever -localhost options set
- Firefox - the web browser
The installation of the above allowed us to configure a display on xvfb which acted as a blank x server. On top of this platform we could call our web browser to run the selenium tests.
Configuring the PHP project
To run tests, you need a few extra plugins, which are easy to install through Jenkins plugin manager:
- Jenkins Mailer Plugin – to be able to send emails
- Git Plugin – to update tests / code
- SeleniumRC plugin – if you want to run Selenium tests
- Clover PHP Plugin – to be able to publish code coverage
- xUnit Plugin – for publishing test results
The project has a few sections which enable you to configure what to run, when to run it, and what to do with results.
1) Update code / Tests
Jenkins has excellent support for VCS – SVN, GIT – you can choose or even combine different repositories to update tested code or tests.
2) Build triggerYou can choose when the build is started. We use the option to Trigger builds remotely for builds triggered by post-commit (receive) hooks from Git.
Here is a sample post-receive hook used in a Git repo:
#!/bin/sh
PROJECT="/var/www/vhosts/project"
cd $PROJECT || exit
unset GIT_DIR
git pull
#start jenkins
wget http://jenkins:8080/job/project/build?token=sdsdjsdjksdjds787jk -O /dev/null
Or for functional tests we use the Build periodically option – for example to run tests every 4 hours.
3) Build
In our case, just a simple command to run all unit tests:
phpunit --log-junit results/phpunit/phpunit.xml -c tests/phpunit.xml
4) Publishing results
Because we export test results to junit format, we can use JUnit (Java Unit test) Publish to publish results:
5) Notify developers
By publishing results, Jenkins knows if the build was successful or not. If not, you can notify developers - or even point a finger at the person who broke the build!
6) Selenium
If you need to start the Selenium server, you can start it for every build or check that server is running:
7) Code coverage
Code coverage is optional, but it can give you very interesting information:
Round Up!
So as you can see - it's an absolute breeze, far easier than in the past. I recommend you give it a try or expand to get in to the practice of CI/CD, it's not just for big businesses these days with most SCMs now offering a CI/CD testing facility dictated by a YAML file in your repository.
With the above you can even see how the beginnings of Jenkins Pipelines began to form with similar requests being handled with the above console, just now in a "Groovy" (Not so groovy!) Jenkinsfile!
Getting back to your roots!
What else is possible?
Extending Permissions
