Jenkins Continuous Integration, Zend Framework and Netbeans

PHP sure has matured into a serious programming language from its humble beginnings. As more and more complex software is being built using PHP, it makes sense to use sound software engineering principles to insure robustness of built applications.

A good CI tool like Jenkins will help you get an inside look at your code in the form of detailed metrics. In the tutorial that follows, I describe the full install process of Jenkins on a windows workstation running wampserver. I will also detail the process of exercising code written using the Zend Framework. And, since I use Netbeans for all my development, I will describe a couple of nifty Netbeans plugins that make writing standardized code a lot easier.

Part 1: Preparing for Install

Jenkins server has the role of a conductor that orchestrates myriad php tools which in turn do the real heavy-lifting. Our first order of business is to install a working PEAR setup. If you don’t have it installed, please read my earlier blog about how you can set it up relatively easily on WAMP.

In case you already have pear installed, please make sure that all your packages are up-to-date (“pear list-upgrades” will display outdated packages, and “pear upgrade” will perform the upgrade). You should see a screen that looks like:

image

The following are the PHP tools that will be utilized by Jenkins. Please go ahead and install them as described:

Name Purpose How to Install
PHPUnit For Unit testing PHP code pear config-set auto_discover 1
pear install pear.phpunit.de/PHPUnit
phing Build tool written in PHP (similar to ANT) pear install pear.phing.info/phing
PHP_CodeSniffer Ensure coding standards pear install –a PHP_CodeSniffer
PHPDocumentor Automatically generating documentation/API from docblock comments in your code pear install PHPDocumentor
PHPcpd Copy/Paste Detector (CPD) for PHP code. pear install phpunit/phpCPD
PHPmd PHP mess detector. Analyze code to identify poorly written sections pear install -a pear.phpmd.org/PHP_PMD
PHPDepend Generates various software metrics pear install pear.pdepend.org/PHP_Depend
PHPCodeBrowser Generates browsable code annotated with defects identified by codesniffer, cpd and md pear install -a phpunit/PHP_CodeBrowser
PHPLoc Generates statistics for Lines of Code and other software metrics pear install -a phpunit/phploc

Once you have installed all the above packages, they are instantly made available on the windows path as the installers typically create a .bat file on the same directory as the php executable (which is already set in the windows path by the pear install process). You should therefore be able to open up a command window and type the name of any of the tools above.

Part 2: Install Jenkins

Next, head over to the Jenkins website and grab the native windows installer. Run the setup process. By default, Jenkins is setup at http://localhost:8080. As you can see, installation of Jenkins is really effortless. This is one of the key features that make it hugely popular.

(This part is optional only if you want to change the default port)

On my workstation, I run IIS on port 80 and Wampserver on 8080. I prefer to run Jenkins on port 8888. Making the change is easy. Head over to the Jenkins directory (c:\Program Files\Jenkins by default) and edit the file named “Jenkins.xml”. Change the port to 8888 in the <arguments> section as shown below:

<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar
"%BASE%\jenkins.war" --httpPort=8888</arguments>

The install process sets up a windows service named  “Jenkins” that is set to autostart. Like any other windows process, you can start, stop or restart it (Right click on “My Computer”-> select “Manage”->”Services and Applications”->”Services” and locate “Jenkins” on the list)

image

If all went well, when you restart the service and point your browser to http://localhost:8888 (or http://localhost:8080 if you did not change the default port) you should now see the Jenkins home page.

You can now proceed to install the required plugins. This process is also trivially easy because the Jenkins web interface does an excellent job of managing it. Click on the “Manage Jenkins” link, and then click on the “Manage Plugins” option. Click on the “Available’ Tab. Check all the plugins that need to be installed (list below) and hit the “Install without restart” option. Jenkins will retrieve the plugin from the web, set it up and prompt you to restart!

List of recommended plugins:

Jenkins Plugin Purpose
Checkstyle Generates a summary (and detail) of code style errors returned by the PHP_CodeSniffer tool
Clover_PHP Picks up Coverage_HTML and Coverage_Clover output from phpunit tool
DRY Summarizes output about repeated code generated by the PHPCPD tool
HTML Publisher (Post build) Used to create a snapshot of the documentation (generated by the PHPDocumentor tool) and the code browser (generated by PHPCodeBrowser). The plugin creates and maintains a copy of these two reports for EVERY BUILD. So you can always look at a prior build and analyze the progress.
JDepend Report on output generated by the PDepend tool
Plot Generate graphs from CSV output generated by the PHPLoc tool
PMD Report on output generated by the phpmd tool (PHP Mess Detector)
Violations Generates reports and graphs from output generated by PHP_CodeSniffer, PHPCpd and PHPMD
Mercurial Helps Jenkins acquire source code from Mercurial Source Control System (Ex. Bitbucket)
Phing Integrates Phing build tool into Jenkins

Sebastian Bergmann (creator of PHPUnit and various other tools above) actually has a website dedicated to explaining the nuances of integrating Jenkins with PHP (http://jenkins-php.org/). You might want to peruse through his site if you haven’t already. In part 3 of this tutorial, I borrow heavily from his website.

Part 3: Setup Jenkins for your PHP/Zend Framework Project

There are three distinct phases in any Jenkins job (and these can be batched to run at specified intervals):

  1. Source code retrieval– You will typically want Jenkins to automatically retrieve source code from a repository (I use Mercurial on Bitbucket.. but there are plugins available for almost any source control solution you may be comfortable with).
  2. Build– This is the part where the source code retrieved in step 1 is rigorously analyzed. We will use a phing build file to specify all the actions we want performed on the source code. This step typically generates a bunch of reports in predefined folders.
  3. Post Build – This step takes reports and logs generated by step 2 above and presents pretty reports to the end-user.

Because of the the sheer volume of available tools, processes and options, setting up a Jenkins project that presents meaningful and complete data to the user can be quite challenging. To simplify the adoption of Jenkins for PHP users, Sebastian Bergmann has created a template that users can base their projects off of. This template prefills all the common build and post-build options so the user does not have to bother about them. Keep in mind that setting up a Jenkins job for a project is a one time deal.

  1. Download the template from https://github.com/sebastianbergmann/php-jenkins-template/downloads. The template essentially contains an elaborate XML file that details the required build options.
  2. Extract the contents of the zip file into a folder named ‘php-template-jenkins’ in the c:\Program Files\Jenkins\Jobs folder
    image
  3. Fire up the Jenkins home page (http://localhost”:8888) and click on the “Manage Jenkins” menu option. Click on the “Reload Configuration from Disk” link. This will force Jenkins to recognize the newly copied php template
  4. On the Jenkins home page, click on the “New Job” menu option
  5. Type in your application name in the “Job Name” field. Select the “Copy existing job” option at the end and type ‘php-jenkins-template’ (It has a neat auto-complete box that prompts you with available matches!)
    image
  6. Now, on the homepage, you should see your newly setup job. This job has inherited all settings form the php-template.
    image

We will be using “phing” to bring all our tools together and execute them sequentially. By default, phing looks for a file named “Build.xml” in the project root and executes it. The build file that I use is listed below:

<?xml version="1.0" encoding="UTF-8"?>

<project name="name-of-project" default="build">
    <property name="basedir" value="." />
    <target name="build"
            depends="prepare,lint,phploc,pdepend,phpmd,phpcs,phpcpd,phpdoc,phpunit,phpcb"/>

    <target name="clean" description="Cleanup build artifacts">
        <delete dir="${basedir}/build/api"/>
        <delete dir="${basedir}/build/code-browser"/>
        <delete dir="${basedir}/build/coverage"/>
        <delete dir="${basedir}/build/logs"/>
        <delete dir="${basedir}/build/pdepend"/>
    </target>

    <target name="prepare" depends="clean"
            description="Prepare for build">
        <echo msg="${basedir}" />
        <mkdir dir="${basedir}/build/api"/>
        <mkdir dir="${basedir}/build/code-browser"/>
        <mkdir dir="${basedir}/build/coverage"/>
        <mkdir dir="${basedir}/build/logs"/>
        <mkdir dir="${basedir}/build/pdepend"/>
    </target>

    <target name="lint">
        <phplint>
            <fileset dir="${basedir}">
                <include name="**/*.php" />
                <exclude name="**/tests/**" />
            </fileset>
        </phplint>
    </target>

    <target name="phploc" description="Measure project size using PHPLOC">
        <exec executable="phploc">
            <arg value="--log-csv" />
            <arg value="${basedir}/build/logs/phploc.csv" />
            <arg value="--exclude"/>
            <arg path="${basedir}/tests" />
            <arg value="--suffixes"/>
            <arg value="php" />
            <arg path="${basedir}" />
        </exec>
    </target>

    <target name="pdepend">
        <phpdepend>
            <fileset dir="${basedir}">
                <include name="**/*.php" />
                <exclude name="**/tests/**" />
            </fileset>
            <logger type="jdepend-xml" outfile="${basedir}/build/logs/jdepend.xml"/>
            <logger type="jdepend-chart" outfile="${basedir}/build/pdepend/dependencies.svg"/>
            <logger type="overview-pyramid" outfile="${basedir}/build/pdepend/overview-pyramid.svg"/>
        </phpdepend>
    </target>

    <target name="phpmd"
            description="Generate pmd.xml using PHPMD">
        <phpmd rulesets="codesize,unusedcode,naming,design">
            <fileset dir="${basedir}">
                <include name="**/*.php" />
                <exclude name="**/tests/**" />
            </fileset>
            <formatter type="xml" outfile="${basedir}/build/logs/pmd.xml"/>
        </phpmd>
    </target>

    <target name="phpcs">
        <phpcodesniffer standard="Zend" allowedFileExtensions="php">
            <fileset dir="${basedir}">
                <include name="**/*.php" />
                <exclude name="**/tests/**" />
            </fileset>
            <formatter type="default" usefile="false"  />
            <formatter type="checkstyle" outfile="${basedir}/build/logs/checkstyle.xml" />
        </phpcodesniffer>
    </target>

    <target name="phpcpd">
        <phpcpd>
            <fileset dir="${basedir}">
                <include name="**/*.php" />
            </fileset>
            <formatter type="pmd" outfile="${basedir}/build/logs/pmd-cpd.xml"/>
        </phpcpd>
    </target>

    <target name="phpdoc" description="AutoGenerate php docs">
        <phpdoc title="My API Docs"
                sourcecode="yes"
                destdir="${basedir}/build/api"
                output="HTML:Smarty:PHP">
            <fileset dir="${basedir}">
                <include name="**/*.php" />
                <exclude name="**/tests/**" />
            </fileset>
        </phpdoc>
    </target>

     <target name="phpunit" description="Run unit tests with PHPUnit">
         <exec command="phpunit --configuration=${basedir}/tests/phpunit.xml
        --log-junit ${basedir}/build/logs/junit.xml
        --coverage-clover ${basedir}/build/logs/clover.xml
        --coverage-html ${basedir}/build/coverage"/>
    </target>

    <target name="phpcb"
            description="Aggregate tool output with PHP_CodeBrowser">
        <exec executable="phpcb">
            <arg value="--log" />
            <arg path="${basedir}/build/logs" />
            <arg value="--source" />
            <arg path="${basedir}/application" />
            <arg value="--source" />
            <arg path="${basedir}/library" />
            <arg value="--output" />
            <arg path="${basedir}/build/code-browser" />
        </exec>
    </target>
</project>

This file must be placed in the root of your project. So, if you are coding using Zend Framework, it should be at the same level as “application”, “library” and “public” folders.

The code is fairly self-documenting. A property named “basedir” is declared. This is used throughout the script as ${basedir}. The project has a default build target of “build”. “build” in turn depends on multiple targets that are setup, and all are executed in sequence. “prepare” is the first target executed and that is in charge of creating the required folders.

The “tests” folder is excluded from most metric computations as that is where our unit test cases reside. Note the “–configuration” option in the “phpunit” target – this indicates the location of the phpunit.xml file which in turn contains the location of the bootstrap file, and the folders containing code to test.. It is pretty much the stock phpunit.xml file that gets generated with a new Zend Framework project with the <whitelist/> section added. The “<whitelist/>” section is required – without this, you will have inflated code-coverage reports that document the entire zend framework!

<phpunit bootstrap="./bootstrap.php">
    <testsuite name="Application Test Suite">
        <directory>./application</directory>
    </testsuite>
    <testsuite name="Library Test Suite">
        <directory>./library</directory>
    </testsuite>
  <filter>
        <whitelist>
            <directory suffix=".php">../application</directory>
            <directory suffix=".php">../library</directory>
            <exclude>
                <directory suffix=".phtml">../application</directory>
            </exclude>
        </whitelist>
    </filter>
</phpunit>

I prefer to have actual code for “code-cover generation” inside the phing job and NOT in the phpunit.xml file. The main reasons for this are:

  1. Generating a code coverage report takes time. And this grows along with the length of the code. While a delay in a jenkins job is perfectly acceptable, any delay to the TDD process is not. phpunit.xml is a file that you will run very often from your IDE (if you practice TDD) and the quicker it runs, the better.
  2. Netbeans has an awesome internal code coverage generator that generates wonderful code-coverage reports within the IDE (if you haven’t checked this feature out, I would strongly recommend that you do)
  3. Artifacts generated during code coverage analysis (various html files) do not really belong in the project. The “build/coverage” folder generated by the phing task lies outside the project and is therefore preferred.

Next, we modify our template so that it works with our source code repository and also use phing instead of ANT.

Phing intead of ANT: I prefer not to download and install another binary executable (ant) when we have a perfectly capable native php tool (phing) available for the job.

In Jenkins, click on the newly created job and then on the “Configure” option on your left.

a. Change the tags in the description to use <img/> instead of (I find this works better with the most recent release of Jenkins – 1.454)

b. Provide the source of your code in the “Source Code Management” section (Note how the password is passed in the url)

image

c. Under the “build” section, delete the existing ‘Ant’ target by clicking on the “Delete” button. Click on the “Add build step” button and select “Invoke Phing targets” (this option is made available by the phing plugin for Jenkins)

image

It will add a blank line. Nothing needs to be entered there.. By default, Jenkins will look in the windows path for phing and execute it. Phing looks in the workspace directory for a build.xml file, finds it, and dutifully executes all the targets! Hit the save button. This completes Jenkins setup for our project. Choose the “Build Now” option from the menu and take a good look at all the wonderful reports generated by Jenkins.

image

Part 4: Netbeans and PHP – Still The Perfect Match

Your development process has no doubt become much more organized by the use of Jankins CI server. However, you could be bogged down by error reports if you do not have a solid development methodology to boot. And, this starts with the IDE.

A couple of years ago, I wrote about how Netbeans surpassed all my expectations from an IDE, in spite of it being free! To this day, it remains a  clear winner for php development.

With the addition of a few plugins, you will be able to develop better code and thus have fewer surprises when you submit your code for integration. These have served me very well and I strongly recommend you utilize each one of them.

1. Unix line endings : Windows line endings (CR-LF) causes problems on unix machines (which uses LF). To avoid nasty surprises later on, it is best to install this plugin and set line endings to LF.

2. Path Tools : This plugin makes it incredibly easy to drop into console or explorer. Highlight the file or folder and click on the appropriate toolbar option!

3. phpMD / PHP CodeSniffer Plugin: This plugin uses the phpmd (mess detector) and phpcs (code sniffer) tools and tracks code violations early on and displays them right within the IDE window! Using this plugin is an easy way to standardize your codebase. Please refer to the following website for detailed instructions on how to set up and activate this plugin:  http://www.summasolutions.net/blogposts/applying-zend-coding-standard-netbeans-phpmd-codesniffer

4 thoughts on “Jenkins Continuous Integration, Zend Framework and Netbeans

  1. This is great! I am a CS student and trying to learn how to setup our pipeline for our senior project and this was EXACTLY what I was looking for and needed. Greatly appreciated.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s