#3445 closed Task (fixed)

Setup Jenkins to run PHPUnit test suite

Reported by: matt Owned by: vipsoft
Priority: critical Milestone: 1.11 - Piwik 1.11
Component: Core Keywords:
Cc: Sensitive: no

Description

Jenkins was much needed for running smooth operations at Piwik. without jenkins we have to spend a lot of time running tests manually.

It would be fantastic to setup Jenkins as before, that would run the PHPUnit test suite.

Also would be great to generate the PHP docs for: PHP tracker class. They havent been generated in a while and are missing the last new functions.

We do not need more things for now, but this would be VERY useful.

Change History (28)

comment:1 Changed 18 months ago by vipsoft (robocoder)

Jenkins is running on dev5 (on Debian Squeeze).

Planned/ToDo:

  • install vagrant
  • setup Piwik VM as a slave
  • configure jobs

comment:2 Changed 18 months ago by matt (mattab)

  • Owner set to vipsoft

comment:3 Changed 17 months ago by matt (mattab)

how is it going, have you made some progress?
No problem if you can't finish, but please send us asap the list of tasks left do to setup Jenkins on dev5, so we can organize the work :)

it's going to be awesome to have jenkins back!

comment:4 Changed 17 months ago by vipsoft (robocoder)

dev5:

  • a precise32 VM is running as a Jenkins slave; (precise64 is not compatible with this host)
  • ToDo:
    • migrate users and jobs from dev6
    • fix /etc/init.d/vboxweb-service (it isn't automatically starting vboxwebsrv)

Travis-CI:

Last edited 17 months ago by vipsoft (previous) (diff)

comment:5 Changed 17 months ago by SteveG (sgiehl)

I just migrated the jenkins user to the new server and fixed /etc/init.d/vboxweb-service by creating the config in /etc/default/virtualbox. Now the jenkins slave node is running.

comment:6 Changed 17 months ago by SteveG (sgiehl)

@vipsoft: Do you have time to migrate/configure the jobs? I don't know exactly how to do that using the slave.

comment:7 Changed 17 months ago by vipsoft (robocoder)

(In [7459]) refs #3445 - migrate Piwik CI job

comment:8 Changed 17 months ago by vipsoft (robocoder)

I bumped the VM's max_connections = 200 for MySQL because some tests aren't closing connections.

ToDo (these items can be tracked in another ticket):

  • migrate the remaining jobs from dev6 (i.e., docs, git mirroring, nightly snapshots)
  • add a daily build that runs code coverage if we can workaround the zend_mm_heap corrupted crash

comment:9 Changed 17 months ago by SteveG (sgiehl)

(In [7461]) refs #3445 write junit log to make it possible to display test results
within the jenkins job

comment:10 Changed 17 months ago by SteveG (sgiehl)

(In [7462]) refs #3445 moved junit log creation to phpunit.xml, otherwise skipped
and incomplete tests can't be logged

comment:11 Changed 17 months ago by vipsoft (robocoder)

I migrated the git mirroring jobs.

I'm rebuilding the VM. vboxmanage is powerful, and equally dangerous without a backup. =P

comment:12 Changed 17 months ago by matt (mattab)

it looks like builds are working on Jenkins!! wouhou... awesome!

Speaking of backups, is it possible to make an image of the full box to be able to restore it if anything happens? I'm not familiar with such techniques... but because Jenkins is such a key service for us, I'm curious as to how we could have this extra security?

Do you know why emails are not sent to the piwik-builds mailing list from dev5?

comment:13 Changed 17 months ago by matt (mattab)

The console output in dev5.piwik.org:8080/job/Piwik.CI/70/console says [exec] The Xdebug extension is not loaded. No code coverage will be generated.

Once enabled we could get code coverage reports! :)

comment:14 Changed 17 months ago by vipsoft (robocoder)

@SteveG I see you're configuring build-ci-full.xml. When you get around to running phpunit with code coverage, you'll have to load the xdebug extension manually. (It's disabled by default so that the CI-on-commit is faster.) Example:

    <target name="test-mysqli" depends="process-build-resources-mysqli">
        <echo>MYSQLI unit tests started</echo>

        <exec executable="php" dir="${basedir}/PHPUnit" failonerror="true" failifexecutionfails="true">
            <arg line="-d zend_extension=/usr/lib/php5/20090626/xdebug.so
                       /usr/bin/phpunit
                       --log-junit ${basedir}/build/logs/junit.xml
                       --coverage-html ${basedir}/build/coverage
                       --configuration ${source}/app/phpunit.xml.dist"/>
        </exec>

        <echo>MYSQLI unit tests finished</echo>
    </target>

comment:15 Changed 17 months ago by vipsoft (robocoder)

@matt I reconfigured exim on dev5. Outbound email should now work.

comment:16 Changed 17 months ago by matt (mattab)

Great emails now work!

Also test coverage reports work: http://dev5.piwik.org:8080/job/Piwik.Full/cloverphp/ - Kuddos Stefan

I think they still dont run for Integration tests but hopefully you find the problem.

@vipsoft is it OK now to make qa.piwik.org point to dev5?

comment:17 Changed 17 months ago by vipsoft (robocoder)

Sure, you can update the dns as long as we can still reach dev6.

I've installed Apache on dev5 so we can serve out generated docs on port 80. I hope to move those jobs this week.

re: integration tests and code coverage: coverage is only collectable for the phpunit process.

comment:18 Changed 17 months ago by matt (mattab)

I updated the CNAME to dev5

These URLs linked from the piwik QA page are 404:

Thanks for restoring phpdocs, let me know the new URL i'll update QA page and the tracking API doc.

re: integration tests and code coverage: coverage is only collectable for the phpunit process.

we're lucky, because the integration tests call the API internally so the code coverage will be collectible :) -- except for piwik.php tracker requests

comment:19 follow-up: Changed 17 months ago by SteveG (sgiehl)

I tried to activate the integration tests for full build. the build time went up to about 2 hours. Without the integration tests it runs about 15 minutes. But there are still problems with some tests and xdebug.

comment:20 in reply to: ↑ 19 Changed 16 months ago by matt (mattab)

Replying to SteveG:

I tried to activate the integration tests for full build. the build time went up to about 2 hours. Without the integration tests it runs about 15 minutes. But there are still problems with some tests and xdebug.

What did you mean, it seems integration tests are enabled in mysqli and pdo-mysql and takes 36 minutes total.
http://qa.piwik.org:8080/job/Piwik.CI/178/console

Maybe it would work if the "nightly" build had xdebug enabled and would generate code coverage once a day only? or does it fail completely with xdebug enabled?

comment:21 Changed 16 months ago by SteveG (sgiehl)

The tests are already sometimes failing without running the integration tests.
See http://qa.piwik.org:8080/job/Piwik.Full/164/console - Result 139 should be a segmentation fault. Running the tests within the vm on console has sometimes the same result. But sadly it doesn't seem to be the same causing it each time.

Another problem sometimes occures while generating the codecoverage as html: http://qa.piwik.org:8080/job/Piwik.Full/185/console

comment:22 Changed 16 months ago by matt (mattab)

I moved the "Code coverage" feature request to: #3606 since it appears we are faving at least 2 critical core php bugs.

I think this task is done, but pending one last item: http://qa.piwik.org:8080/phpdocs/ is 404 - @vipsoft is it possible to restore the phpdocs?

Then we can close the ticket -- good times!

comment:23 follow-up: Changed 16 months ago by vipsoft (robocoder)

Yeah, I've seen similar (and other) problems at work when generating code coverage. We were stuck on phpunit 3.7.1 for a while because an autoloader refactoring in phpunit was triggering a php bug (segfault).

The php-xdebug package from Canonical is 2.1.0. We can try asking the Ubuntu folks to backport 2.1.4. (Preferably we won't manage our own custom builds.) Alternately, upgrade the VM from 12.04 to 12.10 (which would give us php 5.4.6 and xdebug 2.2.1).

Yes, I'll take a look at phpdocs soon.

comment:24 in reply to: ↑ 23 Changed 16 months ago by matt (mattab)

The php-xdebug package from Canonical is 2.1.0. We can try asking the Ubuntu folks to backport 2.1.4. (Preferably we won't manage our own custom builds.) Alternately, upgrade the VM from 12.04 to 12.10 (which would give us php 5.4.6 and xdebug 2.2.1).

+1 for upgrading the VM

Yes, I'll take a look at phpdocs soon.

Very appreciated

comment:25 Changed 16 months ago by matt (mattab)

I just saw this related ticket #3054

comment:26 Changed 16 months ago by matt (mattab)

(In [7654]) Refs #3445 Adding doc for --testdox

comment:27 Changed 16 months ago by matt (mattab)

(In [7655]) Refs #3445 try again

comment:28 Changed 15 months ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from new to closed

now working! awesome...

Note: See TracTickets for help on using tickets.