Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup Jenkins to run PHPUnit test suite #3445

Closed
mattab opened this issue Oct 16, 2012 · 27 comments
Closed

Setup Jenkins to run PHPUnit test suite #3445

mattab opened this issue Oct 16, 2012 · 27 comments
Assignees
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Oct 16, 2012

Jenkins was much needed for running smooth operations at Piwik. without jenkins we have to spend a lot of time running tests manually.

It would be fantastic to setup Jenkins as before, that would run the PHPUnit test suite.

Also would be great to generate the PHP docs for: PHP tracker class. They havent been generated in a while and are missing the last new functions.

We do not need more things for now, but this would be VERY useful.

@robocoder
Copy link
Contributor

Jenkins is running on dev5 (on Debian Squeeze).

Planned/ToDo:

  • install vagrant
  • setup Piwik VM as a slave
  • configure jobs

@mattab
Copy link
Member Author

mattab commented Nov 12, 2012

how is it going, have you made some progress?
No problem if you can't finish, but please send us asap the list of tasks left do to setup Jenkins on dev5, so we can organize the work :)

it's going to be awesome to have jenkins back!

@robocoder
Copy link
Contributor

dev5:

  • a precise32 VM is running as a Jenkins slave; (precise64 is not compatible with this host)
  • ToDo:
    • migrate users and jobs from dev6
    • fix /etc/init.d/vboxweb-service (it isn't automatically starting vboxwebsrv)

Travis-CI:

@sgiehl
Copy link
Member

sgiehl commented Nov 12, 2012

I just migrated the jenkins user to the new server and fixed /etc/init.d/vboxweb-service by creating the config in /etc/default/virtualbox. Now the jenkins slave node is running.

@sgiehl
Copy link
Member

sgiehl commented Nov 12, 2012

@vipsoft: Do you have time to migrate/configure the jobs? I don't know exactly how to do that using the slave.

@robocoder
Copy link
Contributor

(In [7459]) refs #3445 - migrate Piwik CI job

@robocoder
Copy link
Contributor

I bumped the VM's max_connections = 200 for MySQL because some tests aren't closing connections.

ToDo (these items can be tracked in another ticket):

  • migrate the remaining jobs from dev6 (i.e., docs, git mirroring, nightly snapshots)
  • add a daily build that runs code coverage if we can workaround the zend_mm_heap corrupted crash

@sgiehl
Copy link
Member

sgiehl commented Nov 13, 2012

(In [7461]) refs #3445 write junit log to make it possible to display test results
within the jenkins job

@sgiehl
Copy link
Member

sgiehl commented Nov 13, 2012

(In [7462]) refs #3445 moved junit log creation to phpunit.xml, otherwise skipped
and incomplete tests can't be logged

@robocoder
Copy link
Contributor

I migrated the git mirroring jobs.

I'm rebuilding the VM. vboxmanage is powerful, and equally dangerous without a backup. =P

@mattab
Copy link
Member Author

mattab commented Nov 20, 2012

it looks like builds are working on Jenkins!! wouhou... awesome!

Speaking of backups, is it possible to make an image of the full box to be able to restore it if anything happens? I'm not familiar with such techniques... but because Jenkins is such a key service for us, I'm curious as to how we could have this extra security?

Do you know why emails are not sent to the piwik-builds mailing list from dev5?

@mattab
Copy link
Member Author

mattab commented Nov 21, 2012

The console output in dev5.piwik.org:8080/job/Piwik.CI/70/console says [exec] The Xdebug extension is not loaded. No code coverage will be generated.

Once enabled we could get code coverage reports! :)

@robocoder
Copy link
Contributor

@SteveG I see you're configuring build-ci-full.xml. When you get around to running phpunit with code coverage, you'll have to load the xdebug extension manually. (It's disabled by default so that the CI-on-commit is faster.) Example:

    <target name="test-mysqli" depends="process-build-resources-mysqli">
        <echo>MYSQLI unit tests started</echo>

        <exec executable="php" dir="${basedir}/PHPUnit" failonerror="true" failifexecutionfails="true">
            <arg line="-d zend_extension=/usr/lib/php5/20090626/xdebug.so
                       /usr/bin/phpunit
                       --log-junit ${basedir}/build/logs/junit.xml
                       --coverage-html ${basedir}/build/coverage
                       --configuration ${source}/app/phpunit.xml.dist"/>
        </exec>

        <echo>MYSQLI unit tests finished</echo>
    </target>

@robocoder
Copy link
Contributor

@matt I reconfigured exim on dev5. Outbound email should now work.

@mattab
Copy link
Member Author

mattab commented Nov 26, 2012

Great emails now work!

Also test coverage reports work: http://dev5.piwik.org:8080/job/Piwik.Full/cloverphp/ - Kuddos Stefan

I think they still dont run for Integration tests but hopefully you find the problem.

@vipsoft is it OK now to make qa.piwik.org point to dev5?

@robocoder
Copy link
Contributor

Sure, you can update the dns as long as we can still reach dev6.

I've installed Apache on dev5 so we can serve out generated docs on port 80. I hope to move those jobs this week.

re: integration tests and code coverage: coverage is only collectable for the phpunit process.

@mattab
Copy link
Member Author

mattab commented Nov 30, 2012

I updated the CNAME to dev5

These URLs linked from the piwik QA page are 404:

Thanks for restoring phpdocs, let me know the new URL i'll update QA page and the tracking API doc.

re: integration tests and code coverage: coverage is only collectable for the phpunit process.
we're lucky, because the integration tests call the API internally so the code coverage will be collectible :) -- except for piwik.php tracker requests

@sgiehl
Copy link
Member

sgiehl commented Nov 30, 2012

I tried to activate the integration tests for full build. the build time went up to about 2 hours. Without the integration tests it runs about 15 minutes. But there are still problems with some tests and xdebug.

@mattab
Copy link
Member Author

mattab commented Dec 13, 2012

Replying to SteveG:

I tried to activate the integration tests for full build. the build time went up to about 2 hours. Without the integration tests it runs about 15 minutes. But there are still problems with some tests and xdebug.

What did you mean, it seems integration tests are enabled in mysqli and pdo-mysql and takes 36 minutes total.
http://qa.piwik.org:8080/job/Piwik.CI/178/console

Maybe it would work if the "nightly" build had xdebug enabled and would generate code coverage once a day only? or does it fail completely with xdebug enabled?

@sgiehl
Copy link
Member

sgiehl commented Dec 13, 2012

The tests are already sometimes failing without running the integration tests.
See http://qa.piwik.org:8080/job/Piwik.Full/164/console - Result 139 should be a segmentation fault. Running the tests within the vm on console has sometimes the same result. But sadly it doesn't seem to be the same causing it each time.

Another problem sometimes occures while generating the codecoverage as html: http://qa.piwik.org:8080/job/Piwik.Full/185/console

@mattab
Copy link
Member Author

mattab commented Dec 13, 2012

I moved the "Code coverage" feature request to: #3606 since it appears we are faving at least 2 critical core php bugs.

I think this task is done, but pending one last item: http://qa.piwik.org:8080/phpdocs/ is 404 - @vipsoft is it possible to restore the phpdocs?

Then we can close the ticket -- good times!

@robocoder
Copy link
Contributor

Yeah, I've seen similar (and other) problems at work when generating code coverage. We were stuck on phpunit 3.7.1 for a while because an autoloader refactoring in phpunit was triggering a php bug (segfault).

The php-xdebug package from Canonical is 2.1.0. We can try asking the Ubuntu folks to backport 2.1.4. (Preferably we won't manage our own custom builds.) Alternately, upgrade the VM from 12.04 to 12.10 (which would give us php 5.4.6 and xdebug 2.2.1).

Yes, I'll take a look at phpdocs soon.

@mattab
Copy link
Member Author

mattab commented Dec 13, 2012

The php-xdebug package from Canonical is 2.1.0. We can try asking the Ubuntu folks to backport 2.1.4. (Preferably we won't manage our own custom builds.) Alternately, upgrade the VM from 12.04 to 12.10 (which would give us php 5.4.6 and xdebug 2.2.1).

+1 for upgrading the VM

Yes, I'll take a look at phpdocs soon.

Very appreciated

@mattab
Copy link
Member Author

mattab commented Dec 14, 2012

I just saw this related ticket #3054

@mattab
Copy link
Member Author

mattab commented Dec 19, 2012

(In [7654]) Refs #3445 Adding doc for --testdox

@mattab
Copy link
Member Author

mattab commented Dec 19, 2012

(In [7655]) Refs #3445 try again

@mattab
Copy link
Member Author

mattab commented Jan 31, 2013

now working! awesome...

@mattab mattab added this to the 1.11 - Piwik 1.11 milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Projects
None yet
Development

No branches or pull requests

3 participants