Tests¶

We strongly encourage developers to apply TDD. Not only as a test tool but as a design tool.

Run tests¶

Tuleap comes with a handy test environment, based on SimpleTest and PHPUnit. Core tests (for things in src directory) can be found in tests/simpletest directory with same subdirectory organization (eg. src/common/frs/FRSPackage.class.php tests are in tests/simpletest/common/frs/FRSPackageTest.php). Plugins tests are in each plugin tests directory:

Old plugins have their tests written with SimpleTest (eg. plugins/tracker/include/Tracker.class.php tests are in plugins/tracker/tests/TrackerTest.php).
Recent plugins have their tests written with PHPUnit (eg. plugins/timetracking/include/Time/DateFormatter.php tests are in plugins/timetracking/phpunit/Time/DateFormatterTest.php).

To run tests you can either use multiple CLI commands (at the root of Tuleap sources):

make simpletest-73
make phpunit-docker-73

Run tests with docker¶

We have docker images to run unit tests on all environments:

CentOS 6 + PHP 7.3 with simpletest: enalean/tuleap-simpletest:c6-php73
CentOS 6 + PHP 7.2 with simpletest: enalean/tuleap-simpletest:c6-php72

Executing tests is as simple as, from root of Tuleap sources:

$> docker run --rm=true -v $PWD:/tuleap:ro enalean/tuleap-simpletest:c6-php73 \
    /tuleap/tests/simpletest /tuleap/tests/integration /tuleap/plugins

If there is only one file or directory you are interested in:

$> docker run --rm=true -v $PWD:/tuleap:ro enalean/tuleap-simpletest:c6-php73 --nodb \
    /tuleap/tests/simpletest/common/project/ProjectManagerTest.php

Note

Please note the --nodb switch, it allows a faster start when there is no DB involved.

REST tests¶

There is also a docker image for REST tests, just run the following command:

$> make tests_rest_73

It will execute all REST tests in a docker container. This container is stopped and removed once the tests are finished. If you need to run tests manually, do the following instead:

$> make tests_rest_setup_73
$root@d4601e92ca3f> ./tests/rest/bin/test_suite.sh <optional_path_to_tests_you_want_to_run>

In case of failure, you may need to attach to this running container in order to parse logs for example:

$> docker exec -ti <name-of-the-container> bash
$root@d4601e92ca3f> tail -f /var/opt/remi/php73/log/php-fpm/error.log

Note

If you’re using an old version of docker, you might encounter error unknown flag: –mount

You can run your test container with:

docker run -ti --rm -v "$(pwd)":/usr/share/tuleap --tmpfs /tmp -w /usr/share/tuleap enalean/tuleap-test-rest:c6-php73-mysql57 bash

Cypress tests¶

All end-to-end tests are written with Cypress.

If you want to run all cypress tests locally just launch:

$> make tests_cypress

You will be able to see the results of the test execution in tuleap/test_results_e2e_full.

If you want to add new tests, you should use the cypress dev image:

$> make tests_cypress_dev

It will launch a local container with a bunch of projects (defined in tests/e2e/_fixtures). Once the container has started, you must be able to launch the Cypress electron app.

$> cd tests/e2e/full/
$> npx cypress open

The electron app will launch tests on https://tuleap/. You have to add a new entry in /etc/hosts file, the IP should correspond to the IP of your container tuleap_runtests_backend-web-e2e.

$> sudo vi /etc/hosts
$> 172.19.0.3   tuleap

Note

The electron app will be able to run only when container is fully monted. If https://tuleap/ is unreachable make sure that container initialisation has finished. If it does not solve your issue, verify the IP in your /etc/hosts

Organize your tests¶

All the tests related to one class (therefore to one file) should be kept in one test file (src/common/foo/Bar.class.php tests should be in tests/simpletest/common/foo/BarTest.php). However, we strongly encourage you to split test cases in several classes to leverage on setUp.

declare(strict_types=1);

class Bar_IsAvailableTest extends TuleapTestCase
{
    //... Will test Bar->isAvailable() public method
}

class Bar_ComputeDistanceTest extends TuleapTestCase
{
    //... Will test Bar->computeDistance() public method
}

Of course, it’s by no mean mandatory and always up to the developer to judge if it’s relevant or not to split tests in several classes. A good indicator would be that you can factorize most of tests set up in the setUp() method. But if the setUp() contains things that are only used by some tests, it’s probably a sign that those tests (and corresponding methods) should be in a dedicated class.

Write a test¶

What makes a good test:

It’s simple
It has an explicit name that fully describes what is tested
It tests only ONE thing at a time

Differences with simpletest:

tests methods can start with itXxx keyword instead of testXxx. Example:

public function itThrowsAnExceptionWhenCalledWithNull()

On top of simpletest we added a bit of syntactic sugar to help writing readable tests. Most of those helpers are meant to help dealing with mock objects.

<?php

declare(strict_types=1);

class Bar_IsAvailableTest extends TuleapTestCase
{

    public function itThrowsAnExceptionWhenCalledWithNull() : void
    {
        $this->expectException();
        $bar = new Bar();
        $bar->isAvailable(null);
    }

    public function itIsAvailableIfItHasMoreThan3Elements() : void
    {
        $foo = mock(Foo::class);
        stub($foo)->count()->returns(4);
        // Syntaxic sugar for :
        // $foo = new MockFoo();
        // $foo->setReturnValue('count', 4);

        $bar = new Bar();
        $this->assertTrue($bar->isAvailable($foo));
    }

    public function itIsNotAvailableIfItHasLessThan3Elements() : void
    {
        $foo = stub(Foo::class)->count()->returns(2);

        $bar = new Bar();
        $this->assertFalse($bar->isAvailable($foo));
    }
}

Available syntaxic sugars:

$foo = mock(Foo::class);
stub($foo)->bar($arg1, $arg2)->returns(123);
stub($foo)->bar($arg1, $arg2)->once();
stub($foo)->bar()->never();
stub($foo)->bar(arg1, arg2)->at(2);
stub($foo)->bar()->count(4);

See details and more helpers in tests/lib/MockBuilder.php.

Helpers and database¶

Hint

A bit of vocabulary

Interactions between Tuleap and the database should be done via DataAccessObject (aka. dao) objects (see src/common/dao/include/DataAccessObject.class.php) A dao that returns rows from database wrap the result in a DataAccessResult (aka. dar) object (see src/common/dao/include/DataAccessResult.class.php)

Tuleap test helpers ease interaction with database objects. If you need to interact with a query result you can use mock’s returnsDar(), returnsEmptyDar() and returnsDarWithErrors().

public function itDemonstrateHowToUseReturnsDar() : void
{

    $project_id = 15;
    $project    = stub(Project::class)->getId()->returns($project_id);

    $dao        = stub(FooBarDao::class)->searchByProjectId($project_id)->returnsDar(
        array(
            'id'  => 1
            'name' => 'foo'
        ),
        array(
            'id'  => 2
            'name' => 'klong'
        ),
    );

    $some_factory = new Some_Factory($dao);
    $some_stuff   = $some_factory->getByProject($project);
    $this->assertEqual($some_stuff[0]->getId(), 1);
    $this->assertEqual($some_stuff[1]->getId(), 2);
}

Builders¶

Keep tests clean, small and readable is a key for maintainability (and avoid writing crappy tests). A convenient way to simplify tests is to use Builder Pattern to wrap build of complex objects.

Note: this is not an alternative to partial mocks and should be used only on “Data” objects (logic less, transport objects). It’s not a good idea to create a builder for a factory or a manager.

At time of writing, there are 2 builders in Core aUser.php and aRequest.php:

public function itDemonstrateHowToUseUserAndRequest() : void
{

    $current_user = aUser()->withId(12)->withUserName('John Doe')->build();
    $new_user     = aUser()->withId(655957)->withUserName('Usain Bolt')->build();

    $request = aRequest()
        ->withUser($current_user)
        ->withParam('func', 'add_user')
        ->withParam('user_id', 655957)
        ->build();

    $some_manager = new Some_Manager($request);
    $some_manager->createAllNewUsers();
}

There are plenty of builders in plugins/tracker/tests/builders and you are strongly encouraged to add new one when relevant.

Integration tests for REST API of plugins¶

If your new plugin provides some new REST routes, you should implement new integration tests. These tests must be put in the tests/rest/ directory of your plugin.

If you want more details about integration tests for REST, go have a look at tuleap/tests/rest/README.md.