Running PHPUnit tests

PHPUnit runs all the tests in Drupal 8 and above. Earlier versions of Drupal used the Simpletest module for testing.

PHPUnit in DDEV

Contrib modules: DDEV Drupal Contrib

DDEV Drupal Contrib is a DDEV add-on which makes development on contrib modules easier. That includes but is not limited to running PHPUnit, PHPCS, PHPCBF, and keeping the module repo as the root workspace while having Drupal and its dependencies installed.

Drupal core: DDEV Selenium Standalone Chrome

DDEV Selenium Standalone Chrome add-on has made it super easy to run Drupal core PHPUnit and Nightwatch tests in DDEV. It has noVNC support, so you can even follow along in the browser.

PHPUnit in Lando

There are a couple options for configuring PHPUnit and running tests in Lando. See Running PHPUnit tests in Lando.

Setting up to run PHPUnit tests manually

Ensure testing dependencies are installed

A. Composer-based method (recommended)

If your site is installed using the drupal/recommended-project project template, then development dependencies can be installed by requiring the drupal/core-dev metapackage as a --dev dependency of your project as described here. (Remember that development dependencies should never be installed in a production or a publicly accessible server for security reasons. See the announcement about our dev dependency packaging changes for more information.)

$ composer require drupal/core-dev --dev --update-with-all-dependencies

B. Git-based

If you installed using Git then install Composer and run composer install.

C. Tarball-based (not recommended)

If you installed from a tarball (eg. .tar.gz or .zip file) downloaded from Drupal.org you still have some options:

1. Install Composer and run composer install (or try composer update on an existing D8 or D9 root).
2. Use a development snapshot (for example, 9.0.x-dev) instead of a tagged release for your development site.
3. Install the development dependencies you need manually into Drupal's vendor directory or elsewhere.

JavaScript tests have additional dependencies - see Running PHPUnit JavaScript tests.

Configure PHPUnit

PHPUnit stores configuration in the phpunit.xml file. Drupal core provides an example at core/phpunit.xml.dist. Make a copy of this file with a name phpunit.xml and place it to one of the following locations depending on your testing workflow:

• The PHPUnit executable expects to find the phpunit.xml file in the current directory. This can be overridden with the -c option.
• If you are using Composer to manage Drupal core, then updating core will overwrite the core/ folder and delete your phpunit.xml file.

PHPUnit v9 (for use with Drupal 10 and below)

In phpunit.xml make the following changes:

• Set the SIMPLETEST_BASE_URL variable to the URL of your site.
• Set the SIMPLETEST_DB variable to point to the URL of your Drupal database.
• If you are placing phpunit.xml somewhere other than core, change the value of the phpunit tag's 'bootstrap' attribute to reflect the new location. Change also the <file> value for each testsuite.
• For kernel and functional tests, set the BROWSERTEST_OUTPUT_DIRECTORY.

The result should look something like this:

<phpunit bootstrap="tests/bootstrap.php" colors="true"
         beStrictAboutTestsThatDoNotTestAnything="true"
         beStrictAboutOutputDuringTests="true"
         beStrictAboutChangesToGlobalState="true">

<php>
  <!-- Set error reporting to E_ALL. -->
  <ini name="error_reporting" value="32767"/>
  <!-- Do not limit the amount of memory tests take to run. -->
  <ini name="memory_limit" value="-1"/>
  <!-- Example SIMPLETEST_BASE_URL value: http://localhost -->
  <env name="SIMPLETEST_BASE_URL" value="http://drupal8.localhost"/>
  <!-- Example SIMPLETEST_DB value: mysql://username:password@localhost/databasename#table_prefix -->
  <env name="SIMPLETEST_DB" value="mysql://drupal8:drupal8@localhost/drupal8"/>
  <!-- Example BROWSERTEST_OUTPUT_DIRECTORY value: /path/to/webroot/sites/simpletest/browser_output -->
  <env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/sites/simpletest/browser_output"/>
  <!-- Display a list of files generated as HTML output. -->
  <env name="BROWSERTEST_OUTPUT_VERBOSE" value="true"/>
</php> 
</phpunit>

Once this is all done, you can run any test you want without doing all this configuration again.

Example values for DDEV

  <php>
    <!-- Set error reporting to E_ALL. -->
    <ini name="error_reporting" value="32767"/>
    <!-- Do not limit the amount of memory tests take to run. -->
    <ini name="memory_limit" value="-1"/>
    <env name="SIMPLETEST_BASE_URL" value="http://localhost"/>
    <env name="SIMPLETEST_DB" value="mysql://db:db@db/db"/>
    <env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/html/web/sites/simpletest/browser_output"/>

PHPUnit v10 (for use with Drupal 11+)

The phpunit.xml file should include the following (replace PATH-TO, LOCAL-SITE-URL, and USERNAME/PASSWORD/HOST/DBNAME as appropriate:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  bootstrap="PATH-TO/core/tests/bootstrap.php"
  colors="true"
  beStrictAboutTestsThatDoNotTestAnything="true"
  beStrictAboutOutputDuringTests="true"
  beStrictAboutChangesToGlobalState="true"
  cacheResult="false"
  xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.5/phpunit.xsd"
  cacheDirectory=".phpunit.cache">
  <php>
    <ini name="error_reporting" value="32767" />
    <ini name="memory_limit" value="-1" />
    <env name="SIMPLETEST_BASE_URL" value="http://LOCAL-SITE-URL" force="true" />
    <env name="SIMPLETEST_DB" value="mysql://USERNAME:PASSWORD@HOST/DBNAME" />
  </php>
  <extensions>
    <bootstrap class="Drupal\TestTools\Extension\HtmlLogging\HtmlOutputLogger">
      <parameter name="outputDirectory" value="PATH-TO/sites/simpletest/browser_output" />
      <parameter name="verbose" value="true" />
    </bootstrap>
  </extensions>

Create a directory for HTML output

Functional tests can output the HTML pages that the test running code sees. These are output as plain HTML files. To provide a location for these to be written to, create a directory called sites/simpletest and make sure that it is writable by the webserver. It's okay to make this directory "world-writable", i.e.:

mkdir -p sites/simpletest/browser_output
chmod -R 777 sites/simpletest

Locate the PHPUnit binary

The relative location of the PHPUnit binary depends on how you install Drupal.

• If the site is installed from a tarball (eg. .tar.gz or .zip file) downloaded from Drupal.org, then run composer update, the vendor directory is within the Drupal root (adjacent to 'core'). The instructions on this page assume this is your setup.
• If you installed using composer require drupal/drupal, you end up with a vendor directory outside the Drupal root (above 'core'). You may need to adjust the path to PHPUnit in the commands given on this page: vendor/bin/phpunit becomes ../vendor/bin/phpunit and ../vendor/bin/phpunit becomes ../../vendor/bin/phpunit.

Depending on your development workflow, you may find it useful to soft link PHPUnit from /usr/local/bin ( cd /usr/local/bin; ln -s /var/www/html/vendor/bin/phpunit), or similar, or add vendor/bin to your PATH.

Verify that runtime assertions are being checked

When running phpunit tests, it is important to make sure that runtime assertion statements are being checked. See the note on running tests locally on the change notice about runtime assertions for more information.

Running tests

All example commands in this document assumes that you and the configuration file phpunit.xml are located in the core directory:

cd core

To run \Drupal\Tests\datetime\Unit\Plugin\migrate\field\DateFieldTest, for example, your command would look like this:

../vendor/bin/phpunit modules/datetime/tests/src/Unit/Plugin/migrate/field/DateFieldTest.php

Your path to the phpunit executable may be different than the above, see Locate the PHPUnit binary.

To run a core test in the DDEV container, use the following:

$ ddev ssh
$ cd web
../vendor/bin/phpunit -c core core/modules/announcements_feed

Successful execution with DDEV, looks like this:

PHPUnit 9.5.23 #StandWithUkraine

Warning:       Your XML configuration validates against a deprecated schema.
Suggestion:    Migrate your XML configuration using "--migrate-configuration"!

Testing /var/www/html/web/core/modules/action
....S..                                                             7 / 7 (100%)

Time: 03:25.204, Memory: 8.00 MB

OK, but incomplete, skipped, or risky tests!
Tests: 7, Assertions: 229, Skipped: 1.

HTML output was generated
http://localhost/sites/simpletest/browser_output/Drupal_Tests_action_Functional_ActionListTest-1-51362070.html
http://localhost/sites/simpletest/browser_output/Drupal_Tests_action_Functional_ActionListTest-2-51362070.html
http://localhost/sites/simpletest/browser_output/Drupal_Tests_action_Functional_ActionListTest-3-51362070.html
http://localhost/sites/simpletest/browser_output/Drupal_Tests_action_Functional_ActionListTest-4-51362070.html
http://localhost/sites/simpletest/browser_output/Drupal_Tests_action_Functional_ActionUninstallTest-1-80829434.html
...

You can also run the test in the containers from the host with:

$ ddev exec ./vendor/bin/phpunit -c web/core web/core/modules/action

Run all PHPUnit unit tests

To run the unit tests on OS X, Linux or other *nix systems:

../vendor/bin/phpunit --testsuite=unit 

Note: All tests, including those located in [drupalroot]/modules or [drupalroot]/sites/*/modules, are run from the core folder with the ../vendor/bin/phpunit command

Note also that you don't need to have a working Drupal installation to run PHPUnit-based unit tests this way. PHPUnit tests are isolated from Drupal and don't need it in order to run.

On Windows, the symlink stored in vendor/bin/phpunit will not work. You need to use the full path to the PHPUnit executable:

../vendor/phpunit/phpunit/phpunit

Run kernel test and browser tests

For kernel tests, you need a working database connection. For browser tests, your Drupal installation needs to be reachable via a web server. However, you should never install Drupal yourself as the test handles this automatically. 

Permission problems

Functional tests have to be invoked with a user in the same group as the webserver user. You can either configure Apache (or Nginx) to run as your own system user or run tests as a privileged user instead.

To develop locally, a straightforward - but also less secure - approach is to run tests as your own system user. To achieve that, change the default Apache user to run as your system user. Typically, you'd need to modify `/etc/apache2/envvars` on Linux or `/etc/apache2/httpd.conf` on Mac.

Example for Linux:

export APACHE_RUN_USER=<your-user>
export APACHE_RUN_GROUP=<your-group>

Example for Mac:

User your-user
Group your-group

If the default user is e.g. `www-data`, the above functional tests will have to be invoked with sudo instead:

export SIMPLETEST_DB='mysql://root@localhost/dev_d8'
export SIMPLETEST_BASE_URL='http://d8.dev'
sudo -u www-data -E ../vendor/bin/phpunit --testsuite functional
sudo -u www-data -E ../vendor/bin/phpunit --testsuite functional-javascript

If you have permission problems accessing files after running tests, try putting

$settings['file_chmod_directory'] = 02775;

in your settings.php or local.settings.php file.

You may need to use absolute paths in your phpunit.xml file, and/or in your PHPUnit command arguments.

Run one specific test

Simply specify the file name, example;

../vendor/bin/phpunit tests/Drupal/Tests/Core/Password/PasswordHashingTest.php 

Run groups of tests

To facilitate running specific tests, test authors use annotations to place their tests in one or more groups.

List all tests groups:
../vendor/bin/phpunit --list-groups

Run one group of tests:
../vendor/bin/phpunit --group Groupname

Run multiple groups of tests:
../vendor/bin/phpunit --group Group1,Group2

Exclude a group of tests:
../vendor/bin/phpunit --exclude-group Groupname

Run a specific method

../vendor/bin/phpunit --filter=MyMethodTest

Generate a code coverage report

../vendor/bin/phpunit --coverage-html /tmp/report
And then open /tmp/report/index.html in your browser to review.

For a complete discussion of command-line options when running tests, see PHPUnit's The Command-Line Test Runner

Run All PHPUnit Tests The Way The Testbot Does It

Running tests as described above is quick and easy, but it can be helpful to see how the drupal.org testbot runs your tests. This takes considerably longer but helps you understand things from the testbot's point of view.

The first step is to make sure you have a fully working Drupal installation, with the Testing module enabled. The drupal.org testbot assumes that contributed modules will be installed inside the modules/contrib directory and Drupal's application root for unit tests is set to a value assuming this directory structure.

Then, from the command line you can type:

php core/scripts/run-tests.sh PHPUnit

This specifies the run-tests.sh script should run the PHPUnit group of tests. This group is internally generated by the script and its integration with SimpleTest and represents any test that inherits \PHPUnit_Framework_TestCase including Drupal\Tests\UnitTestCase.

When using a base URL other than http://localhost the script requires a --url parameter

--url
   The base URL of the root directory of this Drupal checkout

For example:

php core/scripts/run-tests.sh --url http://drupal8.dev PHPUnit

Do not use core/scripts/run-tests.sh for Javascript tests. When ChromeDriver is not running, it will say tests passed when in fact they did not run. Use phpunit directly.

What to do with skipped tests

Skipped tests are usually an indication that your test environment is missing something, often a database connection. You can get more information about the skipped test by adding the -v parameter to your phpunit command.

If you get an error similar to:

InvalidArgumentException: There is no database connection so no tests can be run. You must provide a SIMPLETEST_DB environment variable, like "sqlite://localhost//tmp/test.sqlite", to run PHPUnit based functional tests outside of run-tests.sh. 

That means you have not set up your local phpunit.xml file copy correctly, see the kernel tests and browser tests section above.

Tests found when calling PHPUnit directly, but no tests found by drupal.org's testbot

The Drupal test runner (core/scripts/run-tests.sh) discovers tests in a different way than running PHPUnit directly. Confirm that your test conforms to the Drupal test runner standard as documented in PHPUnit file structure, namespace, and required metadata: Contributed Modules.

Most likely the Drupal test runner is not able to find your class in the autoloader, which means that the namespace and directory do not conform to the PSR-4 standard.

Tests pass locally but fail when run by drupal.org's testbot

One possible explanation for this is that you are introducing a new dependency to your module and testbot is not yet aware of this. If this is the case consider adding a test_dependencies property to your mymodule.info.yml file and committing it immediately. After pushing this change to drupal.org it can take up to 24 hours for testbot to become aware of your new dependency.

Another reason for locally-passing tests failing on the testbot is that locally you may be using a later 'dev' version of a 3rd-party dependency module, but the Drupal testbot is using only the most recent tagged release. If code changes have been made in the dev release which are necessary for your tests to pass, these will not be available to testbot in the official tagged release of the dependency module and could be the cause of the failures.

Using __DIR__ or UnitTestCase::$root to check the Drupal root may fail when your test assumes that the module is not underneath a modules/contrib directory. You can override this behavior in your test classes (see UnitTestCase::__construct).

One thing to check is the PHP and database versions. Some modules may support older versions of PHP, while Drupal may not support them. View PHP support for Drupal. It is a good idea to check to make sure that the automated test on drupal.org matches PHP and database versions of what you have locally.

Another thing may help in certain cases is that drupal.org's testbot runs tests in a subdirectory Drupal setup, for certain path related logic if it can not handle subdirectory properly, it might cause the failure. 

Have another possible explanation for this mismatch? Add it here...