Introduction

  • This course will teach you how to write effective tests and ensure the quality and reliability of your research software.
  • No prior testing experience is required.
  • You can catch up on practicals by copying the corresponding folder from the learners/files directory of this course’s materials.

Why Test My Code?

  • Automated testing helps to catch hard to spot errors in code & find the root cause of complex issues.
  • Tests reduce the time spent manually verifying (and re-verifying!) that code works.
  • Tests help to ensure that code works as expected when changes are made.
  • Tests are especially useful when working in a team, as they help to ensure that everyone can trust the code.

Simple Tests

  • The assert keyword is used to check if a statement is true.
  • Pytest is invoked by running the command pytest ./ in the terminal.
  • pytest will run all the tests in the current directory, found by looking for files that start with test_.
  • The output of a test is displayed in the terminal, with green text indicating a successful test and red text indicating a failed test.
  • It’s best practice to write tests in a separate file from the code they are testing. Eg: scripts.py and test_scripts.py.

Interacting with Tests

  • You can run multiple tests at once by running pytest in the terminal.
  • Pytest searches for tests in files that start or end with ‘test’ in the current directory and subdirectories.
  • The output of pytest tells you which tests have passed and which have failed and precisely why they failed.
  • Pytest accepts many additional flags to change which tests are run, give more detailed output, etc.

Unit tests & Testing Practices

  • Complex functions can be broken down into smaller, testable units.
  • Testing each unit separately is called unit testing.
  • The AAA pattern is a good way to structure your tests.
  • Test driven development can help you to write clean, maintainable code.
  • Randomness in tests can be made deterministic using random seeds.
  • Adding tests to an existing project can be done incrementally, starting with regression tests.

Testing for Exceptions

  • Use pytest.raises to check that a function raises an exception.

Floating Point Data

  • When comparing floating point data, you should use relative/absolute tolerances instead of testing for equality.
  • Numpy arrays cannot be compared using the == operator. Instead, use numpy.testing.assert_array_equal and numpy.testing.assert_allclose.

Fixtures

  • Fixtures are useful way to store data, objects and automations to re-use them in many different tests.
  • Fixtures are defined using the @pytest.fixture decorator.
  • Tests can use fixtures by passing them as arguments.
  • Fixtures can be placed in a separate file or in the same file as the tests.

Parametrization

  • Parametrization is a way to run the same test with different parameters in a concise and more readable way, especially when there is a lot of repetition in the setup for each of the different test cases.
  • Use the @pytest.mark.parametrize decorator to define a parametrized test.

Regression Tests

  • Regression testing ensures that the output of a function remains consistent between test runs.
  • The pytest plugin, snaptol, can be used to simplify this process and cater for floating point numbers that may need tolerances on assertion checks.

Continuous Integration with GitHub Actions

  • Continuous Integration (CI) is the practice of automating the merging of code changes into a project.
  • GitHub Actions is a feature of GitHub that allows you to automate the testing of your code.
  • GitHub Actions are defined in yaml files and are stored in the .github/workflows directory in your repository.
  • You can use GitHub Actions to ensure your tests pass before merging new code into your main branch.