Writing Tests and Examples

Unit Tests

VPLanet includes “unit tests” to maintain code results as the project grows. These tests are short versions of the examples that run in a few seconds, which is sufficient time to ensure that integration is accurate. Unit tests are also used to track the amount of the code that is executed over the sum of all tests, further enabling confidence in the code’s accuracy.

Test scripts live in the tests/ directory and are executed automatically on every pull request with GitHub Actions. To perform the tests, we use pytest, which will find all Python files with the word test somewhere in their name (side note: if it’s not meant to be a test, don’t put test in the file name!). Under the tests directory are subdirectories whose names consists of VPLanet modules, e.g. BINARY and AtmescEqtideStellar. These subdirectories contain the tests available for the module combinations of the subdirectory name.

The VPLanet team has made it easy to add or revise tests. Inside the tests/ directory is a file called maketest.py, which will generate a unit test from a set of valid infiles. So a typical procedure to create a new test is to:

  • Create a simulation that executes previously untested functionality

  • Verify the results are accurate! (Obvious, we know, but please don’t forget!)

  • Create a subdirectory with a unique name for your test.

  • Copy the .in files for your test into your new subdirectory

  • cd to the tests directory and run `python maketest.py <modules/subdir>`, which the argument is the relative to path to your new test

  • Commit your .in files and the new :code:`test_<subdir>.py” file to the repository

  • Issue a PR!

These unit tests not only ensure new modification don’t break parts of the code that already work. In addition, they are used to compute the fraction of the code that is verified with CodeCov <https://app.codecov.io/gh/VirtualPlanetaryLaboratory/vplanet> Thus, to approach (and someday maintain) 100% verification requires all new features to include a unit test. Finally, unit tests are valuable for ensuring that no memory issues are present through periodic valgrind tests.

Writing Examples

Examples live in the examples/ directory in the top-level repo folder, and, just like tests, there should be one aptly-named folder per example. Include the input files needed to run the example and a README.rst file that describes the example and its expected output in detail:

Example name
============

Overview
--------

===================   ============
**Date**              MM/DD/YY
**Author**            AUTHOR_NAME
**Modules**           `MODULE1_NAME <../src/MODULE1_NAME.html>`_
                      `MODULE2_NAME <../src/MODULE2_NAME.html>`_
**Approx. runtime**   RUNTIME_IN_SECONDS
**Source code**       `GitHub <https://github.com/VirtualPlanetaryLaboratory/vplanet-private/tree/master/examples/EXAMPLE_DIR_NAME>`_
===================   ============

DETAILED DESCRIPTION OF THE EXAMPLE

To run this example
-------------------

EXACT INSTRUCTIONS TO REPRODUCE EXAMPLE

.. code-block:: bash

    python makeplot.py


Expected output
---------------

FIGURES SHOWCASING THE EXAMPLE OUTPUT

.. figure:: UNIQUE_FIGURE_NAME.png
   :width: 600px
   :align: center

   FIGURE CAPTION
note:

All example figures are generated by the same command: python makplot.py.

When creating an example, run it to figure out the approximate runtime (see above) and create low resolution PNG images (no more than about 800x800 pixels) with unique file names (so they don’t overwrite images from other examples when building the docs) in the example directory. To commit these images, you’ll need to force add them since png files are in the .gitignore by default:

git add -f UNIQUE_FIGURE_NAME.png