Writing Tests and Examples¶
VPLanet includes “unit tests” to maintain code results as the project grows. These tests are short versions of the examples that run in a few seconds, which is sufficient time to ensure that integration is accurate. Unit tests are also used to track the amount of the code that is executed over the sum of all tests, further enabling confidence in the code’s accuracy.
Test scripts live in the
tests/ directory in the top-level repo folder.
Tests are executed automatically on Travis using
py.test, which will
find all Python files with the word
test somewhere in their name (side note:
if it’s not meant to be a test, don’t put
test in the file name!).
py.test suite specifically checks for cases where an
is raised, so in your tests you should use
assert statements to check whether
the output is equal to (or very close to) a benchmarked value.
To write a test, create a directory in
tests/ with a descriptive name
(such as the modules it’s meant to test or a specific application of the code).
.in files needed to run the test and create a Python
test_<TEST_NAME>.py. This file should follow this basic structure:
import vplanet import numpy as np import pathlib # Path to this directory path = pathlib.Path(__file__).parents.absolute() def test_something(): """Brief description of what we're testing.""" # Run vplanet output = vplanet.run(path / "vpl.in") # Run our comparisons assert np.isclose(output.log.final.planet.Eccentricity, 0) assert np.isclose(output.log.final.system.TotEnergy, output.log.initial.system.TotEnergy) assert np.allclose(output.star.Luminosity, np.array([0.0672752 , 0.0672752 , 0.0080845...]))
In the example above, we’re checking three things: that the final planet eccentricity (from the log file) is zero; that the final system energy (from the log file) is equal to the initial system energy; and that the star’s luminosity (from the forward file) is equal to some array of values.
These unit tests not only ensure new modification don’t break parts of the code that already work. In addition, they are used to compute the fraction of the code that is verified with CodeCov <https://app.codecov.io/gh/VirtualPlanetaryLaboratory/vplanet> Thus, to approach (and someday maintain) 100% verification requires all new features to include a unit test. Finally, unit tests are valuable for ensuring that no memory issues are present through periodic valgrind tests.
Examples live in the
examples/ directory in the top-level repo folder, and,
just like tests, there should be one aptly-named folder per example. Include
the input files needed to run the example and a
README.rst file that
describes the example and its expected output in detail:
Example name ============ Overview -------- =================== ============ **Date** MM/DD/YY **Author** AUTHOR_NAME **Modules** `MODULE1_NAME <../src/MODULE1_NAME.html>`_ `MODULE2_NAME <../src/MODULE2_NAME.html>`_ **Approx. runtime** RUNTIME_IN_SECONDS **Source code** `GitHub <https://github.com/VirtualPlanetaryLaboratory/vplanet-private/tree/master/examples/EXAMPLE_DIR_NAME>`_ =================== ============ DETAILED DESCRIPTION OF THE EXAMPLE To run this example ------------------- EXACT INSTRUCTIONS TO REPRODUCE EXAMPLE .. code-block:: bash python makeplot.py Expected output --------------- FIGURES SHOWCASING THE EXAMPLE OUTPUT .. figure:: UNIQUE_FIGURE_NAME.png :width: 600px :align: center FIGURE CAPTION
All example figures are generated by the same command: python makplot.py.
When creating an example, run it to figure out the approximate
runtime (see above) and create low resolution PNG images (no more
than about 800x800 pixels) with unique file names (so they don’t
overwrite images from other examples when building the docs) in the
example directory. To commit these images, you’ll need to force add them
png files are in the .gitignore by default:
git add -f UNIQUE_FIGURE_NAME.png