Test fixtures

on.exit()

The first function you need to know about is base R’s on.exit(). on.exit() calls the code to supplied to its first argument when the current function exits, regardless of whether it returns a value or errors. You can use on.exit() to clean up after yourself by ensuring that every mess-making function call is paired with an on.exit() call that cleans up.

We can use this idea to turn sloppy() into neat():

neat <- function(x, sig_digits) {
  op <- options(digits = sig_digits)
  on.exit(options(op), add = TRUE, after = FALSE)
  print(x)
}

pi
#> [1] 3.141593
neat(pi, 2)
#> [1] 3.1
pi
#> [1] 3.141593

Here we make use of a useful pattern options() implements: when you call options(digits = sig_digits) it both sets the digits option and (invisibly) returns the previous value of digits. We can then use that value to restore the previous options.

on.exit() also works in tests:

test_that("can print one digit of pi", {
  op <- options(digits = 1)
  on.exit(options(op), add = TRUE, after = FALSE)
  
  expect_output(print(pi), "3")
})
#> Test passed 🌈
pi
#> [1] 3.141593

There are three main drawbacks to on.exit():

• You should always call it with add = TRUE and after = FALSE. These ensure that the call is added to the list of deferred tasks (instead of replaces) and is added to the front of the stack (not the back, so that cleanup occurs in reverse order to setup). These arguments only matter if you’re using multiple on.exit() calls, but it’s a good habit to always use them to avoid potential problems down the road.

• It doesn’t work outside a function or test. If you run the following code in the global environment, you won’t get an error, but the cleanup code will never be run:

  op <- options(digits = 1)
  on.exit(options(op), add = TRUE, after = FALSE)

  This is annoying when you are running tests interactively.

• You can’t program with it; on.exit() always works inside the current function so you can’t wrap up repeated on.exit() code in a helper function.

To resolve these drawbacks, we use withr::defer().

withr::defer()

withr::defer() resolves the main drawbacks of on.exit(). First, it has the behaviour we want by default; no extra arguments needed:

neat <- function(x, sig_digits) {
  op <- options(digits = sig_digits)
  withr::defer(options(op))
  print(x)
}

Second, it works when called in the global environment. Since the global environment isn’t perishable, like a test environment is, you have to call deferred_run() explicitly to execute the deferred events. You can also clear them, without running, with deferred_clear().

withr::defer(print("hi"))
#> Setting deferred event(s) on global environment.
#>   * Execute (and clear) with `deferred_run()`.
#>   * Clear (without executing) with `deferred_clear()`.

withr::deferred_run()
#> [1] "hi"

Finally, withr::defer() lets you pick which function to bind the clean up behaviour too. This makes it possible to create helper functions.

“Local” helpers

Imagine we have many functions where we want to temporarily set the digits option. Wouldn’t it be nice if we could write a helper function to automate? Unfortunately we can’t write a helper with on.exit():

local_digits <- function(sig_digits) {
  op <- options(digits = sig_digits)
  on.exit(options(op), add = TRUE, after = FALSE)
}
neater <- function(x, sig_digits) {
  local_digits(1)
  print(x)
}
neater(pi)
#> [1] 3.141593

This code doesn’t work because the cleanup happens too soon, when local_digits() exits, not when neat() finishes.

Fortunately, withr::defer() allows us to solve this problem by providing an envir argument that allows you to control when cleanup occurs. The exact details of how this works are rather complicated, but fortunately there’s a common pattern you can use without understanding all the details. Your helper function should always have an env argument that defaults to parent.frame(), which you pass to the second argument of defer():

local_digits <- function(sig_digits, env = parent.frame()) {
  op <- options(digits = sig_digits)
  withr::defer(options(op), env)
}

neater(pi)
#> [1] 3

Just like on.exit() and defer(), our helper also works within tests:

test_that("withr lets us write custom helpers for local state manipulation", {
  local_digits(1)
  expect_output(print(exp(1)), "3")
  
  local_digits(3)
  expect_output(print(exp(1)), "2.72")
})
#> Test passed 🎊

print(exp(1))
#> [1] 2.718282

We always call these helper functions local_; “local” here refers to the fact that the state change persists only locally, for the lifetime of the associated function or test.

Pre-existing helpers

But before you write your own helper function, make sure to check out the wide range of local functions already provided by withr:

We can use withr::local_options() to write yet another version of neater():

neatest <- function(x, sig_digits) {
  withr::local_options(list(digits = sig_digits))
  print(x)
}
neatest(pi, 3)
#> [1] 3.14

Each local_*() function has a companion with_() function, which is a nod to with(), and the inspiration for withr’s name. We won’t use the with_*() functions here, but you can learn more about them at withr.r-lib.org.

Case study: usethis

One place that we use test fixtures extensively is in the usethis package (usethis.r-lib.org), which provides functions for looking after the files and folders in R projects, especially packages. Many of these functions only make sense in the context of a package, which means to test them, we also have to be working inside an R package. We need a way to quickly spin up a minimal package in a temporary directory, then test some functions against it, then destroy it.

To solve this problem we create a test fixture, which we place in R/test-helpers.R so that’s it’s available for both testing and interactive experimentation:

local_create_package <- function(dir = file_temp(), env = parent.frame()) {
  old_project <- proj_get_()
  
  # create new folder and package
  create_package(dir, open = FALSE) # A
  withr::defer(fs::dir_delete(dir), envir = env) # -A
  
  # change working directory
  setwd(dir) # B
  withr::defer(setwd(old_project), envir = env) # -B
  
  # switch to new usethis project
  proj_set(dir) # C
  withr::defer(proj_set(old_project, force = TRUE), envir = env) # -C
  
  dir
}

Note that the cleanup automatically unfolds in the opposite order from the setup. Setup is A, then B, then C; cleanup is -C, then -B, then -A. This is important because we must create directory dir before we can make it the working directory; and we must restore the original working directory before we can delete dir; we can’t delete dir while it’s still the working directory!

local_create_package() is used in over 170 tests. Here’s one example that checks that usethis::use_roxygen_md() does the setup necessary to use roxygen2 in a package, with markdown support turned on. All 3 expectations consult the DESCRIPTION file, directly or indirectly. So it’s very convenient that local_create_package() creates a minimal package, with a valid DESCRIPTION file, for us to test against. And when the test is done — poof! — the package is gone.

test_that("use_roxygen_md() adds DESCRIPTION fields", {
  pkg <- local_create_package()
  use_roxygen_md()
  
  expect_true(uses_roxygen_md())
  expect_equal(desc::desc_get("Roxygen", pkg)[[1]], "list(markdown = TRUE)"))
  expect_true(desc::desc_has_fields("RoxygenNote", pkg))
})

File

If you move the local_() call outside of a test_that() block, it will affect all tests that come after it. This means that by calling the test fixture at the top of the file you can change the behaviour for all tests. This has both advantages and disadvantages:

• If you would otherwise have called the fixture in every test, you’ve saved yourself a bunch of work and duplicate code.

• On the downside, if your test fails and you want to recreate the failure in an interactive environment so you can debug, you need to remember to run all the setup code at the top of the file first.

Generally, I think it’s better to copy and paste test fixtures across many tests — sure, it adds some duplication to your code, but it makes debugging test failures so much easier.

Package

To run code before any test is run, you can create a file called tests/testthat/setup.R. If the code in this file needs clean up, you can use the special teardown_env():

# Run before any test
write.csv("mtcars.csv", mtcars)

# Run after all tests
withr::defer(unlink("mtcars.csv"), teardown_env())

Setup code is typically best used to create external resources that are needed by many tests. It’s best kept to a minimum because you will have to manually run it before interactively debugging tests.