| Title: | Test Coverage for Packages |
|---|---|
| Description: | Track and report code coverage for your package and (optionally) upload the results to a coverage service like 'Codecov' <https://about.codecov.io> or 'Coveralls' <https://coveralls.io>. Code coverage is a measure of the amount of code being exercised by a set of tests. It is an indirect measure of test quality and completeness. This package is compatible with any testing methodology or framework and tracks coverage of both R code and compiled C/C++/FORTRAN code. |
| Authors: | Jim Hester [aut, cre], Willem Ligtenberg [ctb], Kirill Müller [ctb], Henrik Bengtsson [ctb], Steve Peak [ctb], Kirill Sevastyanenko [ctb], Jon Clayden [ctb], Robert Flight [ctb], Eric Brown [ctb], Brodie Gaslam [ctb], Will Beasley [ctb], Robert Krzyzanowski [ctb], Markus Wamser [ctb], Karl Forner [ctb], Gergely Daróczi [ctb], Jouni Helske [ctb], Kun Ren [ctb], Jeroen Ooms [ctb], Ken Williams [ctb], Chris Campbell [ctb], David Hugh-Jones [ctb], Qin Wang [ctb], Doug Kelkhoff [ctb], Ivan Sagalaev [ctb, cph] (highlight.js library), Mark Otto [ctb] (Bootstrap library), Jacob Thornton [ctb] (Bootstrap library), Bootstrap contributors [ctb] (Bootstrap library), Twitter, Inc [cph] (Bootstrap library) |
| Maintainer: | Jim Hester <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 3.6.4.9003 |
| Built: | 2024-06-06 05:02:05 UTC |
| Source: | https://github.com/r-lib/covr |
covr tracks and reports code coverage for your package and (optionally) upload the results to a coverage service like 'Codecov' https://about.codecov.io or 'Coveralls' https://coveralls.io. Code coverage is a measure of the amount of code being exercised by a set of tests. It is an indirect measure of test quality and completeness. This package is compatible with any testing methodology or framework and tracks coverage of both R code and compiled C/C++/FORTRAN code.
A coverage report can be used to inspect coverage for each line in your
package. Using report() requires the additional dependencies DT and htmltools.
# If run with no arguments `report()` implicitly calls `package_coverage()` report()
covr uses the following options() to configure behaviour:
covr.covrignore: A filename to use as an ignore file,
listing glob-style wildcarded paths of files to ignore for coverage
calculations. Defaults to the value of environment variable
COVR_COVRIGNORE, or ".covrignore" if the neither the option nor the
environment variable are set.
covr.exclude_end: Used along with covr.exclude_start, an optional
regular expression which ends a line-exclusion region. For more
details, see ?exclusions.
covr.exclude_pattern: An optional line-exclusion pattern. Lines
which match the pattern will be excluded from coverage. For more details,
see ?exclusions.
covr.exclude_start: Used along with covr.exclude_end, an optional
regular expression which starts a line-exclusion region. For more
details, see ?exclusions.
covr.filter_non_package: If TRUE (the default behavior), coverage
of files outside the target package are filtered from coverage output.
covr.fix_parallel_mcexit:
covr.flags:
covr.gcov: If the appropriate gcov version is not on your path you
can use this option to set the appropriate location. If set to "" it will
turn off coverage of compiled code.
covr.gcov_additional_paths:
covr.gcov_args:
covr.icov:
covr.icov_args:
covr.icov_flags:
covr.icov_prof:
covr.rstudio_source_markers: A logical value. If TRUE (the
default behavior), source markers are displayed within the RStudio IDE
when using zero_coverage.
covr.record_tests: If TRUE (default NULL), record a listing of
top level test expressions and associate tests with covr traces
evaluated during the test's execution. For more details, see
?covr.record_tests.
covr.showCfunctions:
Maintainer: Jim Hester [email protected]
Other contributors:
Willem Ligtenberg [contributor]
Kirill Müller [contributor]
Henrik Bengtsson [contributor]
Steve Peak [contributor]
Kirill Sevastyanenko [contributor]
Jon Clayden [contributor]
Robert Flight [contributor]
Eric Brown [contributor]
Brodie Gaslam [contributor]
Will Beasley [contributor]
Robert Krzyzanowski [contributor]
Markus Wamser [contributor]
Karl Forner [contributor]
Gergely Daróczi [contributor]
Jouni Helske [contributor]
Kun Ren [contributor]
Jeroen Ooms [contributor]
Ken Williams [contributor]
Chris Campbell [contributor]
David Hugh-Jones [contributor]
Qin Wang [contributor]
Doug Kelkhoff [contributor]
Ivan Sagalaev (highlight.js library) [contributor, copyright holder]
Mark Otto (Bootstrap library) [contributor]
Jacob Thornton (Bootstrap library) [contributor]
Bootstrap contributors (Bootstrap library) [contributor]
Twitter, Inc (Bootstrap library) [copyright holder]
Useful links:
Report bugs at https://github.com/r-lib/covr/issues
Convert a counters object to a coverage object
as_coverage(counters = NULL, ...)counters |
An environment of covr trace results to convert to a coverage
object. If |
... |
Additional attributes to include with the coverage object. |
For tests produced with options(covr.record_tests), prune any unused
records in the $tests$tally matrices of each trace and get rid of the
wrapping $tests environment (reassigning with value of $tests$tally)
as_coverage_with_tests(counters)counters |
An environment of covr trace results to convert to a coverage
object. If |
Run covr on a package and output the result so it is available on Azure Pipelines
azure(
...,
coverage = package_coverage(..., quiet = quiet),
filename = "coverage.xml",
quiet = TRUE
)... |
arguments passed to |
coverage |
an existing coverage object to submit, if |
filename |
the name of the Cobertura XML file |
quiet |
if |
This function is useful for testing, and is a thin wrapper around
file_coverage() because parseData is not populated properly
unless the functions are defined in a file.
code_coverage(
source_code,
test_code,
line_exclusions = NULL,
function_exclusions = NULL,
...
)source_code |
A character vector of source code |
test_code |
A character vector of test code |
line_exclusions |
a named list of files with the lines to exclude from each file. |
function_exclusions |
a vector of regular expressions matching function
names to exclude. Example |
... |
Additional arguments passed to |
source <- "add <- function(x, y) { x + y }"
test <- "add(1, 2) == 3"
code_coverage(source, test)Run covr on a package and upload the result to codecov.io
codecov(
...,
coverage = NULL,
base_url = "https://codecov.io",
token = NULL,
commit = NULL,
branch = NULL,
pr = NULL,
flags = NULL,
quiet = TRUE
)... |
arguments passed to |
coverage |
an existing coverage object to submit, if |
base_url |
Codecov url (change for Enterprise) |
token |
a codecov upload token, if
|
commit |
explicitly set the commit this coverage result object
corresponds to. Is looked up from the service or locally if it is
|
branch |
explicitly set the branch this coverage result object
corresponds to, this is looked up from the service or locally if it is
|
pr |
explicitly set the pr this coverage result object corresponds to,
this is looked up from the service if it is |
flags |
A flag to use for this coverage upload see https://docs.codecov.com/docs/flags for details. |
quiet |
if |
## Not run:
codecov(path = "test")
## End(Not run)Convert a coverage dataset to a list
coverage_to_list(x = package_coverage())x |
a coverage dataset, defaults to running |
A list containing coverage result for each individual file and the whole package
Run covr on a package and upload the result to coveralls
coveralls(
...,
coverage = NULL,
repo_token = Sys.getenv("COVERALLS_TOKEN"),
service_name = Sys.getenv("CI_NAME", "travis-ci"),
quiet = TRUE
)... |
arguments passed to |
coverage |
an existing coverage object to submit, if |
repo_token |
The secret repo token for your repository, found at the bottom of your repository's page on Coveralls. This is useful if your job is running on a service Coveralls doesn't support out-of-the-box. If set to NULL, it is assumed that the job is running on travis-ci |
service_name |
the CI service to use, if environment variable ‘CI_NAME’ is set that is used, otherwise ‘travis-ci’ is used. |
quiet |
if |
By setting options(covr.record_tests = TRUE), the result of covr coverage
collection functions will include additional data pertaining to the tests
which are executed and an index of which tests, at what stack depth, trigger
the execution of each trace.
This functionality requires that the package code and tests are installed and
sourced with the source. For more details, refer to R options, keep.source,
keep.source.pkgs and keep.parse.data.pkgs.
Within the covr result, you can explore this information in two places:
attr(,"tests"): A list of call stacks, which results in target code
execution.
$<srcref>$tests: For each srcref count in the coverage object, a
$tests field is now included which contains a matrix with three columns,
"test", "call", "depth" and "i" which specify the test number
(corresponding to the index of the test in attr(,"tests"), the number
of times the test expression was evaluated to produce the trace hit, the
stack depth into the target code where the trace was executed, and the
order of execution for each test.
The content of test traces are dependent on the unit testing framework that is used by the target package. The behavior is contingent on the available information in the sources kept for the testing files.
Test traces are extracted by the following criteria:
If any srcref files are are provided by a file within covr's temporary
library, all calls from those files are kept as a test trace. This will
collect traces from tests run with common testing frameworks such as
testthat and RUnit.
Otherwise, as a conservative fallback in situations where no source references are found, or when none are from within the temporary directory, the entire call stack is collected.
These calls are subsequently subset for only those up until the call to
covr's internal count function, and will always include the last call in
the call stack prior to a call to count.
fcode <- '
f <- function(x) {
if (x)
f(!x)
else
FALSE
}'
options(covr.record_tests = TRUE)
cov <- code_coverage(fcode, "f(TRUE)")
# extract executed test code for the first test
tail(attr(cov, "tests")[[1L]], 1L)
# [[1]]
# f(TRUE)
# extract test itemization per trace
cov[[3]][c("srcref", "tests")]
# $srcref
# f(!x)
#
# $tests
# test call depth i
# [1,] 1 1 2 4
# reconstruct the code path of a test by ordering test traces by [,"i"]
lapply(cov, `[[`, "tests")
# $`source.Ref2326138c55:4:6:4:10:6:10:4:4`
# test call depth i
# [1,] 1 1 1 2
#
# $`source.Ref2326138c55:3:8:3:8:8:8:3:3`
# test call depth i
# [1,] 1 1 1 1
# [2,] 1 1 2 3
#
# $`source.Ref2326138c55:6:6:6:10:6:10:6:6`
# test call depth i
# [1,] 1 1 2 4A single test expression might be evaluated many times. Each time the same expression is called, the call count is incremented.
current_test_call_count()An integer value representing the number of calls of the current call into the package from the testing suite.
Calculate coverage of an environment
environment_coverage(
env = parent.frame(),
test_files,
line_exclusions = NULL,
function_exclusions = NULL
)env |
The environment to be instrumented. |
test_files |
Character vector of test files with code to test the functions |
line_exclusions |
a named list of files with the lines to exclude from each file. |
function_exclusions |
a vector of regular expressions matching function
names to exclude. Example |
covr supports a couple of different ways of excluding some or all of a file.
The line_exclusions argument to package_coverage() can be used
to exclude some or all of a file. This argument takes a list of filenames
or named ranges to exclude.
Alternatively function_exclusions can be used to exclude R functions
based on regular expression(s). For example print\\\.* can be used to
exclude all the print methods defined in a package from coverage.
In addition you can exclude lines from the coverage by putting special comments
in your source code. This can be done per line or by specifying a range.
The patterns used can be specified by the exclude_pattern, exclude_start,
exclude_end arguments to package_coverage() or by setting the global
options covr.exclude_pattern, covr.exclude_start, covr.exclude_end.
## Not run:
# exclude whole file of R/test.R
package_coverage(exclusions = "R/test.R")
# exclude lines 1 to 10 and 15 from R/test.R
package_coverage(line_exclusions = list("R/test.R" = c(1:10, 15)))
# exclude lines 1 to 10 from R/test.R, all of R/test2.R
package_coverage(line_exclusions = list("R/test.R" = 1:10, "R/test2.R"))
# exclude all print and format methods from the package.
package_coverage(function_exclusions = c("print\\.", "format\\."))
# single line exclusions
f1 <- function(x) {
x + 1 # nocov
}
# ranged exclusions
f2 <- function(x) { # nocov start
x + 2
} # nocov end
## End(Not run)The files in source_files are first sourced into a new environment
to define functions to be checked. Then they are instrumented to track
coverage and the files in test_files are sourced.
file_coverage(
source_files,
test_files,
line_exclusions = NULL,
function_exclusions = NULL,
parent_env = parent.frame()
)source_files |
Character vector of source files with function definitions to measure coverage |
test_files |
Character vector of test files with code to test the functions |
line_exclusions |
a named list of files with the lines to exclude from each file. |
function_exclusions |
a vector of regular expressions matching function
names to exclude. Example |
parent_env |
The parent environment to use when sourcing the files. |
# For the purpose of this example, save code containing code and tests to files
cat("add <- function(x, y) { x + y }", file="add.R")
cat("add(1, 2) == 3", file="add_test.R")
# Use file_coverage() to calculate test coverage
file_coverage(source_files = "add.R", test_files = "add_test.R")
# cleanup
file.remove(c("add.R", "add_test.R"))A coverage report for a specific file
file_report(
x = package_coverage(),
file = NULL,
out_file = file.path(tempdir(), paste0(get_package_name(x), "-file-report.html")),
browse = interactive()
)x |
a coverage dataset, defaults to running |
file |
The file to report on, if |
out_file |
The output file |
browse |
whether to open a browser to view the report. |
Calculate test coverage for a specific function.
function_coverage(fun, code = NULL, env = NULL, enc = parent.frame())fun |
name of the function. |
code |
expressions to run. |
env |
environment the function is defined in. |
enc |
the enclosing environment which to run the expressions. |
add <- function(x, y) { x + y }
function_coverage(fun = add, code = NULL) # 0% coverage
function_coverage(fun = add, code = add(1, 2) == 3) # 100% coverageUtilize internal GitLab static pages to publish package coverage. Creates local covr report in a package subdirectory. Uses the pages GitLab job to publish the report.
gitlab(..., coverage = NULL, file = "public/coverage.html", quiet = TRUE)... |
arguments passed to |
coverage |
an existing coverage object to submit, if |
file |
The report filename. |
quiet |
if |
Is the source bound to the expression
has_srcref(expr)expr |
A language object which may have a |
A logical value indicating whether the language object has source
covr functions set the environment variable R_COVR when they are running.
in_covr() returns TRUE if this environment variable is set and FALSE
otherwise.
in_covr()if (require(testthat)) {
testthat::skip_if(in_covr())
}Is the expression a call to covr:::count
is_covr_count_call(expr)expr |
A language object |
A logical value indicating whether the object is a call to
covr:::count.
Quickly dismiss the need to update the current test if we can. To test if we're still in the last test, check if the same srcref (or call, if source is not kept) exists at the last recorded calling frame prior to entering a covr trace. If this has changed, do a more comprehensive test to see if any of the test call stack has changed, in which case we are onto a new test.
is_current_test_finished()Initialize a test counter, a matrix used to tally tests, their stack depth
and the execution order as the trace associated with key is hit. Each
test trace is an environment, which allows assignment into a pre-allocated
tests matrix with minimall reallocation.
new_test_counter(key)key |
generated with |
The tests matrix has columns tests, depth and i,
corresponding to the test index (the index of the associated test in
.counters$tests), the stack depth when the trace is evaluated and the
number of traces that have been hit so far during test evaluation.
This function calculates the test coverage for a development package on the
path. By default it runs only the package tests, but it can also run
vignette and example code.
package_coverage(
path = ".",
type = c("tests", "vignettes", "examples", "all", "none"),
combine_types = TRUE,
relative_path = TRUE,
quiet = TRUE,
clean = TRUE,
line_exclusions = NULL,
function_exclusions = NULL,
code = character(),
install_path = temp_file("R_LIBS"),
...,
exclusions,
pre_clean = TRUE
)path |
file path to the package. |
type |
run the package ‘tests’, ‘vignettes’, ‘examples’, ‘all’, or ‘none’. The default is ‘tests’. |
combine_types |
If |
relative_path |
whether to output the paths as relative or absolute paths. If a string, it is interpreted as a root path and all paths will be relative to that root. |
quiet |
whether to load and compile the package quietly, useful for debugging errors. |
clean |
whether to clean temporary output files after running, mainly useful for debugging errors. |
line_exclusions |
a named list of files with the lines to exclude from each file. |
function_exclusions |
a vector of regular expressions matching function
names to exclude. Example |
code |
A character vector of additional test code to run. |
install_path |
The path the instrumented package will be installed to
and tests run in. By default it is a path in the R sessions temporary
directory. It can sometimes be useful to set this (along with |
... |
Additional arguments passed to |
exclusions |
‘Deprecated’, please use ‘line_exclusions’ instead. |
pre_clean |
whether to delete all objects present in the src directory before recompiling |
This function uses tools::testInstalledPackage() to run the
code, if you would like to test your package in another way you can set
type = "none" and pass the code to run as a character vector to the
code parameter.
Parallelized code using parallel's mcparallel() needs to
use a patched parallel:::mcexit. This is done automatically if the
package depends on parallel, but can also be explicitly set using the
environment variable COVR_FIX_PARALLEL_MCEXIT or the global option
covr.fix_parallel_mcexit.
exclusions() For details on excluding parts of the
package from the coverage calculations.
Calculate the total percent coverage from a coverage result object.
percent_coverage(x, ...)x |
the coverage object returned from |
... |
additional arguments passed to |
The total percentage as a numeric(1).
Print a coverage object
## S3 method for class 'coverage'
print(x, group = c("filename", "functions"), by = "line", ...)x |
the coverage object to be printed |
group |
whether to group coverage by filename or function |
by |
whether to count coverage by line or expression |
... |
additional arguments ignored |
The coverage object (invisibly).
Display covr results using a standalone report
report(
x = package_coverage(),
file = file.path(tempdir(), paste0(get_package_name(x), "-report.html")),
browse = interactive()
)x |
a coverage dataset, defaults to running |
file |
The report filename. |
browse |
whether to open a browser to view the report. |
## Not run:
x <- package_coverage()
report(x)
## End(Not run)Tally coverage by line or expression
tally_coverage(x, by = c("line", "expression"))x |
the coverage object returned from |
by |
whether to tally coverage by line or expression |
a data.frame of coverage tallied by line or expression.
Create a
cobertura-compliant XML report following this DTD.
Because there are two DTDs called coverage-04.dtd and some tools do not seem to
adhere to either of them, the parser you're using may balk at the file. Please see
this github discussion for
context. Where covr doesn't provide a coverage metric (branch coverage,
complexity), a zero is reported.
to_cobertura(cov, filename = "cobertura.xml")cov |
the coverage object returned from |
filename |
the name of the Cobertura XML file |
Note: This functionality requires the xml2 package be installed.
This functionality requires the xml2 package be installed.
to_sonarqube(cov, filename = "sonarqube.xml")cov |
the coverage object returned from |
filename |
the name of the SonarQube Generic XML file |
Talkdesk Inc.
A helper to circumvent R errors when deserializing large call objects from
Rds. Trims the number of arguments in a call object, and replaces the last
argument with a <truncated> symbol.
truncate_call(call_obj, limit = 10000)call_obj |
A (possibly large) |
limit |
A |
The call_obj with arguments trimmed
Retrieve the value from an object
value(x, ...)x |
object from which to retrieve the value |
... |
additional arguments passed to methods |
When examining the test coverage of a package, it is useful to know if there are any locations where there is 0 test coverage.
zero_coverage(x, ...)x |
a coverage object returned |
... |
additional arguments passed to
|
if used within RStudio this function outputs the results using the Marker API.
A data.frame with coverage data where the coverage is 0.