diff --git a/browser_reprex/R_logo.png b/browser_reprex/R_logo.png new file mode 100644 index 0000000..4b5505a Binary files /dev/null and b/browser_reprex/R_logo.png differ diff --git a/browser_reprex/breakpoints.webm b/browser_reprex/breakpoints.webm new file mode 100644 index 0000000..58db220 Binary files /dev/null and b/browser_reprex/breakpoints.webm differ diff --git a/browser_reprex/browser_1.webm b/browser_reprex/browser_1.webm new file mode 100644 index 0000000..e664e7b Binary files /dev/null and b/browser_reprex/browser_1.webm differ diff --git a/browser_reprex/browser_2.webm b/browser_reprex/browser_2.webm new file mode 100644 index 0000000..0e55a79 Binary files /dev/null and b/browser_reprex/browser_2.webm differ diff --git a/browser_reprex/index.qmd b/browser_reprex/index.qmd new file mode 100644 index 0000000..1112f8c --- /dev/null +++ b/browser_reprex/index.qmd @@ -0,0 +1,421 @@ +--- +title: "Debugging with reprex() and browser()" +description: "How to efficiently solve bugs." +author: "Etienne Bacher" +date: "2026-06-25" +categories: [r] +difficulty: Intermediate +image: R_logo.png +toc: true +format: + html: default + revealjs: + output-file: index-slides.html +execute: + warning: false + message: false + freeze: auto +editor: + markdown: + wrap: 72 +--- + +Knowing how to debug code is an essential skill, whether one writes an analysis for a paper or develops an R package. +This module covers several tools that aim at making the debugging process easier. + + +```{css} +/*| eval: true*/ +/*| echo: false*/ +code.sourceCode.diff span.va { + color: #b4f1b4; + background: #1b4721; +} + +code.sourceCode.diff span.st { + color: #ffd8d3; + background-color: #78191b; +} +``` + + +The following code is taken from the ["Latitudinal trends in Phanerozoic reefs" vignette](https://palaeoverse.palaeoverse.org/articles/phanerozoic-reefs.html), but let's pretend we have written this code from scratch: + +```{r} +#| error: true +library(palaeoverse) + +data(reefs) +reef_counts <- group_apply(occdf = reefs, group = "interval", fun = nrow) +# Assign a common time scale based on an interval key +reefs <- look_up(occdf = reefs, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +reefs <- subset(reefs, interval_max_ma <= 541) +# Extract Phanerozoic stage-level stages for time bins +bins <- time_bins(interval = "Phanerozoic", rank = "stage") + +bin_time(occdf = reefs, bins = bins, method = "mid") +``` + +We now have an error, but its cause is not immediately clear. +Therefore, we'll have to do some debugging. + + +# Reducing the problem + +The longer the code you try to debug, the more potential causes you have to explore. The first thing to do when you can't find the bug in your code right away is to build a minimal **repr**oducible **ex**ample, aka a *reprex* (this is also known as a MRE, for "Minimal Reproducible Example"). + +In our example above, we can reduce the problem in two ways: we can reduce the data first, and then reduce the code. + + +## Reduce the size of the data + +In the debugging process, we will likely run the code at least a handful of times. +Therefore, we want to make sure that it runs as fast as possible so that we don't lose our focus. +The easiest way to do this is to reduce the size of our input data. + +In addition to making the error appear faster, it is also easier to compare the actual and expected outputs when the data is as small as possible. + +In this example, the dataset we are using has thousands of rows and 14 columns: + +```{r} +dim(reefs) +``` + +Let's try to reduce the size of our data: + +```{r} +#| error: true +reefs2 <- head(reefs) + +reef_counts <- group_apply(occdf = reefs2, group = "interval", fun = nrow) +# Assign a common time scale based on an interval key +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +reefs2 <- subset(reefs2, interval_max_ma <= 541) +# Extract Phanerozoic stage-level stages for time bins +bins <- time_bins(interval = "Phanerozoic", rank = "stage") + +bin_time(occdf = reefs2, bins = bins, method = "mid") +``` + +We still see the same error but now we only have 6 rows left, much better! +Let's try to reduce that to one row and to remove some columns. + + +```{r} +#| error: true +reefs2 <- head(reefs, n = 1) +reefs2 <- reefs2[, 1:5] + +reef_counts <- group_apply(occdf = reefs2, group = "interval", fun = nrow) +# Assign a common time scale based on an interval key +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +reefs2 <- subset(reefs2, interval_max_ma <= 541) +# Extract Phanerozoic stage-level stages for time bins +bins <- time_bins(interval = "Phanerozoic", rank = "stage") + +bin_time(occdf = reefs2, bins = bins, method = "mid") +``` + +Hmm, we don't have the same error anymore, so we removed too many columns. +Let's try to keep 6 instead of 5: + +```{r} +#| error: true +reefs2 <- head(reefs, n = 1) +reefs2 <- reefs2[, 1:6] + +reef_counts <- group_apply(occdf = reefs2, group = "interval", fun = nrow) +# Assign a common time scale based on an interval key +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +reefs2 <- subset(reefs2, interval_max_ma <= 541) +# Extract Phanerozoic stage-level stages for time bins +bins <- time_bins(interval = "Phanerozoic", rank = "stage") + +bin_time(occdf = reefs2, bins = bins, method = "mid") +``` + +Bingo! We have the same error as before, but now we only have one row and six columns. + +Let's now try to reduce the size of the code. + + +## Reduce the size of the code + +This is an important process: we want to remove as much code as possible to lower the number of potential causes leading to the error. + +Starting from this: +```{r} +#| error: true +reefs2 <- head(reefs, n = 1) +reefs2 <- reefs2[, 1:6] + +reef_counts <- group_apply(occdf = reefs2, group = "interval", fun = nrow) +# Assign a common time scale based on an interval key +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +reefs2 <- subset(reefs2, interval_max_ma <= 541) +# Extract Phanerozoic stage-level stages for time bins +bins <- time_bins(interval = "Phanerozoic", rank = "stage") + +bin_time(occdf = reefs2, bins = bins, method = "mid") +``` + +We see that `reef_counts` is defined but never actually used, so this cannot be the root cause. Let's remove it. + +The `subset()` call is also suspicious: now that he have only one row left in our test data, does this `subset()` actually do anything? + +```{r} +reefs2 <- head(reefs, n = 1) +reefs2 <- reefs2[, 1:6] + +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) + +identical(reefs2, subset(reefs2, interval_max_ma <= 541)) +``` + +It doesn't affect `reefs2`, so let's remove it as well. + +We now have this: +```{r} +#| error: true +reefs2 <- head(reefs, n = 1) +reefs2 <- reefs2[, 1:6] + +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +bins <- time_bins(interval = "Phanerozoic", rank = "stage") + +bin_time(occdf = reefs2, bins = bins, method = "mid") +``` + +We have removed two out of five function calls, not bad. +We still don't really understand the cause though. + +Let's assume that the documentation also doesn't give any hint about this error message. +If we are completely stuck, then we may want to ask other people, such as package maintainers on Github or people on StackOverflow. We have done our best to reduce the size of the problem, making it easier for external people to help us. + +Still, we need to ensure that our reprex is correctly made and we must find a nice way to share it. + + +# How to ensure that your reprex actually is a reprex + +It seems like we have done everything we could to make the problem easier to reproduce, even for other people (e.g. package developers). However, there is one thing that still needs to be cleaned: our **environment**. + +Our environment may have tons of variables, datasets, and options that were defined before but are not relevant to our problem. Some of those objects may interact in some way with our problematic code, meaning that we need to clean our environment to ensure that we can correctly reproduce the issue. + +One way to do this is to restart the R session, but this can easily be forgotten. A more robust solution is to use the package [`reprex`](https://reprex.tidyverse.org/). This package provides a function called `reprex()` in which we can put all the code related to our problem. `reprex()` will start a separate R session in the background to run this code and then report its results. Since this is a completely separate R session, it will error if our code isn't completely self-contained, and this would indicate that our reproducible example isn't actually... reproducible. + +This is extremely useful to solve the problem yourself, but it is even more useful when reporting a bug to colleagues and package developers: if your example runs in `reprex()`, then it is almost guaranteed that it can be easily reproduced by other people with different environments. + +```{r} +reprex::reprex({ + reefs2 <- head(reefs, n = 1) + reefs2 <- reefs2[, 1:6] + + reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) + bins <- time_bins(interval = "Phanerozoic", rank = "stage") + + bin_time(occdf = reefs2, bins = bins, method = "mid") +}) +``` + +``` r +reefs2 <- head(reefs, n = 1) +#> Error: +#> ! object 'reefs' not found +reefs2 <- reefs2[, 1:6] +#> Error: +#> ! object 'reefs2' not found + +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +#> Error in `look_up()`: +#> ! could not find function "look_up" +bins <- time_bins(interval = "Phanerozoic", rank = "stage") +#> Error in `time_bins()`: +#> ! could not find function "time_bins" + +bin_time(occdf = reefs2, bins = bins, method = "mid") +#> Error in `bin_time()`: +#> ! could not find function "bin_time" +``` + +Our reprex fails, but not with the error we experienced so far! +We forgot to add the calls to `library()` and `data()`. + + +```{r} +#| eval: false +reprex::reprex({ + library(palaeoverse) + data(reefs) + reefs2 <- head(reefs, n = 1) + reefs2 <- reefs2[, 1:6] + + reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) + bins <- time_bins(interval = "Phanerozoic", rank = "stage") + + bin_time(occdf = reefs2, bins = bins, method = "mid") +}) +``` + +``` r +library(palaeoverse) +data(reefs) +reefs2 <- head(reefs, n = 1) +reefs2 <- reefs2[, 1:6] + +reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) +bins <- time_bins(interval = "Phanerozoic", rank = "stage") + +bin_time(occdf = reefs2, bins = bins, method = "mid") +#> Error in `[.data.frame`: +#> ! undefined columns selected +``` + +This is the situation we had so far, so we can now share this output with other people and they should be able to reproduce the issue. + +Note that it is good practice to show more information on our session, including the version of packages used. This can be done with the argument `si` in `reprex()`: + +```{r} +#| eval: false +reprex::reprex({ + library(palaeoverse) + data(reefs) + reefs2 <- head(reefs, n = 1) + reefs2 <- reefs2[, 1:6] + + reefs2 <- look_up(occdf = reefs2, + early_interval = "interval", + late_interval = "interval", + int_key = interval_key) + bins <- time_bins(interval = "Phanerozoic", rank = "stage") + + bin_time(occdf = reefs2, bins = bins, method = "mid") +}, si = TRUE) +``` + +You now know how to use `reprex()`! Note that the steps before actually using it are key: very often, other people will ask you to minimize your issue as much as possible before reporting it. In many cases, you can also find the issue on your own just by reducing it. + + +# Debugging a function + +Debugging a function that you wrote (either in a package or as part of a project) can be annoying because an entire (internal) block of code gets executed and you only see the final error. + +One reflex is to put various `print()` statement in the function body to see which ones get evaluated (meaning that the code ran fine so far) and which ones didn't because the function errored before. + +However, a more robust and convenient approach is to use a function that will stop the evaluation and put us in the evaluating environment: `browser()`. This allows us to run line by line *in the function body*, and it is therefore a key tool when you start writing your own functions. + +We will use [issue #280](github.com/palaeoverse/palaeoverse/issues/280) in `palaeoverse` as an example throughout this section. + +## `browser()` + +The bug report above gives a reprex to reproduce the error (note that at the time you are reading this, this bug may have been fixed): + +```{r} +#| error = TRUE +library(palaeoverse) + +occdf <- reefs[1:5, ] +bin_space(occdf = occdf, spacing = 1000, sub_grid = 250, return = TRUE) +``` + +The error happens in `bin_space()`, which is a long function: + +``` r +palaeoverse::bin_space +#> function (occdf, lng = "lng", lat = "lat", spacing = 100, sub_grid = NULL, +#> return = FALSE, plot = FALSE) +#> { +#> if (!is.data.frame(occdf)) { +#> stop("occdf should be of class dataframe") +#> } +#> +#> [ MANUALLY TRUNCATED ] +#> +#> } +``` + +One approach to narrow down the cause of the error would be to add multiple `print()` statements, so that we can check which messages are printed (indicating that the code ran without error) and which are not because the error occurred before reaching them. + +However, it would be way more convenient to be able to execute the internal code line by line so that we can see precisely which line errors. This is the objective of `browser()`. We can start by putting a `browser()` at the top, re-load the function, and execute it again: + +``` diff +function (occdf, lng = "lng", lat = "lat", spacing = 100, sub_grid = NULL, + return = FALSE, plot = FALSE) +{ ++ browser() + if (!is.data.frame(occdf)) { + stop("occdf should be of class dataframe") + } + + [ MANUALLY TRUNCATED ] + +} +``` + +In the video below, running the code with the function that now includes `browser()` doesn't error. Instead, it stops the execution at the `browser()` call: + +![](browser_1.webm) + +There are two interesting things in this video: + +1. we have access to the function environment, including the user-provided inputs. Note for instance that `sub_grid` is `NULL` by default in the function definition, but printing `sub_grid` in the console gives `250` because the user provided `sub_grid = 250`. + +2. We can execute the remaining function body statement by statement by pressing "Enter" in the console. Note that this goes doesn't necessarily execute *all* the code: an `if()` statement whose condition is `FALSE` will not be executed. `browser()` follows exactly the path taken by our code. + +Pressing "Esc" or executing `Q` allows us to exit this "browsing" mode. + +Using this approach, we can find the exact source of the error: + +![](browser_2.webm) + +In our case, the error comes from the line: + +```r +occdf <- list(occdf, grid, base_grid, primary, secondary) +``` + +Fixing this requires reading all the code prior to this line, but this is out of scope for this blog post. + +If you are confident that some lines are irrelevant to the bug, you can move the `browser()` call past them so that you don't need to execute them all every time you enter "browsing" mode. + + +## Breakpoints + +Breakpoints are a feature directly integrated in the software used to run R (e.g. RStudio or Positron. This is also known as IDE for "Integrated Development Environment"). + +They are a way to mimic `browser()` calls without actually modifying the function. You can add one by clicking on the left of line numbers: + +![](breakpoints.webm)