[Rcpp-devel] `checkUserInterrupt()` might not be safe if Rcout is used.

[Wush Wu]

Hi all,

I just learned the function `checkUserInterrupt` and played with it in my
package today. At first, everything was good. However, I sensed something
wrong when I interrupted my function and relaunched it. In my case, the
thread number of OpenMP decreased to 1 after an user interruption.

According to the documentation, the `checkUserInterrupt` will throw an
exception to trigger the C++ destructors on the stack. However, if I use
`Rcout` several times (verbose mode of my function), then it will not throw
the exception but leave the function directly.

Here is a toy example to demonstrate:
https://gist.github.com/wush978/36c4e5d8324dd14040eecb4b1dd1c631

It uses `Rcout` / `std::cout` as a progressbar and check the user
interruption. If the exception is thrown correctly, then a catch clause
will handle the exception and write something to the `std::cerr`.

If I turn off the verbose mode, then the `checkUserInterrupt` always throws
the exception.

If I turn on the verbose mode and use `jsize` to select the number of dot
on the screen, then the `checkUserInterrupt` will exit the function
immediately if `jsize` is large. For example, `checkUserInterrupt` throws
exception if `jsize = 5` but does not throw if `jsize = 100`.

If I use `std::cout` instead of `Rcpp::Rcout`, then everything goes right
again. So, there might be something wrong even if `R_TopLevelExec` is used.

Well, I guess this bug might be hard to debug, but we should let the others
aware about this at least, right?

Best,
Wush

[Kevin Ushey]

What environment are you executing this on? (R in terminal, or with a GUI;
which OS?)

One thing to be cautious of is that many R APIs in graphical environments
will also call for processing of events, and this in turn can also imply a
check for, and handling of, interrupts. Rprintf() is in fact one such API
(and this is called behind the scenes by Rcout).

This implies something quite unfortunate: attempts to use any R APIs which
call for processing of events within a C++ context can cause a longjmp that
bypass C++ destructors and leave you in a bad state.

Fortunately, things will be better with the newer Rcpp evaluation system
from R 3.5 and above, thanks for R_UnwindProtect() and work from Lionel to
integrate that with the Rcpp evaluation model. See
https://github.com/RcppCore/Rcpp/pull/789 for some of the details on the
initial PR. We're still settling out some final details on the API but once
that's ready we'll have some documentation + hopefully an Rcpp gallery
example describing its use.

(We might also consider in Rcpp wrapping our calls to Rprintf() in
R_ToplevelExec() and 'catching' and 'rethrowing' interrupts seen, but this
might have some unintended side-effects)

Best,
Kevin

Post by Wush Wu
Hi all,
I just learned the function `checkUserInterrupt` and played with it in my
package today. At first, everything was good. However, I sensed something
wrong when I interrupted my function and relaunched it. In my case, the
thread number of OpenMP decreased to 1 after an user interruption.
According to the documentation, the `checkUserInterrupt` will throw an
exception to trigger the C++ destructors on the stack. However, if I use
`Rcout` several times (verbose mode of my function), then it will not throw
the exception but leave the function directly.
https://gist.github.com/wush978/36c4e5d8324dd14040eecb4b1dd1c631
It uses `Rcout` / `std::cout` as a progressbar and check the user
interruption. If the exception is thrown correctly, then a catch clause
will handle the exception and write something to the `std::cerr`.
If I turn off the verbose mode, then the `checkUserInterrupt` always
throws the exception.
If I turn on the verbose mode and use `jsize` to select the number of dot
on the screen, then the `checkUserInterrupt` will exit the function
immediately if `jsize` is large. For example, `checkUserInterrupt` throws
exception if `jsize = 5` but does not throw if `jsize = 100`.
If I use `std::cout` instead of `Rcpp::Rcout`, then everything goes right
again. So, there might be something wrong even if `R_TopLevelExec` is used.
Well, I guess this bug might be hard to debug, but we should let the
others aware about this at least, right?
Best,
Wush
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Wush Wu]

Post by Kevin Ushey
What environment are you executing this on? (R in terminal, or with a GUI;
which OS?)

Here is my session Info:

```
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.4 LTS

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
LC_TIME=zh_TW.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=zh_TW.UTF-8 LC_MESSAGES=en_US.UTF-8
LC_PAPER=zh_TW.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
LC_MEASUREMENT=zh_TW.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] supc_0.2.1

loaded via a namespace (and not attached):
[1] compiler_3.4.4 tools_3.4.4 Rcpp_0.12.17
```

And I have tested it in Rstudio and console. Both of them trigger the same
error.

I have not tested R 3.5. I can give one tomorrow.
Maybe it is worth constructing a unit test.

Post by Kevin Ushey
One thing to be cautious of is that many R APIs in graphical environments
will also call for processing of events, and this in turn can also imply a
check for, and handling of, interrupts. Rprintf() is in fact one such API
(and this is called behind the scenes by Rcout).
This implies something quite unfortunate: attempts to use any R APIs which
call for processing of events within a C++ context can cause a longjmp that
bypass C++ destructors and leave you in a bad state.
Fortunately, things will be better with the newer Rcpp evaluation system
from R 3.5 and above, thanks for R_UnwindProtect() and work from Lionel to
integrate that with the Rcpp evaluation model. See https://github.com/
RcppCore/Rcpp/pull/789 for some of the details on the initial PR. We're
still settling out some final details on the API but once that's ready
we'll have some documentation + hopefully an Rcpp gallery example
describing its use.
(We might also consider in Rcpp wrapping our calls to Rprintf() in
R_ToplevelExec() and 'catching' and 'rethrowing' interrupts seen, but this
might have some unintended side-effects)
Best,
Kevin

Post by Wush Wu
Hi all,
I just learned the function `checkUserInterrupt` and played with it in my
package today. At first, everything was good. However, I sensed something
wrong when I interrupted my function and relaunched it. In my case, the
thread number of OpenMP decreased to 1 after an user interruption.
According to the documentation, the `checkUserInterrupt` will throw an
exception to trigger the C++ destructors on the stack. However, if I use
`Rcout` several times (verbose mode of my function), then it will not throw
the exception but leave the function directly.
Here is a toy example to demonstrate: https://gist.github.com/wush978/
36c4e5d8324dd14040eecb4b1dd1c631
It uses `Rcout` / `std::cout` as a progressbar and check the user
interruption. If the exception is thrown correctly, then a catch clause
will handle the exception and write something to the `std::cerr`.
If I turn off the verbose mode, then the `checkUserInterrupt` always
throws the exception.
If I turn on the verbose mode and use `jsize` to select the number of dot
on the screen, then the `checkUserInterrupt` will exit the function
immediately if `jsize` is large. For example, `checkUserInterrupt` throws
exception if `jsize = 5` but does not throw if `jsize = 100`.
If I use `std::cout` instead of `Rcpp::Rcout`, then everything goes right
again. So, there might be something wrong even if `R_TopLevelExec` is used.
Well, I guess this bug might be hard to debug, but we should let the
others aware about this at least, right?
Best,
Wush
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Wush Wu]

Hi Kevin,

I tried the latest master branch ("522e66d9ea04502b4a98bb28642fb4724ff0a479")
in R 3.5.1, but the `checkUserInterrupt` still does not throw the exception.

The R is under a docker container (library/r-base:3.5.1), so there should
be no GUI.

Hope these info helps.

Best,
Wush

Post by Kevin Ushey
What environment are you executing this on? (R in terminal, or with a GUI;
which OS?)
One thing to be cautious of is that many R APIs in graphical environments
will also call for processing of events, and this in turn can also imply a
check for, and handling of, interrupts. Rprintf() is in fact one such API
(and this is called behind the scenes by Rcout).
This implies something quite unfortunate: attempts to use any R APIs which
call for processing of events within a C++ context can cause a longjmp that
bypass C++ destructors and leave you in a bad state.
Fortunately, things will be better with the newer Rcpp evaluation system
from R 3.5 and above, thanks for R_UnwindProtect() and work from Lionel to
integrate that with the Rcpp evaluation model. See
https://github.com/RcppCore/Rcpp/pull/789 for some of the details on the
initial PR. We're still settling out some final details on the API but once
that's ready we'll have some documentation + hopefully an Rcpp gallery
example describing its use.
(We might also consider in Rcpp wrapping our calls to Rprintf() in
R_ToplevelExec() and 'catching' and 'rethrowing' interrupts seen, but this
might have some unintended side-effects)
Best,
Kevin

Post by Wush Wu
Hi all,
I just learned the function `checkUserInterrupt` and played with it in my
package today. At first, everything was good. However, I sensed something
wrong when I interrupted my function and relaunched it. In my case, the
thread number of OpenMP decreased to 1 after an user interruption.
According to the documentation, the `checkUserInterrupt` will throw an
exception to trigger the C++ destructors on the stack. However, if I use
`Rcout` several times (verbose mode of my function), then it will not throw
the exception but leave the function directly.
Here is a toy example to demonstrate: https://gist.github.com/wush97
8/36c4e5d8324dd14040eecb4b1dd1c631
It uses `Rcout` / `std::cout` as a progressbar and check the user
interruption. If the exception is thrown correctly, then a catch clause
will handle the exception and write something to the `std::cerr`.
If I turn off the verbose mode, then the `checkUserInterrupt` always
throws the exception.
If I turn on the verbose mode and use `jsize` to select the number of dot
on the screen, then the `checkUserInterrupt` will exit the function
immediately if `jsize` is large. For example, `checkUserInterrupt` throws
exception if `jsize = 5` but does not throw if `jsize = 100`.
If I use `std::cout` instead of `Rcpp::Rcout`, then everything goes right
again. So, there might be something wrong even if `R_TopLevelExec` is used.
Well, I guess this bug might be hard to debug, but we should let the
others aware about this at least, right?
Best,
Wush
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel