[Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

Discussion:

Helske, Jouni

2017-09-04 08:22:04 UTC

Hi,

In the bssm package (https://github.com/helske/bssm), I let the users define a state space model via own C++ functions which are then exported as Xptr to R side, and passed back to C++ function for SMC/MCMC etc. Normally this works fine, but when user repeatedly modifies the model functions and re-exports them via repeated calls sourceCpp, sometimes whole Rstudio crashes. I haven't been able to pinpoint the reason or minimal example. Have anyone else experienced similar issues or is it just me i.e. error in my own implementations?

Simplest case of this is the discretized SDE models, this is the function which user modifies: https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp, <https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp> and the relevant parts of the package codes are in src/function_pointers.h, src/sde_ssm.cpp and src/R_sde.cpp (I don't really expect anyone to read through all of these).

Best,

Jouni Helske

Iñaki Úcar

2017-09-04 09:43:31 UTC

Permalink

Hi,

2017-09-04 10:22 GMT+02:00 Helske, Jouni <***@jyu.fi>:
> Hi,
>
>
> In the bssm package (https://github.com/helske/bssm), I let the users define
> a state space model via own C++ functions which are then exported as Xptr to
> R side, and passed back to C++ function for SMC/MCMC etc. Normally this
> works fine, but when user repeatedly modifies the model functions and
> re-exports them via repeated calls sourceCpp, sometimes whole Rstudio
> crashes. I haven't been able to pinpoint the reason or minimal example. Have
> anyone else experienced similar issues or is it just me i.e. error in my own
> implementations?

You can try the same by running R from the command line. If this
doesn't crash, the problem is in RStudio. You can also run RStudio
with the "-debug" flag or using valgrind to try to delimit the issue.

> Simplest case of this is the discretized SDE models, this is the function
> which user modifies:
> https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp, and
> the relevant parts of the package codes are in src/function_pointers.h,
> src/sde_ssm.cpp and src/R_sde.cpp (I don't really expect anyone to read
> through all of these).

Let me suggest you the RcppXPtrUtils package (on CRAN) for exporting
XPtrs without adding all the repetitive scaffolding:
http://gallery.rcpp.org//articles/passing-cpp-function-pointers-rcppxptrutils/

Regards,
Iñaki

Simon Barthelmé

2017-09-04 10:14:52 UTC

Permalink

Hi Helske

I've had this happen to me and if memory serves in my case it was due to
a memory leak on the C++ side that didn't bother R too much but caused
sourceCpp to crash. Maybe try valgrind?

Simon

On 04/09/2017 10:22, Helske, Jouni wrote:
>
> Hi,
>
>
> In the bssm package (https://github.com/helske/bssm), I let the users
> define a state space model via own C++ functions which are then
> exported as Xptr to R side, and passed back to C++ function for
> SMC/MCMC etc. Normally this works fine, but when user repeatedly
> modifies the model functions and re-exports them via repeated calls
> sourceCpp, sometimes whole Rstudio crashes. I haven't been able to
> pinpoint the reason or minimal example. Have anyone else experienced
> similar issues or is it just me i.e. error in my own implementations?
>
>
> Simplest case of this is the discretized SDE models, this is the
> function which user modifies:
> https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp, <https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>and
> the relevant parts of the package codes are in
> src/function_pointers.h, src/sde_ssm.cpp and src/R_sde.cpp (I don't
> really expect anyone to read through all of these).
>
>
> Best,
>
>
> Jouni Helske
>
>
>
>
> _______________________________________________
> Rcpp-devel mailing list
> Rcpp-***@lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

Helske, Jouni

2017-09-04 11:07:14 UTC

Permalink

Hi,

I already tried valgrind, that doesn't give any hints. So far I haven't been able to reproduce this in R console, but could be just bad(?) luck as the whole crash happens so randomly. I am now running sourceCpp with rebuild = TRUE and subsequent function calls on a loop, and it takes seemingly random amount of iterations until Rstudio crashes.

Jouni

________________________________
Lähettäjä: Simon Barthelmé <***@gipsa-lab.fr>
Lähetetty: maanantai 4. syyskuuta 2017 13.14.52
Vastaanottaja: Helske, Jouni; Rcpp-***@lists.r-forge.r-project.org
Aihe: Re: [Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

Hi Helske

I've had this happen to me and if memory serves in my case it was due to a memory leak on the C++ side that didn't bother R too much but caused sourceCpp to crash. Maybe try valgrind?

Simon

On 04/09/2017 10:22, Helske, Jouni wrote:

Hi,

In the bssm package (https://github.com/helske/bssm), I let the users define a state space model via own C++ functions which are then exported as Xptr to R side, and passed back to C++ function for SMC/MCMC etc. Normally this works fine, but when user repeatedly modifies the model functions and re-exports them via repeated calls sourceCpp, sometimes whole Rstudio crashes. I haven't been able to pinpoint the reason or minimal example. Have anyone else experienced similar issues or is it just me i.e. error in my own implementations?

Simplest case of this is the discretized SDE models, this is the function which user modifies: https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp, <https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp> and the relevant parts of the package codes are in src/function_pointers.h, src/sde_ssm.cpp and src/R_sde.cpp (I don't really expect anyone to read through all of these).

Best,

Jouni Helske

Dirk Eddelbuettel

2017-09-04 12:32:13 UTC

Permalink

Hi Jouni,

On 4 September 2017 at 08:22, Helske, Jouni wrote:
| In the bssm package (https://github.com/helske/bssm), I let the users
| define a state space model via own C++ functions which are then exported as
| Xptr to R side, and passed back to C++ function for SMC/MCMC etc. Normally
| this works fine, but when user repeatedly modifies the model functions and
| re-exports them via repeated calls sourceCpp, sometimes whole Rstudio
| crashes.

Which version of RStudio? See Help -> About

| I haven't been able to pinpoint the reason or minimal example. Have anyone else experienced similar issues or is it just me i.e. error in my own implementations?

There are a lot of dependencies here [re-indented by me]:

Suggests: KFAS (>= 1.2.1), knitr (>= 1.11), rmarkdown (>= 0.8.1), testthat,
bayesplot
Imports: coda (>= 0.18-1), diagis, ggplot2 (>= 2.0.0), Rcpp (>= 0.12.3)
LinkingTo: BH, Rcpp, RcppArmadillo, ramcmc, sitmo

We do know Boost interactions (between code compiled 'by us' as R extensions
using Boost, and RStudio itself also using an older version of Boost) had
thrown spanners in the past. A while back Kevin updated RStudio internals and
shielded "their" Boost better.

I happen to run RStudio "Daily" builds on all my machines -- those have the
fix and "behave". Maybe worth a try for you.

| Simplest case of this is the discretized SDE models, this is the function
| which user modifies:
| https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp,
| <https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
| and the relevant parts of the package codes are in src/function_pointers.h,
| src/sde_ssm.cpp and src/R_sde.cpp (I don't really expect anyone to read
| through all of these).

We'd probably need a reproducible example to really help.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org

Helske, Jouni

2017-09-05 09:31:15 UTC

Permalink

I was finally able to get reproducible, although likely not minimal segfault even without Rstudio. I built a new package and running a weird loop with rebuilding of the source files and doing some other random stuff as well, I was able to get a segfault. valgrind output looks like this:

==26398== Jump to the invalid address stated on the next line
==26398== at 0x1DD75010: ???
==26398== by 0x4F7D026: R_RunWeakRefFinalizer (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F7D213: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4A679: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4EA11: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4AB99: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4D723: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4AA9F: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4CABE: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F42F67: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F50D55: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F512FF: ??? (in /usr/lib/R/lib/libR.so)
==26398== Address 0x1dd75010 is not stack'd, malloc'd or (recently) free'd
==26398==

*** caught segfault ***
address 0x1dd75010, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
==26398==
==26398== Process terminating with default action of signal 11 (SIGSEGV)
==26398== at 0x547451D: raise (raise.c:53)
==26398== by 0x547466F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.24.so)
==26398== by 0x1DD7500F: ???
==26398==
==26398== HEAP SUMMARY:
==26398== in use at exit: 259,162,338 bytes in 28,465 blocks
==26398== total heap usage: 315,728 allocs, 287,263 frees, 1,311,929,706 bytes allocated
==26398==
==26398== LEAK SUMMARY:
==26398== definitely lost: 0 bytes in 0 blocks
==26398== indirectly lost: 0 bytes in 0 blocks
==26398== possibly lost: 0 bytes in 0 blocks
==26398== still reachable: 259,162,338 bytes in 28,465 blocks
==26398== of which reachable via heuristic:
==26398== newarray : 4,264 bytes in 1 blocks
==26398== suppressed: 0 bytes in 0 blocks
==26398== Reachable blocks (those to which a pointer was found) are not shown.
==26398== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==26398==
==26398== For counts of detected and suppressed errors, rerun with: -v
==26398== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

What is super weird is that currently I have this extra C++ function which is not even used at all:

double log_prior_pdf(const arma::vec& theta) {

double log_pdf = 0.0;
double infinite = -arma::datum::inf; //comment out this line and everything works??
return log_pdf;
}

If I remove the middle line, I seem to get rid of the segfault. However, I also have some extra stuff on the R script (crash.R) as well which looks to be necessary for the segfault to trigger at certain point (third iteration of the outer loop).

Jouni

________________________________
Lähettäjä: Dirk Eddelbuettel <***@gmail.com> käyttäjän Dirk Eddelbuettel <***@debian.org> puolesta
Lähetetty: maanantai 4. syyskuuta 2017 15.32.13
Vastaanottaja: Helske, Jouni
Kopio: Rcpp-***@lists.r-forge.r-project.org
Aihe: Re: [Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

Hi Jouni,

On 4 September 2017 at 08:22, Helske, Jouni wrote:
| In the bssm package (https://github.com/helske/bssm), I let the users
| define a state space model via own C++ functions which are then exported as
| Xptr to R side, and passed back to C++ function for SMC/MCMC etc. Normally
| this works fine, but when user repeatedly modifies the model functions and
| re-exports them via repeated calls sourceCpp, sometimes whole Rstudio
| crashes.

Which version of RStudio? See Help -> About

| I haven't been able to pinpoint the reason or minimal example. Have anyone else experienced similar issues or is it just me i.e. error in my own implementations?

There are a lot of dependencies here [re-indented by me]:

Suggests: KFAS (>= 1.2.1), knitr (>= 1.11), rmarkdown (>= 0.8.1), testthat,
bayesplot
Imports: coda (>= 0.18-1), diagis, ggplot2 (>= 2.0.0), Rcpp (>= 0.12.3)
LinkingTo: BH, Rcpp, RcppArmadillo, ramcmc, sitmo

We do know Boost interactions (between code compiled 'by us' as R extensions
using Boost, and RStudio itself also using an older version of Boost) had
thrown spanners in the past. A while back Kevin updated RStudio internals and
shielded "their" Boost better.

I happen to run RStudio "Daily" builds on all my machines -- those have the
fix and "behave". Maybe worth a try for you.

| Simplest case of this is the discretized SDE models, this is the function
| which user modifies:
| https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp,
[https://avatars2.githubusercontent.com/u/1560448?v=4&s=400]<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>

bssm/sde_functions.cpp at master · helske/bssm · GitHub<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
github.com
bssm - Bayesian Inference of State Space Models

| <https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
[https://avatars2.githubusercontent.com/u/1560448?v=4&s=400]<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>

bssm/sde_functions.cpp at master · helske/bssm · GitHub<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
github.com
bssm - Bayesian Inference of State Space Models

| and the relevant parts of the package codes are in src/function_pointers.h,
| src/sde_ssm.cpp and src/R_sde.cpp (I don't really expect anyone to read
| through all of these).

We'd probably need a reproducible example to really help.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org

Helske, Jouni

2017-09-05 09:32:05 UTC

Permalink

Oops, forgot the link to the repo: https://github.com/helske/crashtest/issues/1

Best,

Jouni

________________________________
Lähettäjä: Helske, Jouni
Lähetetty: tiistai 5. syyskuuta 2017 12.31.15
Vastaanottaja: Dirk Eddelbuettel
Kopio: Rcpp-***@lists.r-forge.r-project.org
Aihe: VS: [Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

I was finally able to get reproducible, although likely not minimal segfault even without Rstudio. I built a new package and running a weird loop with rebuilding of the source files and doing some other random stuff as well, I was able to get a segfault. valgrind output looks like this:

==26398== Jump to the invalid address stated on the next line
==26398== at 0x1DD75010: ???
==26398== by 0x4F7D026: R_RunWeakRefFinalizer (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F7D213: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4A679: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4EA11: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4AB99: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4D723: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4AA9F: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4CABE: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F42F67: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F50D55: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F512FF: ??? (in /usr/lib/R/lib/libR.so)
==26398== Address 0x1dd75010 is not stack'd, malloc'd or (recently) free'd
==26398==

*** caught segfault ***
address 0x1dd75010, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
==26398==
==26398== Process terminating with default action of signal 11 (SIGSEGV)
==26398== at 0x547451D: raise (raise.c:53)
==26398== by 0x547466F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.24.so)
==26398== by 0x1DD7500F: ???
==26398==
==26398== HEAP SUMMARY:
==26398== in use at exit: 259,162,338 bytes in 28,465 blocks
==26398== total heap usage: 315,728 allocs, 287,263 frees, 1,311,929,706 bytes allocated
==26398==
==26398== LEAK SUMMARY:
==26398== definitely lost: 0 bytes in 0 blocks
==26398== indirectly lost: 0 bytes in 0 blocks
==26398== possibly lost: 0 bytes in 0 blocks
==26398== still reachable: 259,162,338 bytes in 28,465 blocks
==26398== of which reachable via heuristic:
==26398== newarray : 4,264 bytes in 1 blocks
==26398== suppressed: 0 bytes in 0 blocks
==26398== Reachable blocks (those to which a pointer was found) are not shown.
==26398== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==26398==
==26398== For counts of detected and suppressed errors, rerun with: -v
==26398== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

What is super weird is that currently I have this extra C++ function which is not even used at all:

double log_prior_pdf(const arma::vec& theta) {

double log_pdf = 0.0;
double infinite = -arma::datum::inf; //comment out this line and everything works??
return log_pdf;
}

If I remove the middle line, I seem to get rid of the segfault. However, I also have some extra stuff on the R script (crash.R) as well which looks to be necessary for the segfault to trigger at certain point (third iteration of the outer loop).

Jouni

________________________________
Lähettäjä: Dirk Eddelbuettel <***@gmail.com> käyttäjän Dirk Eddelbuettel <***@debian.org> puolesta
Lähetetty: maanantai 4. syyskuuta 2017 15.32.13
Vastaanottaja: Helske, Jouni
Kopio: Rcpp-***@lists.r-forge.r-project.org
Aihe: Re: [Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

Hi Jouni,

On 4 September 2017 at 08:22, Helske, Jouni wrote:
| In the bssm package (https://github.com/helske/bssm), I let the users
| define a state space model via own C++ functions which are then exported as
| Xptr to R side, and passed back to C++ function for SMC/MCMC etc. Normally
| this works fine, but when user repeatedly modifies the model functions and
| re-exports them via repeated calls sourceCpp, sometimes whole Rstudio
| crashes.

Which version of RStudio? See Help -> About

| I haven't been able to pinpoint the reason or minimal example. Have anyone else experienced similar issues or is it just me i.e. error in my own implementations?

There are a lot of dependencies here [re-indented by me]:

Suggests: KFAS (>= 1.2.1), knitr (>= 1.11), rmarkdown (>= 0.8.1), testthat,
bayesplot
Imports: coda (>= 0.18-1), diagis, ggplot2 (>= 2.0.0), Rcpp (>= 0.12.3)
LinkingTo: BH, Rcpp, RcppArmadillo, ramcmc, sitmo

We do know Boost interactions (between code compiled 'by us' as R extensions
using Boost, and RStudio itself also using an older version of Boost) had
thrown spanners in the past. A while back Kevin updated RStudio internals and
shielded "their" Boost better.

I happen to run RStudio "Daily" builds on all my machines -- those have the
fix and "behave". Maybe worth a try for you.

| Simplest case of this is the discretized SDE models, this is the function
| which user modifies:
| https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp,
[https://avatars2.githubusercontent.com/u/1560448?v=4&s=400]<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>

bssm/sde_functions.cpp at master · helske/bssm · GitHub<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
github.com
bssm - Bayesian Inference of State Space Models

| <https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
[https://avatars2.githubusercontent.com/u/1560448?v=4&s=400]<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>

bssm/sde_functions.cpp at master · helske/bssm · GitHub<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
github.com
bssm - Bayesian Inference of State Space Models

| and the relevant parts of the package codes are in src/function_pointers.h,
| src/sde_ssm.cpp and src/R_sde.cpp (I don't really expect anyone to read
| through all of these).

We'd probably need a reproducible example to really help.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org

Helske, Jouni

2017-09-05 10:12:57 UTC

Permalink

It looks like the issue has something to do with the cache from the previous sourceCpp calls: If inside of the segfaulting loop (with index i) I set Rcpp::sourceCpp("sde_functions1.cpp", rebuild = TRUE, cacheDir = paste0("dir", i)) everything works fine. Note that by just using cleanupCacheDir = TRUE does not help, which I though would do essentially the same thing.

Best,

Jouni

________________________________
Lähettäjä: rcpp-devel-***@r-forge.wu-wien.ac.at <rcpp-devel-***@r-forge.wu-wien.ac.at> käyttäjän Helske, Jouni <***@jyu.fi> puolesta
Lähetetty: tiistai 5. syyskuuta 2017 12.32.05
Vastaanottaja: Dirk Eddelbuettel
Kopio: Rcpp-***@lists.r-forge.r-project.org
Aihe: Re: [Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

Oops, forgot the link to the repo: https://github.com/helske/crashtest/issues/1

Best,

Jouni

________________________________
Lähettäjä: Helske, Jouni
Lähetetty: tiistai 5. syyskuuta 2017 12.31.15
Vastaanottaja: Dirk Eddelbuettel
Kopio: Rcpp-***@lists.r-forge.r-project.org
Aihe: VS: [Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

I was finally able to get reproducible, although likely not minimal segfault even without Rstudio. I built a new package and running a weird loop with rebuilding of the source files and doing some other random stuff as well, I was able to get a segfault. valgrind output looks like this:

==26398== Jump to the invalid address stated on the next line
==26398== at 0x1DD75010: ???
==26398== by 0x4F7D026: R_RunWeakRefFinalizer (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F7D213: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4A679: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4EA11: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4AB99: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4D723: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4AA9F: Rf_eval (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F4CABE: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F42F67: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F50D55: ??? (in /usr/lib/R/lib/libR.so)
==26398== by 0x4F512FF: ??? (in /usr/lib/R/lib/libR.so)
==26398== Address 0x1dd75010 is not stack'd, malloc'd or (recently) free'd
==26398==

*** caught segfault ***
address 0x1dd75010, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
==26398==
==26398== Process terminating with default action of signal 11 (SIGSEGV)
==26398== at 0x547451D: raise (raise.c:53)
==26398== by 0x547466F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.24.so)
==26398== by 0x1DD7500F: ???
==26398==
==26398== HEAP SUMMARY:
==26398== in use at exit: 259,162,338 bytes in 28,465 blocks
==26398== total heap usage: 315,728 allocs, 287,263 frees, 1,311,929,706 bytes allocated
==26398==
==26398== LEAK SUMMARY:
==26398== definitely lost: 0 bytes in 0 blocks
==26398== indirectly lost: 0 bytes in 0 blocks
==26398== possibly lost: 0 bytes in 0 blocks
==26398== still reachable: 259,162,338 bytes in 28,465 blocks
==26398== of which reachable via heuristic:
==26398== newarray : 4,264 bytes in 1 blocks
==26398== suppressed: 0 bytes in 0 blocks
==26398== Reachable blocks (those to which a pointer was found) are not shown.
==26398== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==26398==
==26398== For counts of detected and suppressed errors, rerun with: -v
==26398== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

What is super weird is that currently I have this extra C++ function which is not even used at all:

double log_prior_pdf(const arma::vec& theta) {

double log_pdf = 0.0;
double infinite = -arma::datum::inf; //comment out this line and everything works??
return log_pdf;
}

If I remove the middle line, I seem to get rid of the segfault. However, I also have some extra stuff on the R script (crash.R) as well which looks to be necessary for the segfault to trigger at certain point (third iteration of the outer loop).

Jouni

________________________________
Lähettäjä: Dirk Eddelbuettel <***@gmail.com> käyttäjän Dirk Eddelbuettel <***@debian.org> puolesta
Lähetetty: maanantai 4. syyskuuta 2017 15.32.13
Vastaanottaja: Helske, Jouni
Kopio: Rcpp-***@lists.r-forge.r-project.org
Aihe: Re: [Rcpp-devel] Rstudio crashes with repeated calls to sourceCpp with Xptr export

Hi Jouni,

On 4 September 2017 at 08:22, Helske, Jouni wrote:
| In the bssm package (https://github.com/helske/bssm), I let the users
| define a state space model via own C++ functions which are then exported as
| Xptr to R side, and passed back to C++ function for SMC/MCMC etc. Normally
| this works fine, but when user repeatedly modifies the model functions and
| re-exports them via repeated calls sourceCpp, sometimes whole Rstudio
| crashes.

Which version of RStudio? See Help -> About

| I haven't been able to pinpoint the reason or minimal example. Have anyone else experienced similar issues or is it just me i.e. error in my own implementations?

There are a lot of dependencies here [re-indented by me]:

Suggests: KFAS (>= 1.2.1), knitr (>= 1.11), rmarkdown (>= 0.8.1), testthat,
bayesplot
Imports: coda (>= 0.18-1), diagis, ggplot2 (>= 2.0.0), Rcpp (>= 0.12.3)
LinkingTo: BH, Rcpp, RcppArmadillo, ramcmc, sitmo

We do know Boost interactions (between code compiled 'by us' as R extensions
using Boost, and RStudio itself also using an older version of Boost) had
thrown spanners in the past. A while back Kevin updated RStudio internals and
shielded "their" Boost better.

I happen to run RStudio "Daily" builds on all my machines -- those have the
fix and "behave". Maybe worth a try for you.

| Simplest case of this is the discretized SDE models, this is the function
| which user modifies:
| https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp,
[https://avatars2.githubusercontent.com/u/1560448?v=4&s=400]<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>

bssm/sde_functions.cpp at master · helske/bssm · GitHub<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
github.com
bssm - Bayesian Inference of State Space Models

| <https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
[https://avatars2.githubusercontent.com/u/1560448?v=4&s=400]<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>

bssm/sde_functions.cpp at master · helske/bssm · GitHub<https://github.com/helske/bssm/blob/master/vignettes/sde_functions.cpp>
github.com
bssm - Bayesian Inference of State Space Models

| and the relevant parts of the package codes are in src/function_pointers.h,
| src/sde_ssm.cpp and src/R_sde.cpp (I don't really expect anyone to read
| through all of these).

We'd probably need a reproducible example to really help.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org

Dirk Eddelbuettel

2017-09-05 10:31:21 UTC

Permalink

On 5 September 2017 at 10:12, Helske, Jouni wrote:
| It looks like the issue has something to do with the cache from the previous sourceCpp calls: If inside of the segfaulting loop (with index i) I set Rcpp::sourceCpp("sde_functions1.cpp", rebuild = TRUE, cacheDir = paste0("dir", i)) everything works fine. Note that by just using cleanupCacheDir = TRUE does not help, which I though would do essentially the same thing.

sourceCpp() does this:

# cleanup the cache dir if requested
if (cleanupCacheDir)
cleanupSourceCppCache(cacheDir, context$cppSourcePath, context$buildDirectory)

I think we should try to focus on which if any files are still left behind by
cleanupSourceCppCache() and also remove them.

This ought to be fixable.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org

Dirk Eddelbuettel

2017-09-05 10:28:27 UTC

Permalink

On 5 September 2017 at 09:32, Helske, Jouni wrote:
| Oops, forgot the link to the repo: https://github.com/helske/crashtest/issues/1

And for those reading here, Jouni and I are now down to looking after
sourceCpp() repeatedly running in the same directory --- this seems to need a
bit a cleanup.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org