[Rcpp-devel] Linking problem when wrapping a armadillo-based C++ library in a R package using RcppArmadillo

[julien chiquet]

Hi,

I am trying to wrap the optim library (https://github.com/kthohr/optim) in
R via an R package using Rcpp/RcppArmadillo. Indeed, optim relies on
armadillo itself. The github repository can be found here
https://github.com/jchiquet/optimLibR

To do that, I put the headers of optim into inst/include/ and the cpp files
into src/. Then I tried to "heuristically" pull everything together in the
src/Makevars files, after my (obviously poor) understanding of the R manual
pages about Makevars and from the various examples that I copied from the
web (especially from the RcppMLPACK1 which, relies on the C++ library
MLPACK, itself based on armadillo
https://github.com/rcppmlpack/RcppMLPACK1/blob/master/src/Makevars).

The package installation (including compilation of optim) successes (at
least, the package can be installed). Then I tried to create two small Rcpp
functions for two examples available in the original optim library,
starting with the following lines

// [[Rcpp::depends(RcppArmadillo)]]// [[Rcpp::plugins(cpp11)]]
#define USE_RCPP_ARMADILLO
#include "optim.hpp"

(The #define command is an option of the optim library.)

However, when I call the corresponding R functions, I get the following
Armadillo errors in R which I do not have when running them in 'pure' C++
(I am pretty sure it is not a syntax problem, the Rcpp code and the
original C++ code are hopefully identical, except for the headers).

library(optimLibR)> ackley_function()error: arma::memory::acquire(): out of memory
library(optimLibR)> logit_optimLib()error: Mat::init(): requested size is not compatible with column vector layout

So it seems to me that the header of armadillo are correctly defined but I
am not sure that I reach the src (cpp) part of armadillo which is required
at the execution.

If anyone can confirm or help pointing the origin of the error, I would be
debtful...

Thanks for your time,

J.

Thanks for your time

[Dirk Eddelbuettel]

On 25 July 2018 at 17:30, julien chiquet wrote:
| Hi,
|
| I am trying to wrap the optim library (https://github.com/kthohr/optim) in
| R via an R package using Rcpp/RcppArmadillo. Indeed, optim relies on
| armadillo itself. The github repository can be found here
| https://github.com/jchiquet/optimLibR
|
| To do that, I put the headers of optim into inst/include/ and the cpp files
| into src/. Then I tried to "heuristically" pull everything together in the
| src/Makevars files, after my (obviously poor) understanding of the R manual
| pages about Makevars and from the various examples that I copied from the
| web (especially from the RcppMLPACK1 which, relies on the C++ library
| MLPACK, itself based on armadillo
| https://github.com/rcppmlpack/RcppMLPACK1/blob/master/src/Makevars).
|
| The package installation (including compilation of optim) successes (at
| least, the package can be installed). Then I tried to create two small Rcpp
| functions for two examples available in the original optim library,
| starting with the following lines
|
| // [[Rcpp::depends(RcppArmadillo)]]// [[Rcpp::plugins(cpp11)]]
| #define USE_RCPP_ARMADILLO
| #include "optim.hpp"
|
| (The #define command is an option of the optim library.)
|
| However, when I call the corresponding R functions, I get the following
| Armadillo errors in R which I do not have when running them in 'pure' C++
| (I am pretty sure it is not a syntax problem, the Rcpp code and the
| original C++ code are hopefully identical, except for the headers).
|
| > library(optimLibR)> ackley_function()error: arma::memory::acquire(): out of memory
|
| > library(optimLibR)> logit_optimLib()error: Mat::init(): requested size is not compatible with column vector layout
|
| So it seems to me that the header of armadillo are correctly defined but I
| am not sure that I reach the src (cpp) part of armadillo which is required
| at the execution.
|
| If anyone can confirm or help pointing the origin of the error, I would be
| debtful...
|
| Thanks for your time,

Good news / bad news. I think the good news is that everything builds, so you
have no linking issue. In fact, no RcppArmadillo package links to Armadillo
as we do everything via the headers.

But you seem to have run-time issues which _could_ stem from the fact that
RcppArmadillo sets a bunch of #define variables and your use may be slightly
different / conflict with these. Maybe you can check with Keith about how
his optim(lib) could / should be called from R.

Dirk

[julien chiquet]

Post by Dirk Eddelbuettel
| Hi,
|
| I am trying to wrap the optim library (https://github.com/kthohr/optim) in
| R via an R package using Rcpp/RcppArmadillo. Indeed, optim relies on
| armadillo itself. The github repository can be found here
| https://github.com/jchiquet/optimLibR
|
| To do that, I put the headers of optim into inst/include/ and the cpp files
| into src/. Then I tried to "heuristically" pull everything together in the
| src/Makevars files, after my (obviously poor) understanding of the R manual
| pages about Makevars and from the various examples that I copied from the
| web (especially from the RcppMLPACK1 which, relies on the C++ library
| MLPACK, itself based on armadillo
| https://github.com/rcppmlpack/RcppMLPACK1/blob/master/src/Makevars).
|
| The package installation (including compilation of optim) successes (at
| least, the package can be installed). Then I tried to create two small Rcpp
| functions for two examples available in the original optim library,
| starting with the following lines
|
| // [[Rcpp::depends(RcppArmadillo)]]// [[Rcpp::plugins(cpp11)]]
| #define USE_RCPP_ARMADILLO
| #include "optim.hpp"
|
| (The #define command is an option of the optim library.)
|
| However, when I call the corresponding R functions, I get the following
| Armadillo errors in R which I do not have when running them in 'pure' C++
| (I am pretty sure it is not a syntax problem, the Rcpp code and the
| original C++ code are hopefully identical, except for the headers).
|
| > library(optimLibR)> ackley_function()error: arma::memory::acquire(): out of memory
|
| > library(optimLibR)> logit_optimLib()error: Mat::init(): requested size
is not compatible with column vector layout
|
| So it seems to me that the header of armadillo are correctly defined but I
| am not sure that I reach the src (cpp) part of armadillo which is required
| at the execution.
|
| If anyone can confirm or help pointing the origin of the error, I would be
| debtful...
|
| Thanks for your time,
Good news / bad news. I think the good news is that everything builds, so you
have no linking issue. In fact, no RcppArmadillo package links to Armadillo
as we do everything via the headers.
But you seem to have run-time issues which _could_ stem from the fact that
RcppArmadillo sets a bunch of #define variables and your use may be slightly
different / conflict with these. Maybe you can check with Keith about how
his optim(lib) could / should be called from R.

Thanks for your prompt and helpful answer. At least I know I do not have to
struggle with the Makevars.

I'll check with Keith.

j

Post by Dirk Eddelbuettel
Dirk
--

[Keith O'Hara]

I’m traveling without my laptop this week. I’ll take a look at the error when I return.

Best,
Keith

Post by Dirk Eddelbuettel
| Hi,
|
| I am trying to wrap the optim library (https://github.com/kthohr/optim) in
| R via an R package using Rcpp/RcppArmadillo. Indeed, optim relies on
| armadillo itself. The github repository can be found here
| https://github.com/jchiquet/optimLibR
|
| To do that, I put the headers of optim into inst/include/ and the cpp files
| into src/. Then I tried to "heuristically" pull everything together in the
| src/Makevars files, after my (obviously poor) understanding of the R manual
| pages about Makevars and from the various examples that I copied from the
| web (especially from the RcppMLPACK1 which, relies on the C++ library
| MLPACK, itself based on armadillo
| https://github.com/rcppmlpack/RcppMLPACK1/blob/master/src/Makevars).
|
| The package installation (including compilation of optim) successes (at
| least, the package can be installed). Then I tried to create two small Rcpp
| functions for two examples available in the original optim library,
| starting with the following lines
|
| // [[Rcpp::depends(RcppArmadillo)]]// [[Rcpp::plugins(cpp11)]]
| #define USE_RCPP_ARMADILLO
| #include "optim.hpp"
|
| (The #define command is an option of the optim library.)
|
| However, when I call the corresponding R functions, I get the following
| Armadillo errors in R which I do not have when running them in 'pure' C++
| (I am pretty sure it is not a syntax problem, the Rcpp code and the
| original C++ code are hopefully identical, except for the headers).
|
| > library(optimLibR)> ackley_function()error: arma::memory::acquire(): out of memory
|
| > library(optimLibR)> logit_optimLib()error: Mat::init(): requested size is not compatible with column vector layout
|
| So it seems to me that the header of armadillo are correctly defined but I
| am not sure that I reach the src (cpp) part of armadillo which is required
| at the execution.
|
| If anyone can confirm or help pointing the origin of the error, I would be
| debtful...
|
| Thanks for your time,
Good news / bad news. I think the good news is that everything builds, so you
have no linking issue. In fact, no RcppArmadillo package links to Armadillo
as we do everything via the headers.
But you seem to have run-time issues which _could_ stem from the fact that
RcppArmadillo sets a bunch of #define variables and your use may be slightly
different / conflict with these. Maybe you can check with Keith about how
his optim(lib) could / should be called from R.

Thanks for your prompt and helpful answer. At least I know I do not have to struggle with the Makevars.
I'll check with Keith.
j

Post by Dirk Eddelbuettel
Dirk
--

_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[julien chiquet]

Great ! And no hurry.

Best,
j



Julien Chiquet
http://julien.cremeriefamily.info
mel: ***@inra.fr
tel: [phone]

Post by Keith O'Hara
I’m traveling without my laptop this week. I’ll take a look at the error
when I return.
Best,
Keith

Post by Dirk Eddelbuettel
| Hi,
|
| I am trying to wrap the optim library (https://github.com/kthohr/optim) in
| R via an R package using Rcpp/RcppArmadillo. Indeed, optim relies on
| armadillo itself. The github repository can be found here
| https://github.com/jchiquet/optimLibR
|
| To do that, I put the headers of optim into inst/include/ and the cpp files
| into src/. Then I tried to "heuristically" pull everything together in the
| src/Makevars files, after my (obviously poor) understanding of the R manual
| pages about Makevars and from the various examples that I copied from the
| web (especially from the RcppMLPACK1 which, relies on the C++ library
| MLPACK, itself based on armadillo
| https://github.com/rcppmlpack/RcppMLPACK1/blob/master/src/Makevars).
|
| The package installation (including compilation of optim) successes (at
| least, the package can be installed). Then I tried to create two small Rcpp
| functions for two examples available in the original optim library,
| starting with the following lines
|
| // [[Rcpp::depends(RcppArmadillo)]]// [[Rcpp::plugins(cpp11)]]
| #define USE_RCPP_ARMADILLO
| #include "optim.hpp"
|
| (The #define command is an option of the optim library.)
|
| However, when I call the corresponding R functions, I get the following
| Armadillo errors in R which I do not have when running them in 'pure' C++
| (I am pretty sure it is not a syntax problem, the Rcpp code and the
| original C++ code are hopefully identical, except for the headers).
|
| > library(optimLibR)> ackley_function()error: arma::memory::acquire(): out of memory
|
| > library(optimLibR)> logit_optimLib()error: Mat::init(): requested
size is not compatible with column vector layout
|
| So it seems to me that the header of armadillo are correctly defined but I
| am not sure that I reach the src (cpp) part of armadillo which is required
| at the execution.
|
| If anyone can confirm or help pointing the origin of the error, I would be
| debtful...
|
| Thanks for your time,
Good news / bad news. I think the good news is that everything builds, so you
have no linking issue. In fact, no RcppArmadillo package links to Armadillo
as we do everything via the headers.
But you seem to have run-time issues which _could_ stem from the fact that
RcppArmadillo sets a bunch of #define variables and your use may be slightly
different / conflict with these. Maybe you can check with Keith about how
his optim(lib) could / should be called from R.

Thanks for your prompt and helpful answer. At least I know I do not have
to struggle with the Makevars.
I'll check with Keith.
j

Post by Dirk Eddelbuettel
Dirk
--

_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Dirk Eddelbuettel]

On 25 July 2018 at 21:12, julien chiquet wrote:
| Great ! And no hurry.

Cool. I'd love to see how this pans out.

In the meantime, nloptr (at least the GitHub version) can be used well enough
from R/Rcpp -- see eg https://github.com/jyypma/nloptr/issues/39 --- now I
realize that the person I helped there is ... you. Small world.

So can you detail why the working example from nloptr does not suffice for
you? Because Jelmer has still not released that nloptr version to CRAN? Or
is there another reason?

Dirk

[julien chiquet]

Post by Dirk Eddelbuettel
So can you detail why the working example from nloptr does not suffice for
you? Because Jelmer has still not released that nloptr version to CRAN?
Or
is there another reason?

Ok let's try an answer... there are several reasons.

As a researcher in (computational) statistics, I need to get access to
efficient, scalable optimization methods to fit some new statistical
models. It seems natural to me to try an check any new fancy solution
provided by the community, like the one offered by Keith. Moreover, I also
often need to tune a bit the optimization algorithms at play for my
particular problems: the solution offered by Keith, which uses armadillo,
is great because it makes writing C++ code (and especially algebra) so easy
even for a statistician. I also always like to check if some particular
implementation is more robust or faster than another.

You could answer me that nloptr also offers an interface to armadillo with
the solution that you provided : that is right, and thank you again for
that ! However, there is an optimization method from the orignal NLOPT
library (called CCSA) working well for a particular problem that I have
(see the git repo https://github.com/jchiquet/PLNmodels) which is curiously
not exported in nloptr. I thus chose to develop this particular R package
by asking the user to install a system nlopt during the install process on
R via a config file (eg. via message asking for apt-get install
libnlopt-dev for Linux or brew install for Mac OS and... sorry for Windows
users for now). Then I can link directly to the system nlopt to get access
to the headers and to the CCSA algorithm. I also like this solution because
it bypasses R during the optimisation process (calling many times objective
and gradient functions) and thus avoids overheads implied by calls to R
functions. But in that case the time saved is not so impressive, so if
Jelmer finally put an updated version with both export to C++ headers and
CCSA algorithm, I would go back to nloptr R to help the portability
(especially for Windows users). CRAN or github is not and problem to me as
long as the R package and the associated stat method are at a "research"
stage.

And a final reason: I also wanted to learn how to properly wrap a C++
library in a R package, an I thought that the optim package from Keith was
a good opportunity for this, that could eventually help the community a bit.

Best,

j

Post by Dirk Eddelbuettel
Dirk
--

[Dirk Eddelbuettel]

On 26 July 2018 at 17:06, julien chiquet wrote:
| Ok let's try an answer... there are several reasons.

Awesome -- they are all perfectly good ones! I just wanted to make sure you
were not hung up on a technicality.

Dirk