Discussion:
[Rcpp-devel] advice on sharing C++ code across R and Python
Simon Barthelmé
2018-04-27 07:43:41 UTC
Permalink
Dear list,

I have an R package that's mostly C++ code (using Eigen, but no other
dependencies) and some glue. We wish to re-use the C++ code for a Python
package. Most of the external-facing C++ functions are pretty simple:
essentially they're all variants of:

NumericVector some_matrix_calculation(NumericMatrix a_matrix);

and all the stuff coming in or out are dense vectors and matrices. Note
that I wrote all this with the idea that the C++ would only be called
from R, so the C++ bits don't amount to a proper library.

I haven't used Python in years but from what I read there's about a
dozen different ways now of interfacing Python and C/C++. Obviously none
of them know about Rcpp objects like NumericVector so I'm guessing it'll
have to be std::vector<double> in and out, unless there's a better way?

So: does anyone have any experience or advice on sharing C++ code
between R and Python, short of spinning off the C++ as a stand-alone
library and writing entirely separate interfaces?

Also, I know you can call R from Python and vice-versa, but we don't
want to require Python users to install R+dependencies and vice-versa.

Many thanks for any advice!

Best regards

Simon Barthelme
David Bellot
2018-04-27 08:01:20 UTC
Permalink
In my experience, it's usually much easier to have a stand-alone C++
library and add the Rcpp and Boost.Python layer on top of it. As you said,
NumericVector and other classes are not known in the Python world.

You can always hook in into them from Boost.Python to expose them to Python
but a clear and simple design is to have an independent C++ library and
then you wrap it into all the languages you want.
I found Boost.Python quite similar to Rcpp (I think the second one got
inspiration from the first one ?). So it shouldn't be too hard...

Cheers,
David

On Fri, Apr 27, 2018 at 8:43 AM, Simon Barthelmé <
Post by Simon Barthelmé
Dear list,
I have an R package that's mostly C++ code (using Eigen, but no other
dependencies) and some glue. We wish to re-use the C++ code for a Python
NumericVector some_matrix_calculation(NumericMatrix a_matrix);
and all the stuff coming in or out are dense vectors and matrices. Note
that I wrote all this with the idea that the C++ would only be called from
R, so the C++ bits don't amount to a proper library.
I haven't used Python in years but from what I read there's about a dozen
different ways now of interfacing Python and C/C++. Obviously none of them
know about Rcpp objects like NumericVector so I'm guessing it'll have to be
std::vector<double> in and out, unless there's a better way?
So: does anyone have any experience or advice on sharing C++ code between
R and Python, short of spinning off the C++ as a stand-alone library and
writing entirely separate interfaces?
Also, I know you can call R from Python and vice-versa, but we don't want
to require Python users to install R+dependencies and vice-versa.
Many thanks for any advice!
Best regards
Simon Barthelme
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Dirk Eddelbuettel
2018-04-27 12:08:19 UTC
Permalink
Some folks told me that pybind11, ie at

https://github.com/pybind/pybind11

is the one to use as Boost Python is stagnant. I have no personal experience
with pybind11 though.

The question is a good. This (old) CRAN package has been doing both R and
Python from a joint C++ basis for years:

https://cran.r-project.org/package=fastcluster

but I also never looked in real detail at what it does.

Dirk
--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org
Kevin Thornton
2018-04-27 18:05:54 UTC
Permalink
Hi all,

pybind11 is the best choice here. It goes quite a bit beyond what Boost
Python did, including nice support for Numpy arrays, the buffer protocol
more generally, and an Eigen interface. I've used it quite a bit.

But the real issue will be separating out the guts of your code from the
Rcpp/pybind11-specific stuff, as others have mentioned.

--Kevin
Post by Dirk Eddelbuettel
Some folks told me that pybind11, ie at
https://github.com/pybind/pybind11
is the one to use as Boost Python is stagnant. I have no personal experience
with pybind11 though.
The question is a good. This (old) CRAN package has been doing both R and
https://cran.r-project.org/package=fastcluster
but I also never looked in real detail at what it does.
Dirk
--
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
--
Kevin Thornton

Associate Professor

Ecology and Evolutionary Biology

UC Irvine

http://www.molpopgen.org

http://github.com/ThorntonLab

http://github.com/molpopgen
Simon Barthelmé
2018-04-28 11:50:03 UTC
Permalink
Thanks for your advice, everyone, I'll look into pybind11.

Best

Simon
Post by Kevin Thornton
Hi all,
pybind11 is the best choice here.  It goes quite a bit beyond what
Boost Python did, including nice support for Numpy arrays, the buffer
protocol more generally, and an Eigen interface.  I've used it quite a
bit.
But the real issue will be separating out the guts of your code from
the Rcpp/pybind11-specific stuff, as others have mentioned.
--Kevin
Some folks told me that pybind11, ie at
https://github.com/pybind/pybind11
is the one to use as Boost Python is stagnant. I have no personal experience
with pybind11 though.
The question is a good.  This (old) CRAN package has been doing
both R and
https://cran.r-project.org/package=fastcluster
but I also never looked in real detail at what it does.
Dirk
--
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
--
Kevin Thornton
Associate Professor
Ecology and Evolutionary Biology
UC Irvine
http://www.molpopgen.org
http://github.com/ThorntonLab
http://github.com/molpopgen
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Loading...