[Rcpp-devel] Passing pointers to objects between libs results in weird UBSAN warning

[Jan van der Laan]

One of my packges gives a message with the undefined behaviour sanitiser
(UBSAN). I have difficulty tracking down/understanding the cause of this
message. However, I have been able to create a small example that
generates the same warning. The mail below is a bit on the long side,
and, I hope it is readable as it mixes quite a lot code with text. I am
sorry for that. A probably better readable version can be found at
https://github.com/djvanderlaan/lvec_ubsan where also the complete
source code and the docker image used for testing can be found.

My example consists of two packages: pkg1 and pkg2. The package pkg2
links to pkg1. Package pkg1 defines a template class `FooBar<T>` which
inherits from virtual class `Foo`. Pointers to the these objects are
passed back to R using a `Rcpp:XPtr` object:

========== pkg1/src/construct.cpp ================
#include "../inst/include/pkg1.h"

// [[Rcpp::export]]
RcppExport SEXP new_foobar(int i) {
BEGIN_RCPP
FooBar<int>* x = new FooBar<int>(i);
return Rcpp::XPtr<Foo>(x, true);
END_RCPP
}

// [[Rcpp::export]]
RcppExport SEXP new_foobard(double i) {
BEGIN_RCPP
FooBar<double>* x = new FooBar<double>(i);
return Rcpp::XPtr<Foo>(x, true);
END_RCPP
}
========== END pkg1/src/construct.cpp ================

All headers of pkg1 are `inst/include`; these are included by `pkg2`:

========== pkg1/inst/include/pkg1.h ================
#include <Rcpp.h>

// ==== VISITOR
template<typename T> class FooBar;

class FooBarVisitor {
public:
virtual void visit(FooBar<double>& vec) = 0;
virtual void visit(FooBar<int>& vec) = 0;
};

// ==== FOO
class Foo {
public:
Foo() {};
virtual ~Foo() {};

virtual void visit(FooBarVisitor* visitor) = 0;
};

// ==== FOOBAR
template<typename T>
class FooBar : public Foo {
public:
FooBar(T i = 0) : Foo(), i_(i) {};
~FooBar() {};

T i() const { return i_;}
void i(T i) { i_ = i;}

void visit(FooBarVisitor* visitor) {
visitor->visit(*this);
}
private:
T i_;
};
#endif
========== END pkg1/inst/include/pkg1.h ================

The XPtr is passed to c++ code in pkg2 and methods of the Foo object are
called:

========== pkg2/src/construct.cpp ================
#include <Rcpp.h>
#include <pkg1.h>

// [[Rcpp::export]]
RcppExport int get_foobar(SEXP xp) {
Rcpp::XPtr<Foo> x(xp);
Foo* y1 = x;
FooBar<int>* y2 = dynamic_cast<FooBar<int>*>(y1);
return y2->i();
}
========== END pkg2/src/construct.cpp ================

This generates the following UBSAN messages:


========== WARNING MESSAGES ================
Testing pkg2
<U+221A> | OK F W S | Context
/ | 0 | fooconstruct.cpp:9:21: runtime error: <null> address
0x000003c21810 which does not point to an object of type 'Foo'
0x000003c21810: note: object is of type 'FooBar<int>'
00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
00 00 00 00 00 00 00 01 03 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'FooBar<int>'
construct.cpp:10:14: runtime error: member call on address
0x000003c21810 which does not point to an object of type 'FooBar<int>'
0x000003c21810: note: object is of type 'FooBar<int>'
00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
00 00 00 00 00 00 00 01 03 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'FooBar<int>'
/usr/local/lib/R/library/pkg1/include/pkg1.h:33:26: runtime error:
member access within address 0x000003c21810 which does not point to an
object of type 'const FooBar<int>'
0x000003c21810: note: object is of type 'FooBar<int>'
00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
00 00 00 00 00 00 00 01 03 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'FooBar<int>'
<U+221A> | 1 | foo
========== END WARNING MESSAGES ================


I have had real problem reproducing this error. Also searching for the
error give little clues as to what the problem is. The following message
on the Rcpp-devel list seems somewhat similar:
http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2017-July/009656.html.
I thought that it might be caused by the fact that I am passing pointers
to template classes between libraries. I also couldn't find much about
that. I did find some messages on passing pointers to STL objects
between libraries. That should be ok, as long as the same compiler is
used for both libraries. Although that might technically still make it
undefined behaviour.

So, help!?

Jan

[Iñaki Ucar]

El lun., 20 ago. 2018 a las 9:01, Jan van der Laan

Post by Jan van der Laan
One of my packges gives a message with the undefined behaviour sanitiser
(UBSAN). I have difficulty tracking down/understanding the cause of this
message. However, I have been able to create a small example that
generates the same warning. The mail below is a bit on the long side,
and, I hope it is readable as it mixes quite a lot code with text. I am
sorry for that. A probably better readable version can be found at
https://github.com/djvanderlaan/lvec_ubsan where also the complete
source code and the docker image used for testing can be found.
My example consists of two packages: pkg1 and pkg2. The package pkg2
links to pkg1. Package pkg1 defines a template class `FooBar<T>` which
inherits from virtual class `Foo`. Pointers to the these objects are
========== pkg1/src/construct.cpp ================
#include "../inst/include/pkg1.h"
// [[Rcpp::export]]
RcppExport SEXP new_foobar(int i) {
BEGIN_RCPP
FooBar<int>* x = new FooBar<int>(i);
return Rcpp::XPtr<Foo>(x, true);
END_RCPP
}

Note that RcppExport SEXP should be just SEXP. BEGIN_RCPP and END_RCPP
should be dropped too.

Post by Jan van der Laan
// [[Rcpp::export]]
RcppExport SEXP new_foobard(double i) {
BEGIN_RCPP
FooBar<double>* x = new FooBar<double>(i);
return Rcpp::XPtr<Foo>(x, true);
END_RCPP
}
========== END pkg1/src/construct.cpp ================
========== pkg1/inst/include/pkg1.h ================
#include <Rcpp.h>
// ==== VISITOR
template<typename T> class FooBar;
class FooBarVisitor {
virtual void visit(FooBar<double>& vec) = 0;
virtual void visit(FooBar<int>& vec) = 0;
};
// ==== FOO
class Foo {
Foo() {};
virtual ~Foo() {};
virtual void visit(FooBarVisitor* visitor) = 0;
};
// ==== FOOBAR
template<typename T>
class FooBar : public Foo {
FooBar(T i = 0) : Foo(), i_(i) {};
~FooBar() {};
T i() const { return i_;}
void i(T i) { i_ = i;}
void visit(FooBarVisitor* visitor) {
visitor->visit(*this);
}
T i_;
};
#endif
========== END pkg1/inst/include/pkg1.h ================
The XPtr is passed to c++ code in pkg2 and methods of the Foo object are
========== pkg2/src/construct.cpp ================
#include <Rcpp.h>
#include <pkg1.h>
// [[Rcpp::export]]
RcppExport int get_foobar(SEXP xp) {
Rcpp::XPtr<Foo> x(xp);
Foo* y1 = x;
FooBar<int>* y2 = dynamic_cast<FooBar<int>*>(y1);
return y2->i();
}
========== END pkg2/src/construct.cpp ================
========== WARNING MESSAGES ================
Testing pkg2
<U+221A> | OK F W S | Context
/ | 0 | fooconstruct.cpp:9:21: runtime error: <null> address
0x000003c21810 which does not point to an object of type 'Foo'
0x000003c21810: note: object is of type 'FooBar<int>'
00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
00 00 00 00 00 00 00 01 03 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'FooBar<int>'
construct.cpp:10:14: runtime error: member call on address
0x000003c21810 which does not point to an object of type 'FooBar<int>'
0x000003c21810: note: object is of type 'FooBar<int>'
00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
00 00 00 00 00 00 00 01 03 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'FooBar<int>'
member access within address 0x000003c21810 which does not point to an
object of type 'const FooBar<int>'
0x000003c21810: note: object is of type 'FooBar<int>'
00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
00 00 00 00 00 00 00 01 03 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'FooBar<int>'
<U+221A> | 1 | foo
========== END WARNING MESSAGES ================
I have had real problem reproducing this error. Also searching for the
error give little clues as to what the problem is. The following message
http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2017-July/009656.html.
I thought that it might be caused by the fact that I am passing pointers
to template classes between libraries. I also couldn't find much about
that. I did find some messages on passing pointers to STL objects
between libraries. That should be ok, as long as the same compiler is
used for both libraries. Although that might technically still make it
undefined behaviour.

pkg2 compiles a shared library using pkg1's headers, but it does not
link against libpkg1.so (in general, this is not possible in R). So
both libpkg2.so and libpkg1.so define two copies of the same types,
Foo and FooBar. pkg2 allocates FooBar from libpkg1.so and then
dynamically casts it as FooBar from libpkg2.so. This is why clang
complains. It has to do with how RTTI comparisons are done across
dynamic objects (see, e.g.,
https://whatofhow.wordpress.com/2015/03/17/odr-rtti-dso/).

I'd say that these errors can be safely ignored, because types are
exactly the same. Anyway, shouldn't it be easier to export get_foobar
in pkg1?

Iñaki

Post by Jan van der Laan
So, help!?
Jan
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Dirk Eddelbuettel]

On 20 August 2018 at 15:35, Iñaki Ucar wrote:
| El lun., 20 ago. 2018 a las 9:01, Jan van der Laan
| (<***@eoos.dds.nl>) escribió:
| >
| >
| > One of my packges gives a message with the undefined behaviour sanitiser
| > (UBSAN). I have difficulty tracking down/understanding the cause of this
| > message. However, I have been able to create a small example that
| > generates the same warning. The mail below is a bit on the long side,
| > and, I hope it is readable as it mixes quite a lot code with text. I am
| > sorry for that. A probably better readable version can be found at
| > https://github.com/djvanderlaan/lvec_ubsan where also the complete
| > source code and the docker image used for testing can be found.
| >
| > My example consists of two packages: pkg1 and pkg2. The package pkg2
| > links to pkg1. Package pkg1 defines a template class `FooBar<T>` which
| > inherits from virtual class `Foo`. Pointers to the these objects are
| > passed back to R using a `Rcpp:XPtr` object:
| >
| > ========== pkg1/src/construct.cpp ================
| > #include "../inst/include/pkg1.h"
| >
| > // [[Rcpp::export]]
| > RcppExport SEXP new_foobar(int i) {
| > BEGIN_RCPP
| > FooBar<int>* x = new FooBar<int>(i);
| > return Rcpp::XPtr<Foo>(x, true);
| > END_RCPP
| > }
|
| Note that RcppExport SEXP should be just SEXP. BEGIN_RCPP and END_RCPP
| should be dropped too.

+1

Or else don't use [[Rcpp::export]] and Rcpp::compileAttributes().

Dirk

| > // [[Rcpp::export]]
| > RcppExport SEXP new_foobard(double i) {
| > BEGIN_RCPP
| > FooBar<double>* x = new FooBar<double>(i);
| > return Rcpp::XPtr<Foo>(x, true);
| > END_RCPP
| > }
| > ========== END pkg1/src/construct.cpp ================
| >
| > All headers of pkg1 are `inst/include`; these are included by `pkg2`:
| >
| > ========== pkg1/inst/include/pkg1.h ================
| > #include <Rcpp.h>
| >
| > // ==== VISITOR
| > template<typename T> class FooBar;
| >
| > class FooBarVisitor {
| > public:
| > virtual void visit(FooBar<double>& vec) = 0;
| > virtual void visit(FooBar<int>& vec) = 0;
| > };
| >
| > // ==== FOO
| > class Foo {
| > public:
| > Foo() {};
| > virtual ~Foo() {};
| >
| > virtual void visit(FooBarVisitor* visitor) = 0;
| > };
| >
| > // ==== FOOBAR
| > template<typename T>
| > class FooBar : public Foo {
| > public:
| > FooBar(T i = 0) : Foo(), i_(i) {};
| > ~FooBar() {};
| >
| > T i() const { return i_;}
| > void i(T i) { i_ = i;}
| >
| > void visit(FooBarVisitor* visitor) {
| > visitor->visit(*this);
| > }
| > private:
| > T i_;
| > };
| > #endif
| > ========== END pkg1/inst/include/pkg1.h ================
| >
| > The XPtr is passed to c++ code in pkg2 and methods of the Foo object are
| > called:
| >
| > ========== pkg2/src/construct.cpp ================
| > #include <Rcpp.h>
| > #include <pkg1.h>
| >
| > // [[Rcpp::export]]
| > RcppExport int get_foobar(SEXP xp) {
| > Rcpp::XPtr<Foo> x(xp);
| > Foo* y1 = x;
| > FooBar<int>* y2 = dynamic_cast<FooBar<int>*>(y1);
| > return y2->i();
| > }
| > ========== END pkg2/src/construct.cpp ================
| >
| > This generates the following UBSAN messages:
| >
| >
| > ========== WARNING MESSAGES ================
| > Testing pkg2
| > <U+221A> | OK F W S | Context
| > / | 0 | fooconstruct.cpp:9:21: runtime error: <null> address
| > 0x000003c21810 which does not point to an object of type 'Foo'
| > 0x000003c21810: note: object is of type 'FooBar<int>'
| > 00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
| > 00 00 00 00 00 00 00 01 03 00 00
| > ^~~~~~~~~~~~~~~~~~~~~~~
| > vptr for 'FooBar<int>'
| > construct.cpp:10:14: runtime error: member call on address
| > 0x000003c21810 which does not point to an object of type 'FooBar<int>'
| > 0x000003c21810: note: object is of type 'FooBar<int>'
| > 00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
| > 00 00 00 00 00 00 00 01 03 00 00
| > ^~~~~~~~~~~~~~~~~~~~~~~
| > vptr for 'FooBar<int>'
| > /usr/local/lib/R/library/pkg1/include/pkg1.h:33:26: runtime error:
| > member access within address 0x000003c21810 which does not point to an
| > object of type 'const FooBar<int>'
| > 0x000003c21810: note: object is of type 'FooBar<int>'
| > 00 00 00 00 58 1c 79 fd 48 7f 00 00 0a 00 00 00 00 00 00 00 20
| > 00 00 00 00 00 00 00 01 03 00 00
| > ^~~~~~~~~~~~~~~~~~~~~~~
| > vptr for 'FooBar<int>'
| > <U+221A> | 1 | foo
| > ========== END WARNING MESSAGES ================
| >
| >
| > I have had real problem reproducing this error. Also searching for the
| > error give little clues as to what the problem is. The following message
| > on the Rcpp-devel list seems somewhat similar:
| > http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2017-July/009656.html.
| > I thought that it might be caused by the fact that I am passing pointers
| > to template classes between libraries. I also couldn't find much about
| > that. I did find some messages on passing pointers to STL objects
| > between libraries. That should be ok, as long as the same compiler is
| > used for both libraries. Although that might technically still make it
| > undefined behaviour.
|
| pkg2 compiles a shared library using pkg1's headers, but it does not
| link against libpkg1.so (in general, this is not possible in R). So
| both libpkg2.so and libpkg1.so define two copies of the same types,
| Foo and FooBar. pkg2 allocates FooBar from libpkg1.so and then
| dynamically casts it as FooBar from libpkg2.so. This is why clang
| complains. It has to do with how RTTI comparisons are done across
| dynamic objects (see, e.g.,
| https://whatofhow.wordpress.com/2015/03/17/odr-rtti-dso/).
|
| I'd say that these errors can be safely ignored, because types are
| exactly the same. Anyway, shouldn't it be easier to export get_foobar
| in pkg1?

The only thought I had so far (and I had no time to play with the code) is to
maybe wrap Rcpp::Shield<> around the XPtr object.

Dirk

|
| Iñaki
|
| >
| > So, help!?
| >
| > Jan
| >
| >
| >
| >
| >
| > _______________________________________________
| > Rcpp-devel mailing list
| > Rcpp-***@lists.r-forge.r-project.org
| > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
|
|
|
| --
| Iñaki Ucar
| _______________________________________________
| Rcpp-devel mailing list
| Rcpp-***@lists.r-forge.r-project.org
| https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Jan van der Laan]

Hi Dirk,

The suggestion of Iñaki has been changed in the github code and also the
visitor stuff is not needed to trigger the exception.

Post by Dirk Eddelbuettel
The only thought I had so far (and I had no time to play with the code) is to
maybe wrap Rcpp::Shield<> around the XPtr object.

I have to say that I am currently on the subway with a spotty internet
connection, so searching is hard, but I have trouble finding
documentation for shield. From what I could find, it would call a
PROTECT on contruction and UNPROTECT on destruction. Is that correct.
Should I do that in pkg1 or pkg2?

Thanks.

Jan

Post by Dirk Eddelbuettel
Dirk

[Dirk Eddelbuettel]

On 20 August 2018 at 16:26, Jan van der Laan wrote:
| Hi Dirk,
|
| The suggestion of Iñaki has been changed in the github code and also the
| visitor stuff is not needed to trigger the exception.
|
|
| On 20-08-18 16:01, Dirk Eddelbuettel wrote:
|
| >
| > The only thought I had so far (and I had no time to play with the code) is to
| > maybe wrap Rcpp::Shield<> around the XPtr object.
|
| I have to say that I am currently on the subway with a spotty internet
| connection, so searching is hard, but I have trouble finding
| documentation for shield. From what I could find, it would call a

There is very little of that :-/

| PROTECT on contruction and UNPROTECT on destruction. Is that correct.
| Should I do that in pkg1 or pkg2?

I would. In essence in C++ you the ctor and dtor "for free" and we use that
to wrap objects. And generally speaking that is better stronger than the
PROTECT and UNPROTECT as we can rely on the scope resolution to have it
triggered. So I would try it. Then again, it may not help, but we find it
made the Rcpp code better.

Dirk

| Thanks.
|
| Jan
|
| >
| > Dirk
| >
|
|
|

[Kevin Ushey]

FWIW, there's an associated bug report here:

https://github.com/google/sanitizers/issues/911

that indicates that ubsan emits false positives when attempting to diagnose
functions opened through dlopen(), which IIUC is the standard way that R
loads symbols from compiled libraries in R packages.

[Jan van der Laan]

Post by Iñaki Ucar
El lun., 20 ago. 2018 a las 9:01, Jan van der Laan

[SNIP]

Post by Iñaki Ucar

Post by Jan van der Laan
========== pkg1/src/construct.cpp ================
#include "../inst/include/pkg1.h"
// [[Rcpp::export]]
RcppExport SEXP new_foobar(int i) {
BEGIN_RCPP
FooBar<int>* x = new FooBar<int>(i);
return Rcpp::XPtr<Foo>(x, true);
END_RCPP
}

Note that RcppExport SEXP should be just SEXP. BEGIN_RCPP and END_RCPP
should be dropped too.

You are right. Thanks. Mixing up old and new code.

[SNIP]

Post by Iñaki Ucar
pkg2 compiles a shared library using pkg1's headers, but it does not
link against libpkg1.so (in general, this is not possible in R). So
both libpkg2.so and libpkg1.so define two copies of the same types,
Foo and FooBar. pkg2 allocates FooBar from libpkg1.so and then
dynamically casts it as FooBar from libpkg2.so. This is why clang
complains. It has to do with how RTTI comparisons are done across
dynamic objects (see, e.g.,
https://whatofhow.wordpress.com/2015/03/17/odr-rtti-dso/).

This is roughly what I suspected. However, if I understand that article
correctly problems should only occur with clang (which I am using;
didn't mention that) and not with g++. However, the problems also occur
with g++
(https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/ldat/ldat-Ex.Rout).

Post by Iñaki Ucar
I'd say that these errors can be safely ignored, because types are
exactly the same. Anyway, shouldn't it be easier to export get_foobar
in pkg1?

Don't know how easy it is to convince the CRAN members to ignore the
warnings. Of course, in this simple case, it is easier to define
get_foobar in pkg1, but in the general case, for performance reasons, I
would like to be able to access FooBar from other packages from c++ code.


Thanks for the response.

Jan

Post by Iñaki Ucar
Iñaki

Post by Jan van der Laan
So, help!?
Jan
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Iñaki Ucar]

El lun., 20 ago. 2018 a las 16:18, Jan van der Laan

Post by Jan van der Laan

Post by Iñaki Ucar
pkg2 compiles a shared library using pkg1's headers, but it does not
link against libpkg1.so (in general, this is not possible in R). So
both libpkg2.so and libpkg1.so define two copies of the same types,
Foo and FooBar. pkg2 allocates FooBar from libpkg1.so and then
dynamically casts it as FooBar from libpkg2.so. This is why clang
complains. It has to do with how RTTI comparisons are done across
dynamic objects (see, e.g.,
https://whatofhow.wordpress.com/2015/03/17/odr-rtti-dso/).

This is roughly what I suspected. However, if I understand that article
correctly problems should only occur with clang (which I am using;
didn't mention that) and not with g++. However, the problems also occur
with g++
(https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/ldat/ldat-Ex.Rout).

I don't know, maybe things have changed in recent versions of gcc.

Post by Jan van der Laan

Post by Iñaki Ucar
I'd say that these errors can be safely ignored, because types are
exactly the same. Anyway, shouldn't it be easier to export get_foobar
in pkg1?

Don't know how easy it is to convince the CRAN members to ignore the
warnings. Of course, in this simple case, it is easier to define
get_foobar in pkg1, but in the general case, for performance reasons, I
would like to be able to access FooBar from other packages from c++ code.

I've just submitted a PR to your repo that fixes the issue:
https://github.com/djvanderlaan/lvec_ubsan/pull/1

Copy&paste of the explanation there, for the interest of this list:

Instead of using useDynLib in the NAMESPACE, this patch loads the
library using library.dynam in .onLoad. In this way, we can take
advantage of library.dynam's dots, which are mapped into the
subsequent dyn.load call. This call has local=TRUE by default, i.e.,
dlopen is called with the RTLD_LOCALattribute. Instead, local=FALSE
sets RTLD_GLOBAL, which makes type_info from pkg1 available globally,
and thus the RTTI error goes away.

Iñaki

Post by Jan van der Laan
Thanks for the response.
Jan

Post by Iñaki Ucar
Iñaki

Post by Jan van der Laan
So, help!?
Jan
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Jan van der Laan]

Post by Iñaki Ucar
El lun., 20 ago. 2018 a las 16:18, Jan van der Laan

Post by Jan van der Laan

Post by Iñaki Ucar
pkg2 compiles a shared library using pkg1's headers, but it does not
link against libpkg1.so (in general, this is not possible in R). So
both libpkg2.so and libpkg1.so define two copies of the same types,
Foo and FooBar. pkg2 allocates FooBar from libpkg1.so and then
dynamically casts it as FooBar from libpkg2.so. This is why clang
complains. It has to do with how RTTI comparisons are done across
dynamic objects (see, e.g.,
https://whatofhow.wordpress.com/2015/03/17/odr-rtti-dso/).

This is roughly what I suspected. However, if I understand that article
correctly problems should only occur with clang (which I am using;
didn't mention that) and not with g++. However, the problems also occur
with g++
(https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/ldat/ldat-Ex.Rout).

I don't know, maybe things have changed in recent versions of gcc.

Post by Jan van der Laan

Post by Iñaki Ucar
I'd say that these errors can be safely ignored, because types are
exactly the same. Anyway, shouldn't it be easier to export get_foobar
in pkg1?

Don't know how easy it is to convince the CRAN members to ignore the
warnings. Of course, in this simple case, it is easier to define
get_foobar in pkg1, but in the general case, for performance reasons, I
would like to be able to access FooBar from other packages from c++ code.

https://github.com/djvanderlaan/lvec_ubsan/pull/1
Instead of using useDynLib in the NAMESPACE, this patch loads the
library using library.dynam in .onLoad. In this way, we can take
advantage of library.dynam's dots, which are mapped into the
subsequent dyn.load call. This call has local=TRUE by default, i.e.,
dlopen is called with the RTLD_LOCALattribute. Instead, local=FALSE
sets RTLD_GLOBAL, which makes type_info from pkg1 available globally,
and thus the RTTI error goes away.

OK, this seems to fix it. Not sure if the CRAN people like it when I
register my symbols globally as this could bite other packages with the
same symbol names. If I understand correctly; I am trying to read in on
this stuff :-S

Tomorrow I will check with lvec and ldat.

Thanks.

Jan

Post by Iñaki Ucar
Iñaki

Post by Jan van der Laan
Thanks for the response.
Jan

Post by Iñaki Ucar
Iñaki

Post by Jan van der Laan
So, help!?
Jan
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Iñaki Ucar]

El lun., 20 ago. 2018 a las 22:16, Jan van der Laan

Post by Jan van der Laan

Post by Iñaki Ucar
El lun., 20 ago. 2018 a las 16:18, Jan van der Laan

Post by Jan van der Laan

Post by Iñaki Ucar
pkg2 compiles a shared library using pkg1's headers, but it does not
link against libpkg1.so (in general, this is not possible in R). So
both libpkg2.so and libpkg1.so define two copies of the same types,
Foo and FooBar. pkg2 allocates FooBar from libpkg1.so and then
dynamically casts it as FooBar from libpkg2.so. This is why clang
complains. It has to do with how RTTI comparisons are done across
dynamic objects (see, e.g.,
https://whatofhow.wordpress.com/2015/03/17/odr-rtti-dso/).

This is roughly what I suspected. However, if I understand that article
correctly problems should only occur with clang (which I am using;
didn't mention that) and not with g++. However, the problems also occur
with g++
(https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/ldat/ldat-Ex.Rout).

I don't know, maybe things have changed in recent versions of gcc.

Post by Jan van der Laan

Post by Iñaki Ucar
I'd say that these errors can be safely ignored, because types are
exactly the same. Anyway, shouldn't it be easier to export get_foobar
in pkg1?

Don't know how easy it is to convince the CRAN members to ignore the
warnings. Of course, in this simple case, it is easier to define
get_foobar in pkg1, but in the general case, for performance reasons, I
would like to be able to access FooBar from other packages from c++ code.

https://github.com/djvanderlaan/lvec_ubsan/pull/1
Instead of using useDynLib in the NAMESPACE, this patch loads the
library using library.dynam in .onLoad. In this way, we can take
advantage of library.dynam's dots, which are mapped into the
subsequent dyn.load call. This call has local=TRUE by default, i.e.,
dlopen is called with the RTLD_LOCALattribute. Instead, local=FALSE
sets RTLD_GLOBAL, which makes type_info from pkg1 available globally,
and thus the RTTI error goes away.

OK, this seems to fix it. Not sure if the CRAN people like it when I
register my symbols globally as this could bite other packages with the
same symbol names. If I understand correctly; I am trying to read in on
this stuff :-S

This is something that could be done on CRAN in those machines that
perform UBSAN checks. I suggested it here:

https://stat.ethz.ch/pipermail/r-devel/2018-August/076636.html

One thing you could do is to try to detect whether lvec is being
compiled with sanitizers and then set that flag. But probably it's not
worth it so much trouble.

Iñaki

Post by Jan van der Laan
Tomorrow I will check with lvec and ldat.
Thanks.
Jan

Post by Iñaki Ucar
Iñaki

Post by Jan van der Laan
Thanks for the response.
Jan

Post by Iñaki Ucar
Iñaki

Post by Jan van der Laan
So, help!?
Jan
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Dirk Eddelbuettel]

On 20 August 2018 at 22:16, Jan van der Laan wrote:
| > Instead of using useDynLib in the NAMESPACE, this patch loads the
| > library using library.dynam in .onLoad. In this way, we can take
| > advantage of library.dynam's dots, which are mapped into the
| > subsequent dyn.load call. This call has local=TRUE by default, i.e.,
| > dlopen is called with the RTLD_LOCALattribute. Instead, local=FALSE
| > sets RTLD_GLOBAL, which makes type_info from pkg1 available globally,
| > and thus the RTTI error goes away.

A looooong time ago I had the same issue with package Rmpi because the Open
MPI library used three different shared libraries and, as I recall, one
needed the RTLD_GLOBAL to have visibilty.

Rmpi still does that:

src/Rmpi.c
77: if (!dlopen("libmpi.so.1", RTLD_GLOBAL | RTLD_LAZY)
78: && !dlopen("libmpi.so.0", RTLD_GLOBAL | RTLD_LAZY)
79: && !dlopen("libmpi.so", RTLD_GLOBAL | RTLD_LAZY)) {

So as a last resort, you could do something like that. But I don't recommend
setting you your own system for shared libraries etc. That way lies madness.

| OK, this seems to fix it. Not sure if the CRAN people like it when I
| register my symbols globally as this could bite other packages with the
| same symbol names. If I understand correctly; I am trying to read in on
| this stuff :-S
|
| Tomorrow I will check with lvec and ldat.

These are all cool packages and we should work on making them easily
interoperable. I like how yours build on top of each other.

Thanks also to Iñaki for the very on-target help.

Dirk