[Rcpp-devel] How to handle std::cout/std::cerr in shared libraries

[Watal M. Iwasaki]

Hi all,

I have been working on an R interface package to a shared library written
in C++. It works almost perfectly, but the output to std::cout is not shown
on R console (on Linux). Of course Rcpp::Rcout should be used instead of
std::cout. But in this case I want to let the shared library be pure C++.
So I tried switching the stream buffers in the Rcpp side:
```
Rcpp::CharacterVector run(const std::vector<std::string>& args) {
std::streambuf* obuf = std::cout.rdbuf(Rcpp::Rcout.rdbuf());
mylib::Simulation simulation(args);
simulation.run(); // std::cout is used in here
std::cout.rdbuf(obuf);
return "something";
}
```

It (seemingly) works! But now `devtools::check()` produces 1 note:
```
❯ checking compiled code ... NOTE
File ‘mylib/libs/mylib.so’:
Found ‘__ZNSt3__14coutE’, possibly from ‘std::cout’ (C++)
Object: ‘run.o’

Compiled code should not call entry points which might terminate R nor
write to stdout/stderr instead of to the console, nor use Fortran I/O
nor system RNGs.
```

Can I suppress this note? or should I just ignore it? any other good
solution?

Thanks,
Watal M. Iwasaki

[Dirk Eddelbuettel]

On 26 October 2018 at 16:31, Watal M. Iwasaki wrote:
| I have been working on an R interface package to a shared library written
| in C++. It works almost perfectly, but the output to std::cout is not shown
| on R console (on Linux). Of course Rcpp::Rcout should be used instead of
| std::cout. But in this case I want to let the shared library be pure C++.
| So I tried switching the stream buffers in the Rcpp side:
| ```
| Rcpp::CharacterVector run(const std::vector<std::string>& args) {
| std::streambuf* obuf = std::cout.rdbuf(Rcpp::Rcout.rdbuf());
| mylib::Simulation simulation(args);
| simulation.run(); // std::cout is used in here
| std::cout.rdbuf(obuf);
| return "something";
| }
| ```
|
| It (seemingly) works! But now `devtools::check()` produces 1 note:
| ```
| ❯ checking compiled code ... NOTE
| File ‘mylib/libs/mylib.so’:
| Found ‘__ZNSt3__14coutE’, possibly from ‘std::cout’ (C++)
| Object: ‘run.o’
|
| Compiled code should not call entry points which might terminate R nor
| write to stdout/stderr instead of to the console, nor use Fortran I/O
| nor system RNGs.
| ```
|
| Can I suppress this note? or should I just ignore it? any other good
| solution?

You need to change it. There is no automatic fix.

[ Longer answer: See eg what we do in RcppArmadillo where we #define a device
which for builds that we do from R fills in Rcpp::Rcout and otherwise
defaults to std::cout. But the essence is the same: _You_ need to change
that library code to conform. Or keep the library outside the R package but
then you have an external dependency. ]

Dirk

[Watal M. Iwasaki]

Dear Dirk,

Thank you for the prompt response. Good to know there is no easy way. I
have made up my mind to change the library code as you suggested. But I
don't like preprocessor macro; therefore, the problem here was solved by
moving/hiding `std::cout.rdbuf()` part into the library as a function that
takes a streambuf pointer, and just calling it from Rcpp side. Now, output
is properly sent to R console, R CMD check complains nothing, and the
library still remains free from R/Rcpp code. Thanks again.

Best,
Watal

Post by Dirk Eddelbuettel
| I have been working on an R interface package to a shared library written
| in C++. It works almost perfectly, but the output to std::cout is not shown
| on R console (on Linux). Of course Rcpp::Rcout should be used instead of
| std::cout. But in this case I want to let the shared library be pure C++.
| ```
| Rcpp::CharacterVector run(const std::vector<std::string>& args) {
| std::streambuf* obuf = std::cout.rdbuf(Rcpp::Rcout.rdbuf());
| mylib::Simulation simulation(args);
| simulation.run(); // std::cout is used in here
| std::cout.rdbuf(obuf);
| return "something";
| }
| ```
|
| ```
| ❯ checking compiled code ... NOTE
| Found ‘__ZNSt3__14coutE’, possibly from ‘std::cout’ (C++)
| Object: ‘run.o’
|
| Compiled code should not call entry points which might terminate R nor
| write to stdout/stderr instead of to the console, nor use Fortran I/O
| nor system RNGs.
| ```
|
| Can I suppress this note? or should I just ignore it? any other good
| solution?
You need to change it. There is no automatic fix.
[ Longer answer: See eg what we do in RcppArmadillo where we #define a device
which for builds that we do from R fills in Rcpp::Rcout and otherwise
defaults to std::cout. But the essence is the same: _You_ need to change
that library code to conform. Or keep the library outside the R package but
then you have an external dependency. ]
Dirk
--

[Dirk Eddelbuettel]

On 26 October 2018 at 22:03, Watal M. Iwasaki wrote:
| Dear Dirk,
|
| Thank you for the prompt response. Good to know there is no easy way. I
| have made up my mind to change the library code as you suggested. But I
| don't like preprocessor macro; therefore, the problem here was solved by
| moving/hiding `std::cout.rdbuf()` part into the library as a function that
| takes a streambuf pointer, and just calling it from Rcpp side. Now, output
| is properly sent to R console, R CMD check complains nothing, and the
| library still remains free from R/Rcpp code. Thanks again.

Nicely done.

We could do with a more general solution to this. If you have ideas ...

Dirk

[Watal M. Iwasaki]

Dear Dirk,

Is it possible for Rcpp to do some pre-execution hook before user code? For
example, if Rcpp system can hijack the std::cout buffer by executing
`std::cout.rdbuf(Rcpp::Rcout.rdbuf())` automatically, then Rcpp users (and
external libraries) no longer have to care about Rcout, and can just stick
to std::cout.

Best,
Watal

Post by Dirk Eddelbuettel
| Dear Dirk,
|
| Thank you for the prompt response. Good to know there is no easy way. I
| have made up my mind to change the library code as you suggested. But I
| don't like preprocessor macro; therefore, the problem here was solved by
| moving/hiding `std::cout.rdbuf()` part into the library as a function that
| takes a streambuf pointer, and just calling it from Rcpp side. Now, output
| is properly sent to R console, R CMD check complains nothing, and the
| library still remains free from R/Rcpp code. Thanks again.
Nicely done.
We could do with a more general solution to this. If you have ideas ...
Dirk
--

[Dirk Eddelbuettel]

Hi Watal,

On 26 October 2018 at 23:36, Watal M. Iwasaki wrote:
| Is it possible for Rcpp to do some pre-execution hook before user code? For
| example, if Rcpp system can hijack the std::cout buffer by executing
| `std::cout.rdbuf(Rcpp::Rcout.rdbuf())` automatically, then Rcpp users (and
| external libraries) no longer have to care about Rcout, and can just stick
| to std::cout.

Sorry, but we need the opposite (which is what Rcpp::Rcout does): we need to
get what _would otherwise go to std::cout_ and feed it to Rcpp::Rcout which
then hands it to R's buffered i/o. That is the whole underlying reason --
see the as always very detailed "Writing R Extensions" manual for a bit more.

Automating that (ie replacing std::cout by Rcpp::Rcout) is what is needed.

Dirk

[Watal M. Iwasaki]

Dear Dirk,

Sorry for my poor explanation. I have read
`inst/include/Rcpp/iostream/Rstreambuf.h` and think I understand the role
of Rcpp::Rcout, but failed to explain my point. By "users [...] can just
stick to std::cout", I did not mean allowing users to write to stdout via
std::cout. It is about changing the destination of std::cout from stdout to
R's i/o. The following code hopefully explains better:

```
// [[Rcpp::export]]
void example_function() {
// Normal state
std::cout << "to stdout; BAD\n";
Rcpp::Rcout << "to R's i/o\n";

// Ideal state
std::streambuf* stdoutbuf = std::cout.rdbuf(Rcpp::Rcout.rdbuf());
std::cout << "to R's i/o via Rcpp::Rcout.buf; GOOD\n"
Rcpp::Rcout << "to R's i/o\n";

// Restore original state
std::cout.rdbuf(stdoutbuf);
std::cout << "to stdout; BAD\n";
Rcpp::Rcout << "to R's i/o\n";
}
```

In other words, we don't have to replace std::cout with Rcpp::Rcout. We
only have to change its buffer.

Thanks,
Watal

Post by Dirk Eddelbuettel
Hi Watal,
| Is it possible for Rcpp to do some pre-execution hook before user code? For
| example, if Rcpp system can hijack the std::cout buffer by executing
| `std::cout.rdbuf(Rcpp::Rcout.rdbuf())` automatically, then Rcpp users (and
| external libraries) no longer have to care about Rcout, and can just stick
| to std::cout.
Sorry, but we need the opposite (which is what Rcpp::Rcout does): we need to
get what _would otherwise go to std::cout_ and feed it to Rcpp::Rcout which
then hands it to R's buffered i/o. That is the whole underlying reason --
see the as always very detailed "Writing R Extensions" manual for a bit more.
Automating that (ie replacing std::cout by Rcpp::Rcout) is what is needed.
Dirk
--

[Dirk Eddelbuettel]

Hi Watal,

Thanks for being patient with me :)

On 27 October 2018 at 01:26, Watal M. Iwasaki wrote:
| Sorry for my poor explanation. I have read
| `inst/include/Rcpp/iostream/Rstreambuf.h` and think I understand the role
| of Rcpp::Rcout, but failed to explain my point. By "users [...] can just
| stick to std::cout", I did not mean allowing users to write to stdout via
| std::cout. It is about changing the destination of std::cout from stdout to
| R's i/o. The following code hopefully explains better:
|
| ```
| // [[Rcpp::export]]
| void example_function() {
| // Normal state
| std::cout << "to stdout; BAD\n";
| Rcpp::Rcout << "to R's i/o\n";
|
| // Ideal state
| std::streambuf* stdoutbuf = std::cout.rdbuf(Rcpp::Rcout.rdbuf());
| std::cout << "to R's i/o via Rcpp::Rcout.buf; GOOD\n"
| Rcpp::Rcout << "to R's i/o\n";
|
| // Restore original state
| std::cout.rdbuf(stdoutbuf);
| std::cout << "to stdout; BAD\n";
| Rcpp::Rcout << "to R's i/o\n";
| }
| ```
|
| In other words, we don't have to replace std::cout with Rcpp::Rcout. We
| only have to change its buffer.

I like it! I think that may fix it.

It may not fix the detection by R CMD check, but if we can show that we do
the right thing maybe this can be adjusted (to also account for Rcpp and
whatever (hidden) macro we may use to provide this.

Dirk

[Watal M. Iwasaki]

Yes, escaping from the check seems difficult. I have created a quick PR on
this for further discussion.

Watal

Post by Dirk Eddelbuettel
Hi Watal,
Thanks for being patient with me :)
| Sorry for my poor explanation. I have read
| `inst/include/Rcpp/iostream/Rstreambuf.h` and think I understand the role
| of Rcpp::Rcout, but failed to explain my point. By "users [...] can just
| stick to std::cout", I did not mean allowing users to write to stdout via
| std::cout. It is about changing the destination of std::cout from stdout to
|
| ```
| // [[Rcpp::export]]
| void example_function() {
| // Normal state
| std::cout << "to stdout; BAD\n";
| Rcpp::Rcout << "to R's i/o\n";
|
| // Ideal state
| std::streambuf* stdoutbuf = std::cout.rdbuf(Rcpp::Rcout.rdbuf());
| std::cout << "to R's i/o via Rcpp::Rcout.buf; GOOD\n"
| Rcpp::Rcout << "to R's i/o\n";
|
| // Restore original state
| std::cout.rdbuf(stdoutbuf);
| std::cout << "to stdout; BAD\n";
| Rcpp::Rcout << "to R's i/o\n";
| }
| ```
|
| In other words, we don't have to replace std::cout with Rcpp::Rcout. We
| only have to change its buffer.
I like it! I think that may fix it.
It may not fix the detection by R CMD check, but if we can show that we do
the right thing maybe this can be adjusted (to also account for Rcpp and
whatever (hidden) macro we may use to provide this.
Dirk
--