[Rcpp-devel] passing a bit64 vector to C++ and returning it

[David Bellot]

Hi,

it looks like I couldn't find a proper solution, even after googling every
where. The question is simple:

I have a bit64 vector of integers (timestamps of financial tick data, you
bet ?) and I want to pass it to a C++ function. The good ol' method is to
convert it to strings, pass it and cast strings back to 64 bits integers in
C++. And vice and versa when I pass my result back. But the size of my
vectors are in the range of millions of items, so the method is just a bit
slow.

Since Rcpp 0.9.8, there is no more support for the LongVector class. Papers
like http://www.sciencedirect.com/science/article/pii/S0098300416307415,
despite very interesting, were not very useful to solve my simple problem,
so I was wondering if there is a better solution ?

reinterpret_cast ?
GenericVector or RawVector ?
Anything so obvious that I just missed it ?

And of course, same questions in the other direction: how to return my
vector back to R and have a bit64 at the end ?

Thanks for your help,
David

[Kevin Ushey]

My understanding is that `integer64` actually stores its contents in an R
numeric vector, so presumedly you should be able to handle this by
forcefully casting the contents as you access it, e.g.

#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
NumericVector timesTwo(NumericVector input) {

R_xlen_t n = input.size();
NumericVector output = no_init(n);

long long* pInput = (long long*) dataptr(input);
long long* pOutput = (long long*) dataptr(output);
for (R_xlen_t i = 0; i < n; i++)
*pOutput++ = *pInput++ * 2;

output.attr("class") = "integer64";
return output;
}

/*** R
library(bit64)
object <- as.integer64(1:16)
timesTwo(object)
*/

Note that the use of 'long long' here requires a C++11 compiler, or a C++98
compiler with 'long long' provided as an extension.

It's not the nicest solution, but at least this should get you started --
just note that you need to create a NumericVector, set the 'integer64'
class, and fill it as though it were a vector of 'long long' (64-bit signed
integer)s.

Kevin

Post by David Bellot
Hi,
it looks like I couldn't find a proper solution, even after googling every
I have a bit64 vector of integers (timestamps of financial tick data, you
bet ?) and I want to pass it to a C++ function. The good ol' method is to
convert it to strings, pass it and cast strings back to 64 bits integers in
C++. And vice and versa when I pass my result back. But the size of my
vectors are in the range of millions of items, so the method is just a bit
slow.
Since Rcpp 0.9.8, there is no more support for the LongVector class.
Papers like http://www.sciencedirect.com/science/article/pii/
S0098300416307415, despite very interesting, were not very useful to
solve my simple problem, so I was wondering if there is a better solution ?
reinterpret_cast ?
GenericVector or RawVector ?
Anything so obvious that I just missed it ?
And of course, same questions in the other direction: how to return my
vector back to R and have a bit64 at the end ?
Thanks for your help,
David
_______________________________________________
Rcpp-devel mailing list
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

[Dirk Eddelbuettel]

On 11 July 2017 at 23:44, David Bellot wrote:
| it looks like I couldn't find a proper solution, even after googling every
| where. The question is simple:
|
| I have a bit64 vector of integers (timestamps of financial tick data, you
| bet ?) and I want to pass it to a C++ function. The good ol' method is to
| convert it to strings, pass it and cast strings back to 64 bits integers in

Have you seen my 'nanotime' package? It does just that: provide nanosecond
resolution by relying on RcppCCTZ for the parsing/formating and int64 for the
storage.

| C++. And vice and versa when I pass my result back. But the size of my
| vectors are in the range of millions of items, so the method is just a bit
| slow.

You never ever want to format + parse that many vectors. It is too slow.
|
| Since Rcpp 0.9.8, there is no more support for the LongVector class. Papers
| like http://www.sciencedirect.com/science/article/pii/S0098300416307415,
| despite very interesting, were not very useful to solve my simple problem,
| so I was wondering if there is a better solution ?
|
| reinterpret_cast ?
| GenericVector or RawVector ?
| Anything so obvious that I just missed it ?
|
| And of course, same questions in the other direction: how to return my
| vector back to R and have a bit64 at the end ?

As Kevin correctly pointed out, int64 really just "rearranges" the 64 bits in
a double. It is implemeneted as a REALSXP, or for us, a NumericVector.

Poke around those two packages of mine and you should get farther.

Dirk

[Kirill Müller]

Post by Dirk Eddelbuettel
| it looks like I couldn't find a proper solution, even after googling every
|
| I have a bit64 vector of integers (timestamps of financial tick data, you
| bet ?) and I want to pass it to a C++ function. The good ol' method is to
| convert it to strings, pass it and cast strings back to 64 bits integers in
Have you seen my 'nanotime' package? It does just that: provide nanosecond
resolution by relying on RcppCCTZ for the parsing/formating and int64 for the
storage.
| C++. And vice and versa when I pass my result back. But the size of my
| vectors are in the range of millions of items, so the method is just a bit
| slow.
You never ever want to format + parse that many vectors. It is too slow.
|
| Since Rcpp 0.9.8, there is no more support for the LongVector class. Papers
| like http://www.sciencedirect.com/science/article/pii/S0098300416307415,
| despite very interesting, were not very useful to solve my simple problem,
| so I was wondering if there is a better solution ?
|
| reinterpret_cast ?
| GenericVector or RawVector ?
| Anything so obvious that I just missed it ?
|
| And of course, same questions in the other direction: how to return my
| vector back to R and have a bit64 at the end ?
As Kevin correctly pointed out, int64 really just "rearranges" the 64 bits in
a double. It is implemeneted as a REALSXP, or for us, a NumericVector.
Poke around those two packages of mine and you should get farther.

For the sake of completeness, the RSQLite package supports reading (but
not yet writing) of 64-bit data, and uses Rcpp for this.


-Kirill

Post by Dirk Eddelbuettel
Dirk

[David Bellot]

Post by Dirk Eddelbuettel
Have you seen my 'nanotime' package? It does just that: provide nanosecond

Post by Dirk Eddelbuettel
resolution by relying on RcppCCTZ for the parsing/formating and int64 for the
storage.

​Yes I use nanotime now. Excellent package.

Also the solution to cast numericvector to long long is perfect too, as I
needed to manipulate my timestamps anyway.
Thanks everyone, problem solved in no time !

[Dirk Eddelbuettel]

On 12 July 2017 at 08:40, David Bellot wrote:
| >
| > Have you seen my 'nanotime' package? It does just that: provide nanosecond
| >> resolution by relying on RcppCCTZ for the parsing/formating and int64 for
| >> the
| >> storage.
| >>
| >
| ​Yes I use nanotime now. Excellent package.

Thanks! Help/suggestion/PRs welcome at its repo.

| Also the solution to cast numericvector to long long is perfect too, as I
| needed to manipulate my timestamps anyway.

Not to be a stickler but I guess these days I might use uint64_t instead.
You should be able to get that losslessly to/from a NumericVector (aka
double).

For all things plotting for now the best bet is a (lossy !!) conversion down
to POSIXct.

| Thanks everyone, problem solved in no time !

Cool.

Dirk