Hi George,
On 9 October 2017 at 10:46, George Vega Yon wrote:
| Hi Dirk,
|
| It seems that something's going on with element modification of
| arma::sp_mat objects. While in most of the cases I use batch insertion to
It looks like we got to the bottom reasonably quicky, but that start with
some off-list emails by @fprive who also created a nice minimal example.
I _think_ a one-line fix of adding sync() before wrap() builds its return
object may do it.
Can you take a look at what we have now at #178 in the RcppArmadillo repo?
[ Never mind, you provided such a great example I did ... ]
Turns out you identified the same issue as @fprive, and this should be fixed
now. Output from my box below.
This gets us output, but from a quick glance it does not match what you get
from Matrix. Can you take a look at what I have below?
Thanks so much for helping to find this!
Dirk
R> Rcpp::sourceCpp("/tmp/rcppArma178p2.cpp")
rcppArma178p2.cpp: In function ‘arma::sp_mat edgelist_to_adjmat_minimal(Rcpp::IntegerMatrix, int)’:
rcppArma178p2.cpp:10:31: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (unsigned int i = 0u; i < edgelist.nrow(); i++)
~~^~~~~~~~~~~~~~~~~
rcppArma178p2.cpp: In function ‘arma::sp_mat edgelist_to_adjmat_minimal2(Rcpp::IntegerMatrix, int)’:
rcppArma178p2.cpp:22:31: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (unsigned int i = 0u; i < edgelist.nrow(); i++)
~~^~~~~~~~~~~~~~~~~
rcppArma178p2.cpp: In function ‘arma::sp_mat edgelist_to_adjmat_minimal3(Rcpp::IntegerMatrix, int)’:
rcppArma178p2.cpp:35:31: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (unsigned int i = 0u; i < edgelist.nrow(); i++)
~~^~~~~~~~~~~~~~~~~
R> library(Matrix)
R> # Creating a random edgelist with n vertices
R> n <- 5
R> set.seed(1)
R> edgelist <- unique(matrix(sample.int(n, 10, TRUE), ncol=2) - 1L)
R> # Coercing to adjacency matrix
R> edgelist_to_adjmat_minimal(edgelist, n)
5 x 5 sparse Matrix of class "dgCMatrix"
[1,] . . . . .
[2,] 1 . . . 1
[3,] . . . 1 .
[4,] . . . . .
[5,] . . . 1 .
R> edgelist_to_adjmat_minimal2(edgelist, n)
5 x 5 sparse Matrix of class "dgCMatrix"
[1,] . . . . .
[2,] 1 . . . 1
[3,] . . . 1 .
[4,] . . . . .
[5,] . . . 1 .
R> edgelist_to_adjmat_minimal3(edgelist, n)
5 x 5 sparse Matrix of class "dgCMatrix"
[1,] . . . . .
[2,] 1 . . . 1
[3,] . . . 1 .
[4,] . . . . .
[5,] . . . 1 .
R> # This is what we should get
R> adjmat_ans <- matrix(0, ncol=n, nrow=n)
R> adjmat_ans[edgelist] <- 1
R> methods::as(adjmat_ans, "dgCMatrix")
5 x 5 sparse Matrix of class "dgCMatrix"
[1,] . . . 1 .
[2,] . . 1 . .
[3,] . . . . .
[4,] . . 1 . .
[5,] . . . . .
R> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 17.04
Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.19.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8
[8] LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] Matrix_1.2-11 dang_0.0.7
loaded via a namespace (and not attached):
[1] compiler_3.4.2 tools_3.4.2 RcppArmadillo_0.8.100.1.0 Rcpp_0.12.13 grid_3.4.2 lattice_0.20-35 fortunes_1.5-4
R>
| create arma::sp_mat objects, in some others I have no other way of doing so
| but by looping through a list of elements to be added(modified). I've
| created a minimal example in which I try to create an adjacency matrix (an
| square matrix of size N) by filling its cells according to an edgelist (a
| matrix with 2 columns providing coordinates). I ran the code using Valgrind
| and no memory leaks were detected, but the code is not behaving as I would
| expect. Here is the code:
|
| ---------------- edgelist_to_adjmat_minimal.cpp --------------------------
|
| #include <RcppArmadillo.h>
| using namespace Rcpp;
|
| // EXAMPLE 1: Element access with -at- and in-place addition
| // [[Rcpp::export]]
| arma::sp_mat edgelist_to_adjmat_minimal(IntegerMatrix edgelist, int n) {
|
| arma::sp_mat adjmat(n,n);
| for (unsigned int i = 0u; i < edgelist.nrow(); i++)
| adjmat.at(edgelist.at(i,0u), edgelist.at(i, 1u)) += 1.0;
|
| return adjmat;
|
| }
|
| // EXAMPLE 2: Element access with -at- and addition.
| // [[Rcpp::export]]
| arma::sp_mat edgelist_to_adjmat_minimal2(IntegerMatrix edgelist, int n) {
|
| arma::sp_mat adjmat(n,n);
| for (unsigned int i = 0u; i < edgelist.nrow(); i++)
| adjmat.at(edgelist.at(i,0u), edgelist.at(i, 1u)) =
| adjmat.at(edgelist.at(i,0u), edgelist.at(i, 1u)) + 1.0;
|
| return adjmat;
|
| }
|
| // EXAMPLE 3: Element access with -()- and addition
| // [[Rcpp::export]]
| arma::sp_mat edgelist_to_adjmat_minimal3(IntegerMatrix edgelist, int n) {
|
| arma::sp_mat adjmat(n,n);
| for (unsigned int i = 0u; i < edgelist.nrow(); i++)
| adjmat(edgelist.at(i,0u), edgelist.at(i, 1u)) =
| adjmat(edgelist.at(i,0u), edgelist.at(i, 1u)) + 1.0;
|
| return adjmat;
|
| }
|
| /***R
| library(Matrix)
|
| # Creating a random edgelist with n vertices
| n <- 5
| set.seed(1)
| edgelist <- unique(matrix(sample.int(n, 10, TRUE), ncol=2) - 1L)
|
| # Coercing to adjacency matrix
| edgelist_to_adjmat_minimal(edgelist, n)
| edgelist_to_adjmat_minimal2(edgelist, n)
| edgelist_to_adjmat_minimal3(edgelist, n)
|
| # This is what we should get
| adjmat_ans <- matrix(0, ncol=n, nrow=n)
| adjmat_ans[edgelist] <- 1
| methods::as(adjmat_ans, "dgCMatrix")
|
| sessionInfo()
| */
|
| ---------------- edgelist_to_adjmat_minimal.cpp --------------------------
|
|
| And here is what I get
|
|
| ----------------- begin of output ------------------------------
| --------------
| > Rcpp::sourceCpp("playground/edgelist_to_adjmat_minimal.cpp")
|
| > library(Matrix)
|
| > # Creating a random edgelist with n vertices
| > n <- 5
|
| > set.seed(1)
|
| > edgelist <- unique(matrix(sample.int(n, 10, TRUE), ncol=2) - 1L)
|
| > # Coercing to adjacency matrix
| > edgelist_to_adjmat_minimal(edgelist, n)
| 5 x 5 sparse Matrix of class "dgCMatrix"
|
| [1,] . . . . .
| [2,] . . . . .
| [3,] . . . . .
| [4,] . . . . .
| [5,] . . . . .
|
| > edgelist_to_adjmat_minimal2(edgelist, n)
| 5 x 5 sparse Matrix of class "dgCMatrix"
|
| [1,] . . . . .
| [2,] . . . . .
| [3,] . . . . .
| [4,] . . . . .
| [5,] . . . . .
|
| > edgelist_to_adjmat_minimal3(edgelist, n)
| 5 x 5 sparse Matrix of class "dgCMatrix"
|
| [1,] . . . . .
| [2,] . . . . .
| [3,] . . . . .
| [4,] . . . . .
| [5,] . . . . .
|
| > # This is what we should get
| > adjmat_ans <- matrix(0, ncol=n, nrow=n)
|
| > adjmat_ans[edgelist] <- 1
|
| > methods::as(adjmat_ans, "dgCMatrix")
| 5 x 5 sparse Matrix of class "dgCMatrix"
|
| [1,] . . . 1 .
| [2,] . . 1 . .
| [3,] . . . . .
| [4,] . . 1 . .
| [5,] . . . . .
|
| > sessionInfo()
| R version 3.4.2 (2017-09-28)
| Platform: x86_64-pc-linux-gnu (64-bit)
| Running under: Ubuntu 14.04.5 LTS
|
| Matrix products: default
| BLAS: /usr/lib/libblas/libblas.so.3.0
| LAPACK: /usr/lib/lapack/liblapack.so.3.0
|
| locale:
| [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
| [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
| [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
| [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
| [9] LC_ADDRESS=C LC_TELEPHONE=C
| [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
|
| attached base packages:
| [1] stats graphics grDevices utils datasets methods base
|
| other attached packages:
| [1] Matrix_1.2-11
|
| loaded via a namespace (and not attached):
| [1] compiler_3.4.2 tools_3.4.2
| [3] RcppArmadillo_0.8.100.1.0 Rcpp_0.12.13
| [5] grid_3.4.2 lattice_0.20-35
|
| ------------------------- end of output ------------------------------
| -----------
|
| I use three different examples thinking that perhaps some element accessing
| method was no longer valid, or even the in-place addition was no longer
| working.
|
| Anything looks wrong in what I've written? From what I've seen in my test
| results, all of the errors that I've checked so far have to do with this.
|
| Best,
|
|
| George G. Vega Yon
| +1 (626) 381 8171 <(626)%20381-8171>
| http://cana.usc.edu/vegayon
|
| On Sat, Oct 7, 2017 at 3:15 PM, Dirk Eddelbuettel <***@debian.org> wrote:
|
| >
| > RcppArmadillo 0.8.100.1.0 is available in the drat repo for the RcppCore
| > org.
| > See the README.md at https://github.com/RcppCore/drat which has this
| > exammple:
| >
| > # first add the repo
| > drat:::add("RcppCore")
| > # either install just one or more given packages
| > install.packages("RcppArmadillo")
| > # or update already installed packages
| > update.packages()
| >
| > You can also add the repo URL by hand to options("repos"), or supply it to
| > install.packages(), or ... I happen to like drat. The files NEWS.Rd and
| > ChangeLog have the goods, I make a fuller announcement if and when it makes
| > it to CRAN.
| >
| > There are a rathre fair number of upstream changes in here -- making it the
| > first Armadillo release by Conrad with an 8.* number. It also has further
| > sparse matrix improvements from our end thanks to Serguei and Binxiang.
| >
| > I had uploaded this to CRAN based on a reverse depends check ... where the
| > previous version was still in the loadpath. Ooops. So I overlooked a few
| > build or test errors for which I need a bit of help from the maintainers.
| > In
| > particular, packages
| >
| > biglasso
| > bigstatsr
| > HSAR
| > netdiffuseR
| > repolr
| >
| > now come up as failing their tests. I am BCCing the maintainers and kindly
| > ask if they could take a look too --- I don't always have all suggested
| > packages installed.
| >
| > In the case of HSAR, I found that a one life of code needs a changed
| > imposed
| > by Conrad as the sparse-to-dense conversion no longer works with with a
| > copy. This patch covers it:
| >
| > diff -ru HSAR.orig/src/diagnostics.cpp HSAR/src/diagnostics.cpp
| > --- HSAR.orig/src/diagnostics.cpp 2016-05-24 13:58:45.000000000 -0500
| > +++ HSAR/src/diagnostics.cpp 2017-10-07 13:24:31.471883248 -0500
| > @@ -10,7 +10,7 @@
| >
| > sp_mat SW = I_sp+rho*W+pow(rho,2)*(W*W)+pow(rho,3)*(W*W*W)+pow(rho,4)*(
| > W*W*W*W)+pow(rho,5)*(W*W*W*W*W);
| >
| > - vec d = SW.diag();
| > + vec d(SW.diag());
| >
| > direct = sum( d )/n * betas ;
| > total = accu( SW )/n * betas ;
| >
| >
| > For the other packages, I had less luck as the failures are generally in
| > the
| > tests and often require packages I have not installed (here at home, and
| > for
| > technical I don't currently have access to the machine where I usually run
| > the tests).
| >
| > Sp I would be grateful if the maintainers of packages
| >
| > biglasso, bigstatsr, netdiffuseR, repolr
| >
| > (or also any interested volunteers) could take a peek. It seems of the
| > now
| > over 400 CRAN package using RcppArmadillo, all others are fine too -- and I
| > suspect these four also need only small and simple changes. Happy to
| > discuss
| > here.
| >
| >
| > Cheers, Dirk
| >
| >
| > --
| > http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org
| >
--
http://dirk.eddelbuettel.com | @eddelbuettel | ***@debian.org