Problem: rxImport() - "Failed to allocate XXXXXXXXX bytes."

This is a general memory allocation error. The usual problem is that Revolution R is trying to read too many rows at once from your data file to process the data in memory for a single chunk of data.

First try the following, to resolve the issue:

Set a small value for the argument 'rowsPerRead' in your rxImport() statement. Try a value of '10000'  or less. You may need to try different settings for

this to find a value that works well and imports the data as quickly as possible.

 

If this doesn't help and your csv file has a lot of columns, it can help to import the data 'x' columns at a time. For example, if your dataset as 5000 columns, you may want to import the data for 50 columns at a time and write out the data for 50 columns to a new XDF file and append to that existing XDF file.

Here is some sample R code to do that:

varNames <- readLines("mycsv.txt", n=1) 
colsPerRead <- 50   ## Set how many columns to read from the csv file at a time. You may want to initially set this to a larger value, say 100. 
numReadsFromFile <- length(varNames/colsPerRead)

for (i in 1:numReadsFromFile) 
{ 
 tempdf <- rxImport(inData = "C:/MyRData/data.csv", varsToKeep = paste(varNames[((i-1)*colsPerRead)+1:(((i-1)*colsPerRead)+1)+colsPerRead], sep = ","), 
 rowsPerRead = 10000) 
 rxDataFrameToXdf(data = tempdf, ouFile = "C:/MyRData/data.xdf", append = "cols") 
}