Problem: rxImport() - "Failed to allocate XXXXXXXXX bytes."

This is a general memory allocation error. The usual problem is that Revolution R is trying to read too many rows at once from your data file to process the data in memory for a single chunk of data.

First try the following, to resolve the issue:

Set a small value for the argument 'rowsPerRead' in your rxImport() statement. Try a value of '10000'  or less. You may need to try different settings for

this to find a value that works well and imports the data as quickly as possible.

 

If this doesn't help and your csv file has a lot of columns, it can help to import the data 'x' columns at a time. For example, if your dataset as 5000 columns, you may want to import the data for 50 columns at a time and write out the data for 50 columns to a new XDF file and append to that existing XDF file.

Here is some sample R code to do that:

varNames <- readLines("mycsv.txt", n=1) colsPerRead <- 50   ## Set how many columns to read from the csv file at a time. You may want to initially set this to a larger value, say 100. numReadsFromFile <- length(varNames/colsPerRead)for (i in 1:numReadsFromFile) {  tempdf <- rxImport(inData = "C:/MyRData/data.csv", varsToKeep = paste(varNames[((i-1)*colsPerRead)+1:(((i-1)*colsPerRead)+1)+colsPerRead], sep = ","),  rowsPerRead = 10000)  rxDataFrameToXdf(data = tempdf, ouFile = "C:/MyRData/data.xdf", append = "cols") }

Note This is a "FAST PUBLISH" article created directly from within the Microsoft support organization. The information contained herein is provided as-is in response to emerging issues. As a result of the speed in making it available, the materials may include typographical errors and may be revised at any time without notice. See Terms of Use for other considerations.
Properties

Article ID: 3103858 - Last Review: 11/01/2015 16:04:00 - Revision: 1.0

Revolution Analytics

  • KB3103858
Feedback