When writing out predictions to XDF files in HDFS, why are results returned in metadata folder?

You should think of both the data and metadata folders and all of the composite XDF files stored in Hadoop in the 'data' folder 
as really all of the information for a single XDF output file. In Hadoop we create a set of composite XDF files(you will see files 
with name (*.xdfm) in the data folder. The metadata folder contains footer information for the XDF file such as number of variables, etc 
that is needed for fast file access.

The reason we store XDF files in Hadoop this way is for fast file access and to take advantage of HDFS block sizes. 
You will need to store the results from rxPredict() as a composite XDf file in Hadoop or as a composite CSV file in HDFS.
Properties

Article ID: 3104123 - Last Review: Oct 31, 2015 - Revision: 1

Revolution Analytics

Feedback