The .xdf format is a binary file format that:
-
Stores data in blocks for efficient reading of arbitrary columns and contiguous rows
-
Contains associated metadata, such as variable names, descriptions, and data storage types
-
Supports a richer set of data storage types than R (eight types of integers, two types of floating point numbers
-
Writes data blocks of rows so that data processing can be optimized
-
Processes data in chunks (groups of blocks)
-
Optimizes the block sizes depends on individual computer bandwidth for I/O
-
Contains meta information at the end of the .xdf file rather than the beginning for easy updating
