nelson.io.datastore.ParquetDatastore

📝Syntax

pds = parquetDatastore(location)
[T, info] = read(pds)
T = readall(pds)
T = preview(pds)
tf = hasdata(pds)
reset(pds)

📥Input Arguments

Parameter	Description
location	a filename, folder name, wildcard pattern, string array, or cell array of character vectors.
pds	a nelson.io.datastore.ParquetDatastore object.

📤Output Arguments

Parameter	Description
T	a table or timetable.
info	a structure containing the current filename and file index.
tf	a logical value.

📄Description

nelson.io.datastore.ParquetDatastore is created by parquetDatastore.

Properties are Files, ReadSize, SelectedVariableNames, OutputType, RowTimes, RowFilter, VariableNamingRule, and dependent property VariableNames.

read returns the next file as a table and advances the datastore. readall concatenates all remaining files after resetting the datastore. preview reads the first rows of the first file. hasdata indicates whether more files can be read. reset moves the datastore back to the first file.

💡Examples

folder = tempdir();
file1 = [folder, 'doc_ParquetDatastore_1.parquet'];
file2 = [folder, 'doc_ParquetDatastore_2.parquet'];
parquetwrite(file1, table([1; 2], [10; 20], 'VariableNames', {'Id', 'Value'}));
parquetwrite(file2, table([3; 4], [30; 40], 'VariableNames', {'Id', 'Value'}));
pds = parquetDatastore([folder, 'doc_ParquetDatastore_*.parquet']);
hasdata(pds)
[T1, readInfo] = read(pds)
reset(pds);
T = readall(pds)