You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Refactor Suggestion: Rename ColumnChunk / RowGroup, and their read<...> to ColumnChunkMeta / RowGroupMeta #183
The issue:
these records does not contain actual bytes, they are also metadata, but their name suggests they do, readRowGroup, readColumnChunk also suggests it loads entire row group / column chunk into memory as a RowGroup / ColumnChunk. (confused me a bit)
It would be conceptually cleaner if we rename them to suggest they are just metadata (contain no actual bytes).
The risk is that, currently there is no export control and the entire module is exposed-module, rename would cause a public API change.
This is a simple change, I can submit a PR if needed. Alternatively we can add haddock documentation to the readColumnChunk / readRowGroup functions and their data types to reduce confusion.
I agree that the names are weird but we'd be diverging from parquet proper.
RE public API I think everything before v1 is public so people can experiment with it and find issues. APIs will break and change. Then at v1 we'll lock in a more permanent API.
Activity
mchav commentedon Mar 14, 2026
The names are carried over directly from the Parquet spec.
https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L1021
I agree that the names are weird but we'd be diverging from parquet proper.
RE public API I think everything before v1 is public so people can experiment with it and find issues. APIs will break and change. Then at v1 we'll lock in a more permanent API.