Skip to content

[C++][Parquet] parquet::arrow::FileWriter does not propagate schema-level metadata when ArrowWriterProperties::store_schema is false #41766

@TheNeuralBit

Description

@TheNeuralBit

Describe the bug, including details regarding any error messages, version, and platform.

When store_schema is true the FileWriter first copies any existing metadata before storing the serialized schema:

std::shared_ptr<KeyValueMetadata> result;
if (schema.metadata()) {
result = schema.metadata()->Copy();
} else {
result = ::arrow::key_value_metadata({}, {});
}

But when store_schema is false, the FileWriter just returns an empty metadata, and custom metadata is not copied:

if (!properties.store_schema()) {
*out = nullptr;
return Status::OK();
}

Could someone confirm if this is intentional or not? It looks like an oversight to me and I have a patch ready to address it.

Component(s)

Parquet

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions