-
Notifications
You must be signed in to change notification settings - Fork 26
Add optional pledgeinsize function to transcoding protocol
#239
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
I wonder if it would be appropriate to call this |
|
I chose the name to match the style of the other functions in the transcoding protocol. This is also slightly stronger than a hint since with Zstd setting the wrong size will cause an error. |
|
I guess the convention here is not to use Could we make it clearer the order in which these functions will be called? Also do we have a mechanism to pledge the size even in streaming mode? |
mkitti
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Document call order and consider potential uses outside of transcode. For example, what if I have a large file of known size that I plan to stream.
|
I added some more details to the order of the function calls. The transcoding protocol can be used outside of the |
|
@mkitti I added some examples of how |
This release adds the optional `pledgeinsize` function to the transcoding protocol (PR #239)
The
pledgeinsize(codec::C, insize::Int64, error::Error)::Symbolmethod is usedwhen
transcodeis called to tell thecodecthe total input size.This is called after
startprocand beforeprocess. Somecompressors can add this total input size to a header, making
expectedsizeaccurate during later decompression. By default this just returns
:ok.If there is an error, the return code must be
:errorand theerrorargumentmust be set to an exception object. Setting an inaccurate
insizemay cause thecodec to error later on while processing data. A negative
insizemeans unknowncontent size.
Ref: JuliaIO/CodecZstd.jl#58
I think this is a nicer alternative to #215