ARROW-9870: [R] Friendly interface for filesystems (S3) #8351
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bkietz
reviewed
Oct 7, 2020
Member
bkietz
left a comment
bkietz
left a comment
There was a problem hiding this comment.
This looks like a nice improvement in concision. A few comments and lots of wording suggestions
nealrichardson
force-pushed
the
r-fs-ui
branch
from
a802ad2 to
a4a95ba
Compare
Co-authored-by: Benjamin Kietzman <[email protected]>
nealrichardson
force-pushed
the
r-fs-ui
branch
from
66da7c0 to
ed2165c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SubTreeFileSystemclass to represent a filesystem+path (since that's what it contains already) and wire that up in the file/dataset reader/writer functions that in a recent PR all got afilesystemargument added to their signatures (removing the extra argument). AddFileSystem$path(string)helper method to create a STFS, and recommend that as the way to pass a filesystem+path to those functions. I recognize that this is an abuse of STFS and will revisit in the future (ARROW-10254), but for now, this yields an improved interface.s3_bucket()as a function to create a STFS containing anS3FileSystemand the bucket's path, which also takes advantage of FileSystemFromUri's ability to auto-detect the bucket's region while also allowing you to specify extra S3Options for authentication etc.copy_files()to take these inputsFileSystem$ls()as convenience over FileSelector + GetFileInfo etc.