[Python] Add support for GCS URI (gs://) to pyarrow.parquet.read_table

[KellyWalker]

Describe the enhancement requested

Currently, this works:

from gcsfs import GCSFileSystem
import pyarrow.parquet as pq
gcs = GCSFileSystem()
parquet_file = pq.read_table("/bucket/path/to/file.parquet", filesystem=gcs)

But this does not:

import pyarrow.parquet as pq
parquet_file = pq.read_table("gs://bucket/path/to/file.parquet")

It would be nice if the latter worked directly without needing to specify the filesystem.

Component(s)

Parquet, Python

[kou]

Does this work?

import pyarrow.dataset as ds
dataset = ds.dataset("gs://bucket/path/to/file.parquet", format="parquet")
parquet_file = dataset.to_table()

[KellyWalker]

No.

In both cases the following error is generated:
pyarrow.lib.ArrowNotImplementedError: Got GCS URI but Arrow compiled without GCS support

[kou]

How did you install your PyArrow? wheel? conda?

[KellyWalker]

Wheel file through poetry.

It is resolved to use pyarrow 12.0.1.

[kou]

Which wheel are you using?
Wheel list: https://pypi.org/project/pyarrow/#files

[KellyWalker]

pyarrow-12.0.1-cp38-cp38-win_amd64.whl

Listed here for 12.0.1: https://pypi.org/project/pyarrow/12.0.1/#files

[kou]

OK. pyarrow 13.0.0 will solve your problem.
See also: #35255/#35193

[KellyWalker]

I confirmed that it does. Thank you so much for the help.

I did search for the issue before I reported it, but maybe I was only looking at open tickets.