_PARTITIONTIME is not returned in version 3.11.4 (is returned in 3.11.3)

[SudhirNune]

_PARTITIONTIME is not returned in 3.11.4 Verion works till 3.11.3

Environment details

• OS type and version: Windows & UNIX
• Python version: Python 3.8.10
• pip version: pip 23.2.1
• google-cloud-bigquery version:
  Name: google-cloud-bigquery
  Version: 3.11.3
  Summary: Google BigQuery API client library
  Home-page: https://github.com/googleapis/python-bigquery
  Author: Google LLC
  Author-email: googleapis-packages@google.com
  License: Apache 2.0
  Location: c:\users\hrq1364\envs\khde\lib\site-packages
  Requires: google-api-core, google-cloud-core, google-resumable-media, grpcio, packaging, proto-plus, protobuf, python-dateutil, requests
  Required-by: dbt-bigquery, kh-de, pandas-gbq

Steps to reproduce

1. Install Version: 3.11.3 of google-cloud-bigquery
2. Import the Package
3. Connect to the BQ with Service Account
4. Create Cursor
5. Run the query (Using the Above) -

SELECT export_time, pricing_as_of_time, product_taxonomy,
billing_account_id, business_entity_name, service, sku, geo_taxonomy,
pricing_unit, pricing_unit_description, account_currency_code,
currency_conversion_rate, list_price, billing_account_price, _PARTITIONTIME,
CAST(TIMESTAMP_TRUNC(export_time, DAY) AS DATE) AS export_date
FROM `billing-us-01.us_billing_data.cloud_pricing_export`
WHERE (_PARTITIONTIME > '2023-09-04 00:00:00')

7. Output has the column "_PARTITIONTIME"
8. Install Version: 3.11.4 of google-cloud-bigquery
9. Import the Package
10. Connect to the BQ with Service Account
11. Create Cursor
12. Run the query (Using the Above) -

SELECT export_time, pricing_as_of_time, product_taxonomy,
billing_account_id, business_entity_name, service, sku, geo_taxonomy,
pricing_unit, pricing_unit_description, account_currency_code,
currency_conversion_rate, list_price, billing_account_price, _PARTITIONTIME,
CAST(TIMESTAMP_TRUNC(export_time, DAY) AS DATE) AS export_date
FROM `billing-us-01.us_billing_data.cloud_pricing_export`
WHERE (_PARTITIONTIME > '2023-09-04 00:00:00')

14. Output Missing the column "_PARTITIONTIME"

Thanks!

[chalmerlowe]

@SudhirNune

I have not been able to recreate this error locally. I will likely close this as not reproducible/won't fix.

I ran analogous code on a partitioned table using both 3.11.3 and 3.11.4 and successfully returned results...

Below is an example with _PARTITIONTIME results in version 3.11.4 based on the code shown below:

Started job: d7f9a02d-531c-46ff-9418-6d5d15800032
3.11.4
Row((datetime.datetime(2014, 8, 2, 22, 15, tzinfo=datetime.timezone.utc), 15.25, None, datetime.datetime(2023, 9, 20, 14, 0, tzinfo=datetime.timezone.utc)), {'timestamp': 0, 'fare': 1, 'company': 2, '_PARTITIONTIME': 3})
Row((datetime.datetime(2014, 7, 1, 23, 45, tzinfo=datetime.timezone.utc), 7.05, None, datetime.datetime(2023, 9, 20, 14, 0, tzinfo=datetime.timezone.utc)), {'timestamp': 0, 'fare': 1, 'company': 2, '_PARTITIONTIME': 3})
Row((datetime.datetime(2014, 7, 21, 15, 0, tzinfo=datetime.timezone.utc), 8.05, None, datetime.datetime(2023, 9, 20, 14, 0, tzinfo=datetime.timezone.utc)), {'timestamp': 0, 'fare': 1, 'company': 2, '_PARTITIONTIME': 3})

Code sample:

query = """
SELECT timestamp, fare, company, _PARTITIONTIME,
FROM `<your_project_name_here>.taxidata.time_fare_company_part3`
WHERE (_PARTITIONTIME > '2023-01-04 00:00:00')
LIMIT 3
"""

from google.cloud import bigquery
from google.cloud.bigquery import version

# Construct a BigQuery client object.
project = '<your_project_name_here>'

client = bigquery.Client()

query_job = client.query(
    query=query,
    location="US",
    project=project,
)

print("Started job: {}".format(query_job.job_id))
print(version.__version__) # show python version being used.
for result in query_job.result():
    print(result)

[chalmerlowe]

NOTE: this linked issue also discusses dropping _PARTITIONTIME, but does not appear to have any relationship to package version, the way this issue does.

[chalmerlowe]

Closing as not reproducible.