Client.insert_rows throws exception when coercing string to float from release 2.21.0

[Alan-Robinson]

The behaviour of Client.insert_rows changed between release 2.20.0 and 2.21.0.

In 2.20.0, where the schema defines a field to be of type FLOAT, a string passed in would be coerced to a float in the resulting JSON.

In 2.21.0, where the schema defines a field to be of type FLOAT, a string passed in throws an exception.

I've tracked this down to #728 , where a check is added for isnan or isinf. However, those functions require a float to be provided so throw an exception if a string is provided. This check occurs before you get to the code that coerces the input to a float.

def _float_to_json(value):
    """Coerce 'value' to an JSON-compatible representation."""
    if value is None:
        return None
    elif math.isnan(value) or math.isinf(value):
        return str(value)
    else:
        return float(value)

Environment details

• OS type and version:
• Python version: Python 3.6.10
• pip version: pip 21.2.1
• google-cloud-bigquery version: 2.22.1

Steps to reproduce

1. Setup table with a field defined to be FLOAT
2. Pass a string to Client.insert_rows() for that field

Code example

from google.cloud.bigquery import Client

client = Client(project='redacted')
client.insert_rows(table=client.get_table('python_test.test'), rows=[{'bar': "0.01"}])

Stack trace

  File "site-packages/google/cloud/bigquery/client.py", line 3329, in insert_rows
    json_rows = [_record_field_to_json(schema, row) for row in rows]
  File "site-packages/google/cloud/bigquery/client.py", line 3329, in <listcomp>
    json_rows = [_record_field_to_json(schema, row) for row in rows]
  File "site-packages/google/cloud/bigquery/_helpers.py", line 522, in _record_field_to_json
    record[subname] = _field_to_json(subfield, subvalue)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 589, in _field_to_json
    return _single_field_to_json(field, row_value)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 565, in _single_field_to_json
    return _scalar_field_to_json(field, row_value)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 464, in _scalar_field_to_json
    return converter(row_value)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 345, in _float_to_json
    elif math.isnan(value) or math.isinf(value):
TypeError: must be real number, not str

[tswast]

Does insert_rows_json work for you? It doesn't do any type conversion.

[plamut]

I was able to reproduce this and it's indeed the change in in #728 that caused this.

Previously, the string values were simply converted to floats, but then extra checks were added to handle special float values (Nan, inf...). Unfortunately, those checks do not like strings.

I'll submit a fix soon.