BigQuery: support `/upload` method for loading data from local files.

[tswast]

See documentation: https://cloud.google.com/bigquery/loading-data-post-request

Python, PHP, Ruby, and Node.js provide an upload-from-file method on the Table object.

This is blocking documentation samples.

[tswast]

Actually, I see we have a LoadJobConfiguration. http://googlecloudplatform.github.io/google-cloud-java/0.5.1/apidocs/com/google/cloud/bigquery/LoadJobConfiguration.html That might be sufficient.

[mziccard]

Tim have a look at BigQuery.writer(), it allows you to upload a file in chunks (generally a good idea as they tend to be quite big files). It should be rather simple to write a whole file using it.

[tswast]

Thanks for the pointer.

[mziccard]

Can we close this?

[tswast]

Is there a way to get the Job the writer must create? The Python version is able to get the number of rows written in it's sample.

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/bigquery/cloud-client/load_data_from_file.py#L51

I am trying to improve the snippets for this (there are no integration tests) and having the returned data from the Job would be really useful.

[tswast]

Or even getting the Job ID would be useful. Then I can fetch the job results after finishing my writes.

[mziccard]

Is there a way to get the Job the writer must create? The Python version is able to get the number of rows written in it's sample.

I need to look into this. I don't know any way of getting the job metadata/id when using resumable upload: the open request just returns the Upload ID, all subsequent POST/PUT seem to have empty response body.

[tswast]

Thanks. I'll dig into it, too.

[tswast]

My current thought is making BigQueryRpc.open() return a Job instead of a String. I will attempt to get a PR out for this.

[mziccard]

Tim I was wrong. The last call of a chunked upload returns all the metadata of the just created job. We can put #1394 on hold until I fix this. I will probably make TableDataWriteChannel public and add a getJob() method that returns the job associated to the upload, only once the channel is closed (i.e. the upload was finalized).

[tswast]

@mziccard Thanks. That sounds like a great solution.