Read protobufs from BigQuery with protobq.MessageLoader.
Protobuf messages can be written directly to BigQuery using the storage write API or Pub/Sub BigQuery subscriptions.
BigQuery schemas for protobuf messages can be generated with protoc-gen-bq-schema, and Pub/Sub schemas can be generated with protoc-gen-pubsub.
protobq.MessageLoader implements the bigquery.ValueLoader interface, and uses the protobuf reflection APIs to reconstruct a protobuf message as it was written to BigQuery.
The goal of this library is simple:
If you were able to put the protobuf message in, protobq.MessageLoader should be able to get it out.
For information about type mappings, see docs/types.md.
package main
import (
"context"
"os"
"cloud.google.com/go/bigquery"
"github.com/way-platform/protobq-go"
"google.golang.org/genproto/googleapis/example/library/v1"
)
func main() {
// 1. Connect to BigQuery.
ctx := context.Background()
client, err := bigquery.NewClient(ctx, os.Getenv("GOOGLE_CLOUD_PROJECT"))
if err != nil {
panic(err)
}
defer client.Close()
// 2. Run a query.
query := client.Query(`
SELECT
"George Orwell" as author,
"1984" as title,
`)
it, err := query.Read(ctx)
if err != nil {
panic(err)
}
// 3. Load the result into a protobuf message.
var book library.Book
if err := it.Next(&protobq.MessageLoader{
Message: &book,
}); err != nil {
panic(err)
}
// 4. Profit.
}This SDK is published under the MIT License.
Security researchers, see the Security Policy.
Be nice. For more info, see the Code of Conduct.