Figure out some integration with Cloud Bigtable?

[jgeewax]

@stephenplusplus : Any ideas ? Are there any popular existing HBase libraries for NodeJS?

https://www.npmjs.com/package/hbase-client (https://github.com/alibaba/node-hbase-client) seems to be the most popular.

https://www.npmjs.com/package/node-hbase (https://github.com/wdavidw/node-hbase) is also up to date, but not as popular on npm

[callmehiphop]

Can we get some more clarification on how we communicate with the remote server? Most of the documentation I'm seeing talks about setting up a local server

[jgeewax]

/cc @dhermes @maxluebbe @carterpage

[lesv]

Do you mean bigtable.googleapis.com? Or something else? I think all you should need is the server name and use the gRPC package. (HTTP/2) Am I misunderstanding the Q?

[callmehiphop]

@lesv yeah, I'm referring to bigtable.googleapis.com, the node clients we're looking at have some default settings I'm not sure apply to us and I didn't see any documentation specific to communicating with the remote server outside of command line usage.

[lesv]

Take a look at https://io2015codelabs.appspot.com/codelabs/gRPC it's a local critter, but I think it has how to query another server from it. If not, I can ask for you.

[callmehiphop]

@lesv thanks, will do!

[lesv]

Take a look at https://github.com/googlesamples/io2015-codelabs/blob/master/gRPC/complete/client.js#L5

[callmehiphop]

@lesv that definitely helps, thanks for linking me to that.

@jgeewax I don't think either of the node libraries mentioned support gRPC, however, one of them linked to a coffeescript library that has protobuf support. Do you know if the .proto file we need to use already exists and if so where?

[jgeewax]

Those libraries definitely won't support gRPC -- we'd need to swap out the transport layer to be gRPC.

Protos are here: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/tree/master/bigtable-protos (I think those are the right ones). You'll need protoc v3 :-/ ... aka, get ready to type make and such...

[dhermes]

FYI

I've done a bunch of work around getting the gRPC C/C++ core installed as well as making the calls to BigTable. I'm happy to have a chat if needed.

[callmehiphop]

@dhermes awesome, thanks! I may just take you up on that 😄

[callmehiphop]

I'm running into an issue (could be a user error) where I try to load bigtable_service.proto but the grpc module does not know how to interpret the rpc options.

JS

var grpc = require('grpc');

var bigtable = grpc.load({
  root: __dirname,
  file: '/google/bigtable/v1/bigtable_service.proto'
});

Ouput

Error: Illegal option value in message undefined, option (google.api.http) at line 37: {
    at Error (native)
    at ProtoBuf.DotProto.ParserPrototype._parseOption 

If I remove the rpc options everything works properly.

I'm sort of at a loss on how to proceed -- I was able to build the proto files in python, etc. via protoc but there's not a JavaScript option, I'm assuming because the standard is to just pass the .proto files directly to the module.

[dhermes]

@nathanielmanistaatgoogle @tbetbetbe Can you refer us to the folks working on the grpc Node library?

[stephenplusplus]

Thanks for all of the help, everyone!

Looks like protobuf.js doesn't understand this syntax:

option (google.api.http) = { post: "/v1/{table_name=projects/*/zones/*/clusters/*/tables/*}/rows:read" body: "*" };

But it does understand this syntax:

option (google.api.http).post = "/v1/{table_name=projects/*/zones/*/clusters/*/tables/*}/rows:read";

I had to change that in the proto files just to get grpc not to crash. I haven't made it further than that, so I don't know if it's going to cause problems down the line. Put it up in a repo here: https://github.com/stephenplusplus/bigtable-grpc-playground so we can keep charging forward.