Bento plugins to transform geographic coordinates from a stream.
Bento is the swiss army of stream processing: Bento solves common data engineering tasks such as transformations, integrations, and multiplexing with declarative and unit testable configuration.
This repo contains multiple Bento plugins as Go modules, that you can build on demand (see cmd/geo-bento).
Note that the h3 plugin is using a CGO free version.
Use country with the following parameters: latitude, longitude.
An example position.json:
{"id":42, "lat": 48.86, "lng": 2.34}A country.yaml pipeline.
input:
file:
paths: ["testdata/position.json"]
codec: all-bytes
pipeline:
threads: 1
processors:
- mapping: |
#!blobl
root = this
root.country = country(this.lat, this.lng)
output:
label: "out"
stdout:
codec: linesEnrich the input with the timezone:
go build -o geo-0bento ./cmd/geo-bento
./geo-bento -c testdata/country.yaml
{"country":["France"],"id":42,"lat":48.86,"lng":2.34}country module is using coord2country.
Use h3 with the following parameters: latitude, longitude, resolution.
An example position.json:
{"id":42, "lat": 48.86, "lng": 2.34}A h3.yaml pipeline.
input:
file:
paths: ["testdata/position.json"]
codec: all-bytes
pipeline:
threads: 1
processors:
- mapping: |
#!blobl
root = this
root.h3 = h3(this.lat, this.lng, 5)
output:
label: "out"
stdout:
codec: linesEnrich the input with the h3 cell:
go build -o geo-bento ./cmd/geo-bento
./geo-bento -c testdata/h3.yaml
{"h3":"851fb467fffffff","id":42,"lat":48.86,"lng":2.34}Use geos2 with the following parameters: latitude, longitude, resolution.
An example position.json:
{"id":42, "lat": 48.86, "lng": 2.34}A s2.yaml pipeline.
input:
file:
paths: ["testdata/position.json"]
codec: all-bytes
pipeline:
threads: 1
processors:
- mapping: |
#!blobl
root = this
root.s2 = geos2(this.lat, this.lng, 15)
output:
label: "out"
stdout:
codec: linesEnrich the input with the s2 cell:
go build -o geo-bento ./cmd/geo-bento
./geo-bento -c testdata/s2.yaml
{"id":42,"lat":48.86,"lng":2.34,"s2":"2/033303031301002"}Use tz with the following parameters: latitude, longitude.
An example position.json:
{"id":42, "lat": 48.86, "lng": 2.34}A tz.yaml pipeline.
input:
file:
paths: ["testdata/position.json"]
codec: all-bytes
pipeline:
threads: 1
processors:
- mapping: |
#!blobl
root = this
root.tz = tz(this.lat, this.lng)
output:
label: "out"
stdout:
codec: linesEnrich the input with the timezone:
go build -o geo-bento ./cmd/geo-bento
./geo-bento -c testdata/tz.yaml
{"tz":"Europe/Paris","id":42,"lat":48.86,"lng":2.34}tz module is using tzf.
This is an input plugin that will generate random coordinates in your range.
input:
randpos:
min_lat: 46.0
max_lat: 48.0
min_lng: 2.0
max_lng: 2.3
output:
label: "out"
stdout:
codec: linesUse the random position generator, to use all plugins.
input:
randpos:
min_lat: 46.0
max_lat: 48.0
min_lng: 2.0
max_lng: 2.3
pipeline:
threads: 1
processors:
- mapping: |
#!blobl
root = this
root.h3 = h3(this.lat, this.lng, 12)
root.country = country(this.lat, this.lng)
root.s2 = geos2(this.lat, this.lng, 15)
root.tz = tz(this.lat, this.lng)
output:
label: "out"
stdout:
codec: linesRun this command and point your browser to http://localhost:4195/
./geo-bento blobl server --no-open --host 0.0.0.0 --input-file ./testdata/position.json -m testdata/s2_mapping.txtA pre built binary is also availale as a docker image:
docker pull ghcr.io/akhenakh/geo-bento:latest- s2 shape index to perform PIP
- spatialite lookup to perform PIP
- random points in a rect
- lat lng to h3
- lat lng to s2
- lat lng to tz
- lat lng to country