Skip to content

Automate GBIF filtering, triggered by new GBIF data, including name generation #77

@zzeppozz

Description

@zzeppozz

New GBIF data in AWS should trigger events in Redshift:

  • generate dataset date, used in GBIF data path and SpNetwork name
  • mount full GBIF dataset
  • filter to appropriate dataset for Sp Network
  • dismount full GBIF dataset
  • summarize data contents by dataset to tables
  • write to S3
  • delete previous month's SpNetwork dataset and summary data tables

Metadata

Metadata

Assignees

Labels

AnalystTasks related to computationsenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions