Skip to content

Allow users to configure sources (Datasets Only) #10

Description

@ylyangtw
  • Refactor earthcube utility to read tenant config from minio
  • Add a sensor feature to fetch the config from the minio
  • Once a config change is detected, the sensor triggers a run of data pipeline automatically
  • A google form for the inputs
  • Have a script to validate the URLs and jsonlds
  • A script to transform google sheet to sitemap and upload it to s3 storage
  • Add the script to data pipeline in scheduler
  • Have a sensor to detect changes in the google sheet then trigger the run in scheduler

Metadata

Metadata

Assignees

Type

No type

Projects

Status
In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions