ETL job to migrate an organization from prod database to test and dev database

[dadiorchen]

We need a function/job/command to incrementally transport the data in our prod database that belongs to a specified organization on Greenstand, and do some transform, then dump into our dev database.

The data need to be transported are:

• New planter under the org
• New capture belongs to the org
• New species created by these tree.
  *New token minted on capture above.
  * Wallet that occupy token above.

Language, could by Python or Node.js.

[Xinyihe123]

I'll work on this one.

[jersey1dev]

I will take up the task of adding additional functionalities for this issue.

[dadiorchen]

@Ayansola Akanmu question:

Answer:

1. organzations has been deprecated, the organization info is saved in this way: entity table and type column equal to O (letter o) so the organzation_id in table planter is linked to this entity table by primary key id.
2. planters table is deprecated, the planter is valid. As an individual planter ( comparing to planter owned by organization), its organization_id is NULL, so that's fine.
3. To find the tree belongs to an org, we need: join trees.planter_id with planter.id, join planter.organzation_id with entity.id

There are some chaos, one reason is that the dev db is tweaked by devs from time to time, and we need to do some clean up, another reason is we are not going to maintain this table because we are migrating to new db domain, those table in new domain is arranged by schema, you can find them on all those schema for example reporting

[sole1907]

thanks @dadiorchen - I'll work on this issue

[dadiorchen]

to Shikhar Sharma:
The goal is to copy trees under specified org to dev db, the tree data is in trees table, but you need to filter the data by org because it is too many.
The relationship is:
trees.planter_id --> planter.id
planter.organization_id --> entity.id

so we specify a entity/org id, say, 123 in the script, then we will copy all the data in planter that belongs to org 123, and all the trees that belongs to all the planters.