Propagation of metadata
We are defining an internal ETL logic so that the metadata is propagation in a smart way from Snapshot to Grapher.
- We keep information about the origins throughout the data pipeline.
- We track how much a dataset has been processed and signal this in the metadata (minor/major processing).
- We track different operations applied to an indicator. For instance, "indicator
A
was obtained from indicatorB
divided by population size".