cowid#
COVID-19 Data pipeline tool by Our World in Data.
cowid [OPTIONS] COMMAND [ARGS]...
Options
- --parallel, --no-parallel#
Parallelize process.
- Default:
True
- --n-jobs <n_jobs>#
Number of threads to use.
- Default:
-2
- -S, --server, --no-server#
Only critical log and final message to slack.
- Default:
False
check#
COVID-19 data pipeline checks.
cowid check [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
hosp#
Upload dataset to DB.
cowid check hosp [OPTIONS]
jhu#
Upload dataset to DB.
cowid check jhu [OPTIONS]
megafile#
Upload dataset to DB.
cowid check megafile [OPTIONS]
test#
Upload dataset to DB.
cowid check test [OPTIONS]
vax#
Generate dataset.
cowid check vax [OPTIONS]
decoupling#
COVID-19 Decoupling data pipeline.
cowid decoupling [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Download and generate our COVID-19 Decoupling dataset.
cowid decoupling generate [OPTIONS]
grapher-db#
Download and generate our COVID-19 Decoupling dataset.
cowid decoupling grapher-db [OPTIONS]
gmobility#
Google Mobility data pipeline.
cowid gmobility [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Download and generate our COVID-19 Hospitalization dataset.
cowid gmobility generate [OPTIONS]
grapher-db#
Step 3: Update Grapher database with generated files.
cowid gmobility grapher-db [OPTIONS]
grapher-io#
Step 2: Generate grapher-ready files.
cowid gmobility grapher-io [OPTIONS]
hosp#
COVID-19 Hospitalization data pipeline.
cowid hosp [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Runs scraping scripts to collect the data from the primary sources, transforms it and exports the result to public/data/hospitalizations/.
By default, the default values for OPTIONS are those specified in the configuration file. The configuration file is a YAML file with the pipeline settings. Note that the environment variable OWID_COVID_CONFIG must be pointing to this file. We provide a default config file in the project folder scripts/config.yaml.
OPTIONS passed via command line will overwrite those from configuration file.
cowid hosp generate [OPTIONS] [COUNTRIES]...
Options
- -s, --skip-countries <skip_countries>#
List of countries to skip (comma-separated)
Arguments
- COUNTRIES#
Optional argument(s)
grapher-db#
Step 3: Update Grapher database with generated files.
cowid hosp grapher-db [OPTIONS]
grapher-io#
Step 2: Generate grapher-ready files.
cowid hosp grapher-io [OPTIONS]
jhu#
COVID-19 Cases/Deaths data pipeline.
cowid jhu [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Step 2: Generate dataset.
cowid jhu generate [OPTIONS]
get#
Downloads all JHU source files into project directory.
cowid jhu get [OPTIONS]
grapher-db#
Step 3: Update Grapher database with generated files.
cowid jhu grapher-db [OPTIONS]
megafile#
COVID-19 data integration pipeline (former megafile)
cowid megafile [OPTIONS]
oxcgrt#
COVID-19 stringency index (by OxCGRT) data pipeline.
cowid oxcgrt [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
get#
Downloads all OxCGRT source files into project directory.
cowid oxcgrt get [OPTIONS]
grapher-db#
Step 3: Update Grapher database with generated files.
cowid oxcgrt grapher-db [OPTIONS]
grapher-io#
Step 2: Generate grapher-ready files.
cowid oxcgrt grapher-io [OPTIONS]
sweden#
COVID-19 Sweden data pipeline.
cowid sweden [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Generate our COVID-19 Sweden dataset.
cowid sweden generate [OPTIONS]
get#
Download source file for Sweden dataset.
cowid sweden get [OPTIONS]
grapher-db#
Download and generate our COVID-19 Decoupling dataset.
cowid sweden grapher-db [OPTIONS]
test#
COVID-19 Testing data pipeline.
cowid test [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
get#
Runs scraping scripts to collect the data from the primary sources of COUNTRIES. Data is exported to project folder scripts/output/testing/. By default, all countries are scraped.
By default, the default values for OPTIONS are those specified in the configuration file. The configuration file is a YAML file with the pipeline settings. Note that the environment variable OWID_COVID_CONFIG must be pointing to this file. We provide a default config file in the project folder scripts/config.yaml.
OPTIONS passed via command line will overwrite those from configuration file.
- Examples:
Run the step using default values, from config.yaml file: cowid test get
Run the step only for Australia: cowid test get australia
Run the step for all countries except Australia: cowid test get -s australia all
Run the step for all incremental processes (can be also done using ‘batch’): cowid test get incremental
cowid test get [OPTIONS] [COUNTRIES]...
Options
- -s, --skip-countries <skip_countries>#
List of countries to skip (comma-separated).
Arguments
- COUNTRIES#
Optional argument(s)
uk-nations#
COVID-19 UK Nations data pipeline.
cowid uk-nations [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Generate dataset.
cowid uk-nations generate [OPTIONS]
grapher-db#
Upload dataset to DB.
cowid uk-nations grapher-db [OPTIONS]
variants#
COVID-19 Variants data pipeline.
cowid variants [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Download and generate our COVID-19 Hospitalization dataset.
cowid variants generate [OPTIONS]
grapher-io#
Step 2: Generate grapher-ready files.
cowid variants grapher-io [OPTIONS]
vax#
COVID-19 Vaccination data pipeline.
cowid vax [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
export#
Step 4: Export vaccination data and merge with global dataset.
cowid vax export [OPTIONS]
generate#
Step 3: Generate vaccination dataset.
cowid vax generate [OPTIONS]
get#
Runs scraping scripts to collect the data from the primary sources of COUNTRIES. Data is exported to project folder scripts/output/vaccinations/. By default, all countries are scraped.
By default, the default values for OPTIONS are those specified in the configuration file. The configuration file is a YAML file with the pipeline settings. Note that the environment variable OWID_COVID_CONFIG must be pointing to this file. We provide a default config file in the project folder scripts/config.yaml.
OPTIONS passed via command line will overwrite those from configuration file.
Examples:
Run the step using default values, from config.yaml file: cowid vax get
Run the step only for Australia: cowid vax get australia
Run the step for all countries except Australia: cowid vax get -s australia all
Run the step for all incremental processes (can be also done using ‘batch’): cowid vax get incremental
cowid vax get [OPTIONS] [COUNTRIES]...
Options
- -s, --skip-countries <skip_countries>#
List of countries to skip (comma-separated)
- -O, --optimize, --no-optimize#
Optimize processes based on older logging times.
- Default:
False
Arguments
- COUNTRIES#
Optional argument(s)
icer#
Download some specific country files. Useful when these are very large in size.
cowid vax icer [OPTIONS]
process#
Process data in folder scripts/output/vaccinations/.
By default, the default values for OPTIONS are those specified in the configuration file. The configuration file is a YAML file with the pipeline settings. Note that the environment variable OWID_COVID_CONFIG must be pointing to this file. We provide a default config file in the project folder scripts/config.yaml.
OPTIONS passed via command line will overwrite those from configuration file.
Example: Process all country data.
cowid vax process
cowid vax process [OPTIONS]
track#
Explore high-level analytics of vaccination dataset.
cowid vax track [OPTIONS]
Options
- --output <output>#
Output CSV file.
- Default:
cowid-vax-track.report.tmp.csv
- -d, --disable-export <disable_export>#
Do not export CSV file, just logging results.
- Default:
False
us-states#
US vaccinations data pipeline.
cowid vax us-states [OPTIONS]
xm#
COVID-19 Excess Mortality data pipeline.
cowid xm [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
generate#
Download and generate our COVID-19 Hospitalization dataset.
cowid xm generate [OPTIONS]