Global food trade data¶
Pablo Rosado • July 1, 2026 (last edit) • Feedback
Introduction¶
This is a technical companion to our article "How does food get traded around the world?". It examines FAOSTAT's Detailed trade matrix (TM), and explains the methodology and choices behind our visualizations.
You can run this notebook on Google Colab, or download the file to run it on your computer.
Definitions¶
Before we delve into the data, let's first define a few key concepts.
An item is a food product, either a primary commodity such as wheat or bananas, or a processed product such as meat.
An item flow is a trade exchange, namely an import or an export, of an item between two countries. Each flow can be measured as a quantity (in tonnes, or number of animals) and as a monetary value (in dollars). Here we will analyse quantities of traded food (in tonnes).
Each entry in the TM dataset corresponds to a trade flow reported by a certain country and year, with a given partner country. For example, in one entry, Brazil reports exporting ~4 million tonnes of soybeans to Argentina in 2023.
Because each flow links a specific pair of countries (the reporter and its partner), this is known as bilateral trade data: it describes trade between two named countries, not a country's total trade with the rest of the world. The same flow can therefore appear twice, reported independently by the exporting country and by the importing country.
When a country does not report a flow, but its partner country does, the partner's version is called mirror data.
Set up¶
If you want to run the code in this file and replicate its results locally or on Google Collab, there are some libraries you may need to install and import.
%%capture
# Whether you run this code in your local computer or on Google Colab, you first need to install the required dependencies:
# * owid-catalog: a library that lets you explore and load datasets from OWID.
# * plotly: a library that lets you create interactive visualizations.
%pip install --upgrade owid-catalog plotly
import contextlib
import io
import numpy as np
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
from owid.catalog import fetch
# Load published tables from OWID's public catalog, so this notebook runs anywhere (e.g. Colab)
# without a local ETL checkout. fetch() returns an indexed Table; reset_index() gives plain columns.
# fetch() also prints an animated spinner that nbconvert captures as many lines, so silence stdout while loading.
#
# The detailed trade matrix has ~52 million rows. Loading it through fetch() transiently peaks at ~8 GB of
# RAM, enough to crash a free Colab kernel. We instead read its published feather directly, which keeps the
# compact categorical dtypes and peaks at ~3 GB. The other (far smaller) tables load fine via fetch().
CATALOG_URL = "https://catalog.ourworldindata.org"
with contextlib.redirect_stdout(io.StringIO()):
tb = pd.read_feather(f"{CATALOG_URL}/garden/faostat/2026-05-07/faostat_tm/faostat_tm.feather")
tb_food_trade = fetch("garden/faostat/2026-05-07/food_trade/food_trade").reset_index()
reg = fetch("garden/regions/2023-01-01/regions/regions").reset_index()
# Map country names to ISO-3 codes, used by the reporting-rate choropleth (regions table from the catalog).
country_to_iso3 = dict(
zip(
reg.loc[reg["region_type"] == "country", "name"].astype(str),
reg.loc[reg["region_type"] == "country", "iso_alpha3"].astype(str),
)
)
# A few selected items combine several FAO codes (a primary commodity plus a mechanically-derived
# form whose trade FAO reports separately). The food_trade step sums them; this mirrors the
# multi-code entries in food_trade.items.yaml. Maps display name -> {FAO code: short form label}.
COMBINED_ITEMS = {
"Beef": {867: "bone-in", 870: "boneless"},
"Pork": {1035: "bone-in", 1038: "boneless"},
"Almonds": {221: "in-shell", 231: "shelled"},
"Cashews": {217: "in-shell", 230: "shelled"},
"Walnuts": {222: "in-shell", 232: "shelled"},
"Hazelnuts": {225: "in-shell", 233: "shelled"},
"Brazil nuts": {216: "in-shell", 229: "shelled"},
"Groundnuts": {242: "in-shell", 243: "shelled"},
"Rice": {31: "milled", 32: "broken", 28: "husked"},
"Sugar": {162: "raw", 164: "refined"},
}
# Map each selected FAO code to its display name. Combined items appear in the published table under
# a synthetic id (100000 + first code), so we expand them back to their underlying FAO codes here,
# letting analyses that join on the trade matrix's own codes still cover every selected item.
selected_code_to_display = {}
for code, item in dict(zip(tb_food_trade["item_code"].astype(int), tb_food_trade["item"].astype(str))).items():
if item in COMBINED_ITEMS:
for member in COMBINED_ITEMS[item]:
selected_code_to_display[member] = item
else:
selected_code_to_display[code] = item
Basic data exploration¶
The TM dataset has ~52 million entries, and over 500 unique items. The items are agricultural products, both food and non-food commodities (from wheat, meat and chocolate to cotton, tobacco and rubber). Fish and other seafood are not included in the dataset.
Many of those food items are very specific, e.g. "Edible offal of pigs, fresh, chilled or frozen", or uncommon, e.g. "kola nuts". To avoid clutter, we created a selected list of items. However, in most of the present analysis we consider all items, except when we explicitly mention that we use the selected list.
We select only entries corresponding to item quantities (in tonnes, not dollars or animals). Around 12% of rows correspond to trade of zero tonnes (which we keep in the data).
# Total number of entries
assert len(tb) > 52e6
# Basic data preparation: restrict to quantity reports in tonnes (this drops the
# alternate-unit rows used for live animals — 'An', '1000 An', 'No' — and the
# value-in-USD rows),
tb = tb[(tb["unit"] == "t")].reset_index(drop=True)
assert len(tb) > 25e6
# Drop self-trade rows, where reporter and partner are the same country (~0.1% of the data).
# These are a residual data artifact confined to older years: they appear only up to 2013 and
# never in the years we analyse (e.g. 2023), so dropping them does not affect any result shown.
self_trade_mask = tb["reporter_country"].astype(str) == tb["partner_country"].astype(str)
assert 0.1 < 100 * len(tb[self_trade_mask]) / len(tb) < 0.2
assert (tb.loc[self_trade_mask, "year"] <= 2013).all(), "Self-trade now affects years after 2013; revisit whether it can be silently dropped."
tb = tb[~self_trade_mask].reset_index(drop=True)
# Share of rows with zero tonnes reported.
assert 100 * len(tb[tb["value"] == 0]) / len(tb) > 12
FAOSTAT flags each quantity row with its provenance. In general, quantity (in tonnes) and value (in dollars) are independently collected in the TM dataset.
About 98% of rows in the TM dataset are flagged "Official figure"; the remaining ~2% are flagged as imputed, estimated, or sourced from an external organization.
We keep all rows regardless of flag.
# Check that there are more than 500 items.
assert len(tb["item_code"].unique()) > 500
# Share of rows flagged "Official figure" across all years (claimed: ~98%).
official_share = (tb[tb["element"].isin(["Import quantity", "Export quantity"])]["flag"] == "Official figure").mean() * 100
assert 97 < official_share < 99
Yearly coverage¶
We would ideally show food trade figures for the most recent year available in the data. However, as the following chart shows, and as of the day when this analysis was created, the latest year is significantly affected by incomplete reporting.
def plot_coverage(tb):
"""Number of distinct reporting countries per year. A drop in the most recent year(s)
flags incomplete reporting, which is why we avoid using the latest year."""
reporters = tb.groupby("year", observed=True)["reporter_country"].nunique().sort_index()
years = reporters.index.astype(int).tolist()
fig = go.Figure()
fig.add_bar(x=years, y=reporters.values, opacity=0.8)
fig.update_layout(
title="Number of distinct reporting countries by year",
xaxis=dict(title="Year"),
yaxis=dict(title="Number of distinct reporting countries"),
showlegend=False,
)
fig.show()
plot_coverage(tb)
So, in the rest of the analysis, we focus on the latest year with a reasonably complete coverage, namely 2023.
# Assert current maximum year and define the one we use for the rest of the article.
assert tb["year"].max() == 2024
# Year to use for year-specific analyses from now on.
YEAR = 2023
Reporting issues¶
Every trade flow has two sides. When Brazil ships soybeans to Argentina, Brazil can record it as an export, and Argentina can record it as an import. So you might expect the two countries to tell the same story: the same shipment, the same number of tonnes.
And sometimes they do. Brazil reports exporting around 4 million tonnes of soybeans to Argentina in 2023, and Argentina reports importing roughly the same amount from Brazil that year.
However, this is very often not the case.
- Item flows are often reported only one way; sometimes the importer, and sometimes the exporter.
- When both countries report the same flow, there are often significant disagreements in the reported quantities.
These differences can arise even when both countries follow the same international guidelines, due to timing, partner country attribution, confidentiality, and other factors. FAOSTAT explicitly notes in their Detailed trade matrix methodology note that "the trade matrix data are un-reconciled".
For example, in 2023 Ukraine reports exporting ~2 million tonnes of wheat to Romania, but Romania only reports importing ~200 thousand tonnes (a factor of ~10 apart).
According to FAOSTAT's Food balance sheets and supply utilization accounts resource handbook 2025:
[...] imports are typically documented more thoroughly and verified more rigorously than exports.
This claim cites the United Nations Statistics Division's International Merchandise Trade Statistics, which mentions:
Because of the greater customs scrutiny to which imports are subjected by customs in most countries, it is usually more feasible to derive estimates of exports from counterpart imports.
[...] Also, the customs administration is generally more interested in the quantity information for imports than in that for exports, since quantity information is, in some cases, utilized to determine both import duties and the unit values used to validate the price and value information declared by the importers.
However they also note that, for certain commodities and in some countries, export data were viewed as being more accurate for the same reasons.
With all this, we therefore adopt the following simple criterion:
- If only one country reports the flow, we use that country's quantity.
- If both sides report the flow, we use the importer-reported quantity.
But note that there is unfortunately no simple way to reconcile reported quantities in bilateral trade flows. For broader context on why bilateral trade reports disagree, see our topic page on trade and globalization, specifically the section on trade data discrepancies.
One-sided versus two-sided reporting¶
For each item flow in a year, we count how many times that flow was reported both ways ("matched") or only one way ("exporter-only" or "importer-only"). Here, "matched" means both countries reported the flow, regardless of whether their reported quantities agree (we compare the quantities later). The result is shown in the following chart.
def plot_reporting_coverage_by_year(tb):
"""Stacked bar chart per year showing what fraction of bilateral flows
are reported by both sides (matched), only by the exporter, or only by
the importer."""
qty = tb[tb["element"].isin(["Export quantity", "Import quantity"])][
["reporter_country", "partner_country", "item_code", "year", "element"]
].copy()
for col in ("reporter_country", "partner_country"):
qty[col] = qty[col].astype(str)
exp_keys = qty.loc[
qty["element"] == "Export quantity",
["reporter_country", "partner_country", "item_code", "year"],
].drop_duplicates()
imp_keys = (
qty.loc[
qty["element"] == "Import quantity",
["reporter_country", "partner_country", "item_code", "year"],
]
.rename(columns={"reporter_country": "partner_country", "partner_country": "reporter_country"})
.drop_duplicates()
)
merged = exp_keys.merge(
imp_keys, how="outer", indicator=True,
on=["reporter_country", "partner_country", "item_code", "year"],
)
by_year = (
merged.groupby("year", observed=True)["_merge"]
.value_counts(normalize=True)
.unstack(fill_value=0.0)
.rename(columns={"both": "matched", "left_only": "exporter-only", "right_only": "importer-only"})
)
by_year = by_year[["matched", "exporter-only", "importer-only"]].reset_index()
long = by_year.melt(id_vars="year", var_name="status", value_name="share")
fig = px.bar(
long, x="year", y="share", color="status",
title="Most trade flows are reported by only one country<br><sup>Share of bilateral flows reported by both sides, only the exporter, or only the importer, by year</sup>",
labels={"year": "Year", "share": "Share of item flows"},
category_orders={"status": ["matched", "exporter-only", "importer-only"]},
)
fig.update_layout(barmode="stack", yaxis_tickformat=".0%")
fig.show()
plot_reporting_coverage_by_year(tb)
We can see that the reporting inconsistency is quite severe. Most item flows are reported only one way (only the importer, or only the exporter). In 2023, only around 40% of item exchanges were reported both ways.
But flows are not all the same size. The chart above counts each flow equally, regardless of how many tonnes were traded. In the chart below, we weight each flow by tonnage (and, when both countries report, we assume the importer's quantity). Now we see that most of the traded tonnes are reported by both sides; around 82% in 2023.
def plot_reporting_coverage_tonnage_by_year(tb):
"""Same as the reporting-coverage chart, but weighting each flow by the importer-reported
tonnage (falling back to the exporter's when the importer is silent, the rule the final flow
table uses) instead of counting flows equally."""
qty = tb[tb["element"].isin(["Export quantity", "Import quantity"])][
["reporter_country", "partner_country", "item_code", "year", "element", "value"]
].copy()
for col in ("reporter_country", "partner_country"):
qty[col] = qty[col].astype(str)
exp = (
qty[qty["element"] == "Export quantity"]
.groupby(["reporter_country", "partner_country", "item_code", "year"], observed=True)["value"].sum()
.reset_index().rename(columns={"reporter_country": "exporter", "partner_country": "importer", "value": "v_exp"})
)
imp = (
qty[qty["element"] == "Import quantity"]
.groupby(["reporter_country", "partner_country", "item_code", "year"], observed=True)["value"].sum()
.reset_index().rename(columns={"reporter_country": "importer", "partner_country": "exporter", "value": "v_imp"})
)
flows = exp.merge(imp, on=["exporter", "importer", "item_code", "year"], how="outer")
has_exp, has_imp = flows["v_exp"].notna(), flows["v_imp"].notna()
flows["tonnage"] = flows["v_imp"].fillna(flows["v_exp"])
flows["status"] = np.where(has_exp & has_imp, "matched", np.where(has_exp, "exporter-only", "importer-only"))
year_total = flows.groupby("year", observed=True)["tonnage"].sum()
by_year = flows.groupby(["year", "status"], observed=True)["tonnage"].sum().reset_index()
by_year["share"] = by_year["tonnage"] / by_year["year"].map(year_total)
fig = px.bar(
by_year, x="year", y="share", color="status",
title="But most traded tonnage is reported by both countries<br><sup>Share of traded tonnage reported by both sides, only the exporter, or only the importer, by year</sup>",
labels={"year": "Year", "share": "Share of traded tonnes"},
category_orders={"status": ["matched", "exporter-only", "importer-only"]},
)
fig.update_layout(barmode="stack", yaxis_tickformat=".0%")
fig.show()
plot_reporting_coverage_tonnage_by_year(tb)
# Simple code to double-check that result for 2023 alone.
exporter_reports = tb[(tb["year"] == YEAR) & (tb["element"] == "Export quantity")].rename(columns={"reporter_country": "exporter", "partner_country": "importer"})
importer_reports = tb[(tb["year"] == YEAR) & (tb["element"] == "Import quantity")].rename(columns={"reporter_country": "importer", "partner_country": "exporter"})
exporter_reports_set = set(exporter_reports["exporter"].astype("string") + exporter_reports["importer"].astype("string") + exporter_reports["item"].astype("string"))
importer_reports_set = set(importer_reports["exporter"].astype("string") + importer_reports["importer"].astype("string") + importer_reports["item"].astype("string"))
# Complete set of item flows reported (by importers, exporters, or both).
all_flows_set = importer_reports_set | exporter_reports_set
# Share of all flows that are reported by importers only.
assert round(100 * len(importer_reports_set - exporter_reports_set) / len(all_flows_set)) == 31
# Share of all flows that are reported by exporters only.
assert round(100 * len(exporter_reports_set - importer_reports_set) / len(all_flows_set)) == 27
# Share of all flows that are reported by both importers and exporters:
assert round(100 * len(exporter_reports_set & importer_reports_set) / len(all_flows_set)) == 42
Larger flows tend to be reported by both countries, while small ones are often one-sided. A handful of bulk staples make up most of the traded tonnage, the kind of large flows that are usually well reported.
Reporting by item¶
Reporting completeness also varies widely from item to item. From our selected list of items, here are the 15 with the highest and the 15 with the lowest share of flows reported by both countries:
def plot_reporting_coverage_by_item(tb, year, n_worst=None, n_best=None):
"""Horizontal bar chart of the share of each selected item's bilateral flows that are
reported by both sides ("matched"), for the given year.
A low matched share means most of that item's flows are reported by only one country.
By default all selected items are shown, sorted worst-first. Pass n_worst and/or n_best
to keep only the n items with the lowest and/or highest matched share (e.g. n_worst=15,
n_best=15 shows the two extremes with the middle omitted)."""
code_to_display = selected_code_to_display
sub = tb[
(tb["year"] == year)
& tb["item_code"].isin(code_to_display)
& tb["element"].isin(["Export quantity", "Import quantity"])
].copy()
# Collapse to display-item level so combined items (e.g. bone-in + boneless beef) form a single bar.
sub["item"] = sub["item_code"].map(code_to_display)
sub["reporter_country"] = sub["reporter_country"].astype(str)
sub["partner_country"] = sub["partner_country"].astype(str)
cols = ["reporter_country", "partner_country", "item"]
exp_keys = sub.loc[sub["element"] == "Export quantity", cols].drop_duplicates()
# Swap reporter/partner on the import side so an export R->P matches the importer P reporting it.
imp_keys = (
sub.loc[sub["element"] == "Import quantity", cols]
.rename(columns={"reporter_country": "partner_country", "partner_country": "reporter_country"})
.drop_duplicates()
)
merged = exp_keys.merge(imp_keys, how="outer", indicator=True, on=cols)
matched = (
merged.groupby("item", observed=True)["_merge"]
.apply(lambda s: (s == "both").mean())
.sort_values(ascending=True)
)
total_items = len(matched)
# Keep the requested extremes (worst at the bottom, best at the top), or all items.
if n_worst or n_best:
parts = ([matched.head(n_worst)] if n_worst else []) + ([matched.tail(n_best)] if n_best else [])
shown = pd.concat(parts)
shown = shown[~shown.index.duplicated(keep="first")]
else:
shown = matched
if n_worst and n_best:
title = f"Food items most and least often reported by both countries, {year}"
subtitle = "From our selected list of items"
elif n_worst:
title = f"The {len(shown)} food items with the most one-sided trade reporting in {year}"
subtitle = "Share of each item's flows reported by both countries."
elif n_best:
title = f"The {len(shown)} food items with the most two-sided trade reporting in {year}"
subtitle = "Share of each item's flows reported by both countries."
else:
title = f"Share of each food item's trade flows reported by both sides in {year}"
subtitle = None
fig = px.bar(
x=shown.values,
y=shown.index,
orientation="h",
color=shown.values,
color_continuous_scale="RdYlGn",
range_color=(0, 1),
labels={"x": "Share of flows reported by both sides", "y": ""},
)
fig.update_traces(text=[f"{v:.0%}" for v in shown.values], textposition="outside", cliponaxis=False)
fig.update_layout(
title=dict(text=title if subtitle is None else f"{title}<br><sup>{subtitle}</sup>"),
xaxis_tickformat=".0%",
xaxis_range=[0, min(1.0, float(shown.max()) + 0.08)],
coloraxis_showscale=False,
height=max(400, 22 * len(shown)),
)
# When showing both extremes, mark the omitted middle with a dotted line.
if n_worst and n_best and (n_worst + n_best) < total_items:
fig.add_hline(y=n_worst - 0.5, line_dash="dot", line_color="gray")
fig.add_annotation(
x=float(shown.max()) * 0.6, y=n_worst - 0.5, yanchor="bottom",
text=f"{total_items - n_worst - n_best} items in between, not shown",
showarrow=False, font=dict(color="gray", size=11),
)
fig.show()
plot_reporting_coverage_by_item(tb, year=YEAR, n_worst=15, n_best=15)
Trade is also highly concentrated by item. Of the more than 500 items in the dataset, just 10 make up almost half of the total traded weight.
def plot_item_tonnage_concentration(tb, year, top_n=15):
"""Horizontal bar of the items with the largest share of all traded tonnage in a year.
A handful of bulk staples dominate, even though 500+ items are traded. Per-item tonnage uses
the importer-first flow value, consistent with the rest of the analysis."""
yr = tb[(tb["year"] == year) & tb["element"].isin(["Export quantity", "Import quantity"])].copy()
for col in ("reporter_country", "partner_country"):
yr[col] = yr[col].astype(str)
exp = (
yr[yr["element"] == "Export quantity"]
.groupby(["reporter_country", "partner_country", "item_code"], observed=True)["value"].sum()
.reset_index().rename(columns={"reporter_country": "exporter", "partner_country": "importer", "value": "v_exp"})
)
imp = (
yr[yr["element"] == "Import quantity"]
.groupby(["reporter_country", "partner_country", "item_code"], observed=True)["value"].sum()
.reset_index().rename(columns={"reporter_country": "importer", "partner_country": "exporter", "value": "v_imp"})
)
flows = exp.merge(imp, on=["exporter", "importer", "item_code"], how="outer")
flows["ton"] = flows["v_imp"].fillna(flows["v_exp"])
per_item = flows.groupby("item_code")["ton"].sum().sort_values(ascending=False)
total = per_item.sum()
names = dict(zip(yr["item_code"].astype(int), yr["item"].astype(str)))
top = per_item.head(top_n)
da = pd.DataFrame({"item": [names.get(int(c), str(c)) for c in top.index], "share": top.to_numpy() / total * 100}).iloc[::-1]
# Trade is highly concentrated: assert the top 10 items carry roughly half the tonnage.
assert 0.40 < per_item.head(10).sum() / total < 0.60
fig = px.bar(
da, x="share", y="item", orientation="h",
text=[f"{v:.1f}%" for v in da["share"]],
color="share", color_continuous_scale="Blues",
labels={"share": "Share of all traded tonnes (%)", "item": ""},
)
fig.update_traces(textposition="outside", cliponaxis=False)
fig.update_layout(
coloraxis_showscale=False, height=560, margin=dict(l=10, r=40, t=70, b=10),
title=dict(text=(
f"A few staples dominate food trade by weight, {year}<br>"
f"<sup>Share of all traded tonnes, by item.</sup>"
)),
)
fig.show()
plot_item_tonnage_concentration(tb, year=YEAR)
Reporting by country¶
For each country, we can compute its reporting rate: the number of item flows reported by a country as a share of all item flows the country is involved in (as either exporter or importer).
As the chart below shows, in 2023, 46 countries traded food but reported nothing at all. This means that everything we know about their food trade comes from what their partners report. Examples of those countries are Russia, Bangladesh, Algeria, Iraq, and Iran. For some of these countries the dataset has never held their own reports; the rest reported in earlier years but have since stopped.
Only one country reported more than 90% of their item flows (Netherlands).
And it is reasonable to expect that some important trade flows will be entirely missing from the dataset. For example, any trade between countries that report nothing will be ignored, e.g. any flow between Russia and Iran.
def country_reporting_rate(tb, year):
"""Per-country reporting rate for the given year, both by share of flows and by share of
tonnage. For each bilateral flow a country is involved in (as exporter or importer) we record
whether the country reported its own side, and the flow's displayed tonnage (the importer's
quantity, falling back to the exporter's). Computed over all items, not the selected list.
Returns one row per country with rate (by flows) and rate_tonnage."""
qty = tb[(tb["year"] == year) & tb["element"].isin(["Export quantity", "Import quantity"])][
["reporter_country", "partner_country", "item_code", "element", "value"]
].copy()
for col in ("reporter_country", "partner_country"):
qty[col] = qty[col].astype(str)
exp = (
qty[qty["element"] == "Export quantity"]
.groupby(["reporter_country", "partner_country", "item_code"], observed=True)["value"].sum()
.reset_index().rename(columns={"reporter_country": "exporter", "partner_country": "importer", "value": "v_exp"})
)
imp = (
qty[qty["element"] == "Import quantity"]
.groupby(["reporter_country", "partner_country", "item_code"], observed=True)["value"].sum()
.reset_index().rename(columns={"reporter_country": "importer", "partner_country": "exporter", "value": "v_imp"})
)
flows = exp.merge(imp, on=["exporter", "importer", "item_code"], how="outer")
flows["by_exp"] = flows["v_exp"].notna()
flows["by_imp"] = flows["v_imp"].notna()
flows["ton"] = flows["v_imp"].fillna(flows["v_exp"]) # displayed importer-first tonnage
# Aggregate from each country's perspective, as exporter and as importer.
ev = (
flows.assign(self_ton=flows["ton"].where(flows["by_exp"], 0.0))
.groupby("exporter", observed=True)
.agg(n_flows=("ton", "size"), n_self=("by_exp", "sum"), ton=("ton", "sum"), ton_self=("self_ton", "sum"))
.reset_index().rename(columns={"exporter": "country"})
)
iv = (
flows.assign(self_ton=flows["ton"].where(flows["by_imp"], 0.0))
.groupby("importer", observed=True)
.agg(n_flows=("ton", "size"), n_self=("by_imp", "sum"), ton=("ton", "sum"), ton_self=("self_ton", "sum"))
.reset_index().rename(columns={"importer": "country"})
)
pc = pd.concat([ev, iv]).groupby("country", as_index=False).sum()
pc["rate"] = pc["n_self"] / pc["n_flows"]
pc["rate_tonnage"] = pc["ton_self"] / pc["ton"]
return pc
def plot_country_reporting_rate_distribution(tb, year):
"""Histogram of country reporting rates for the given year.
For each country C, the reporting rate is the fraction of bilateral
flows involving C (as either exporter or importer) for which C itself
reported its own side of the flow. A rate near 0% means we know about C's
trade only because C's partners report it; a rate near 100% means C
publishes its own side of almost every flow it's involved in.
The "0%" bar is given its own bucket so the silent group stands out
from the rest of the (0, 100%] distribution. Bars are shaded red (low,
barely self-reporting) to green (high), and the two extremes are
annotated."""
pc = country_reporting_rate(tb, year)
# Build 11 bins: an exact-0% bucket plus ten 10%-wide buckets covering
# (0, 100%]. A tiny epsilon separates the 0% bucket from "(0, 10%]".
eps = 1e-9
bin_edges = [-eps, eps, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0 + eps]
bin_labels = ["0%", "0-10%", "10-20%", "20-30%", "30-40%", "40-50%",
"50-60%", "60-70%", "70-80%", "80-90%", "90-100%"]
pc["bin"] = pd.cut(pc["rate"], bins=bin_edges, labels=bin_labels, include_lowest=True)
counts = pc.groupby("bin", observed=False).size().reset_index(name="n_countries")
# Representative rate per bin, used to shade each bar from red (low) to green (high).
bin_rate = dict(zip(bin_labels, [0.0, 0.05, 0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95]))
counts["rate"] = counts["bin"].map(bin_rate).astype(float)
# Numbers behind the callouts, computed so they stay correct across years.
n_silent = int((pc["rate"] == 0).sum())
# Only one country reports more than 90% of its own flows; assert it so we catch any change.
top_reporters = sorted(pc.loc[pc["rate"] > 0.9, "country"])
assert top_reporters == ["Netherlands"], f"Expected only the Netherlands above 90%, got {top_reporters}"
fig = px.bar(
counts,
x="bin",
y="n_countries",
text="n_countries",
color="rate",
color_continuous_scale="RdYlGn",
range_color=(0, 1),
category_orders={"bin": bin_labels},
labels={"bin": "Share of its own flows the country reports", "n_countries": "Number of countries"},
)
fig.update_traces(textposition="outside", cliponaxis=False)
fig.update_layout(
title=dict(text=(
f"Most countries report only part of their own food trade ({year})<br>"
f"<sup>Countries grouped by the share of their trade flows they report.</sup>"
)),
coloraxis_showscale=False,
showlegend=False,
bargap=0.15,
)
# Call out the two extremes, parking the text in the empty space beside each bar.
fig.add_annotation(
x="0%", y=n_silent,
text=f"<b>{n_silent} countries report nothing</b><br>we rely entirely on their partners",
showarrow=True, arrowhead=2, ax=40, ay=-15, xanchor="left", align="left",
bgcolor="rgba(255,255,255,0.85)",
)
fig.add_annotation(
x="90-100%", y=1,
text="<b>Only the Netherlands</b><br>reports over 90%",
showarrow=True, arrowhead=2, ax=-140, ay=-100, xanchor="center", align="center",
bgcolor="rgba(255,255,255,0.85)",
)
fig.show()
plot_country_reporting_rate_distribution(tb, year=YEAR)
# List of countries involved in at least one item flow, that have zero reported flows in 2023:
assert len((set(tb[(tb["year"] == YEAR)]["partner_country"]) | set(tb[(tb["year"] == YEAR)]["reporter_country"])) - set(tb[(tb["year"] == YEAR)]["reporter_country"]))
Again, weighting flows by tonnage changes the picture: if a country does report, it usually reports the bulk of its trade by volume. The majority of countries report over 90% of their trade by tonnage, so the reporting gap is concentrated in the fully-silent countries rather than in partial reporting.
# Prove the claim above directly (no helper): how much of each country's trade, by tonnes, does it report itself?
yr = tb[(tb["year"] == YEAR) & tb["element"].isin(["Export quantity", "Import quantity"])].copy()
for col in ("reporter_country", "partner_country"):
yr[col] = yr[col].astype(str)
# One row per (exporter, importer, item), with the tonnes each side reported for it.
yr["exporter"] = np.where(yr["element"] == "Export quantity", yr["reporter_country"], yr["partner_country"])
yr["importer"] = np.where(yr["element"] == "Export quantity", yr["partner_country"], yr["reporter_country"])
exp = yr[yr["element"] == "Export quantity"].groupby(["exporter", "importer", "item_code"])["value"].sum()
imp = yr[yr["element"] == "Import quantity"].groupby(["exporter", "importer", "item_code"])["value"].sum()
flows = pd.DataFrame({"by_exporter": exp, "by_importer": imp}).reset_index()
flows["tonnes"] = flows["by_importer"].fillna(flows["by_exporter"]) # flow size: the importer's figure, or the exporter's if missing
def share_reported(country):
# Of the tonnage of all flows this country is part of, the share where it filed its own side.
involved = flows[(flows["exporter"] == country) | (flows["importer"] == country)]
reported = involved[
((involved["exporter"] == country) & involved["by_exporter"].notna())
| ((involved["importer"] == country) & involved["by_importer"].notna())
]
return reported["tonnes"].sum() / involved["tonnes"].sum()
# The majority of countries report over 90% of their own trade by tonnage.
countries = sorted(set(flows["exporter"]) | set(flows["importer"]))
over_90 = sum(share_reported(c) > 0.9 for c in countries)
assert over_90 / len(countries) > 0.6
def plot_country_reporting_rate_by_tonnage(tb, year):
"""Same as the reporting-rate histogram, but each country's rate is by traded tonnage: the
share of its own trade tonnage (importer-first) that it reports itself, not the share of its
flows. The fully-silent countries stay at 0%, but most others move to the right."""
pc = country_reporting_rate(tb, year)
eps = 1e-9
bin_edges = [-eps, eps, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0 + eps]
bin_labels = ["0%", "0-10%", "10-20%", "20-30%", "30-40%", "40-50%",
"50-60%", "60-70%", "70-80%", "80-90%", "90-100%"]
pc["bin"] = pd.cut(pc["rate_tonnage"], bins=bin_edges, labels=bin_labels, include_lowest=True)
counts = pc.groupby("bin", observed=False).size().reset_index(name="n_countries")
bin_rate = dict(zip(bin_labels, [0.0, 0.05, 0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95]))
counts["rate"] = counts["bin"].map(bin_rate).astype(float)
n_silent = int((pc["rate_tonnage"] == 0).sum())
n_over_90 = int((pc["rate_tonnage"] > 0.9).sum())
# By tonnage the large majority of reporters clear 90%; assert that so we catch any change.
assert n_over_90 > 100, f"Expected most countries above 90% by tonnage, got {n_over_90}"
fig = px.bar(
counts, x="bin", y="n_countries", text="n_countries",
color="rate", color_continuous_scale="RdYlGn", range_color=(0, 1),
category_orders={"bin": bin_labels},
labels={"bin": "Share of its own traded tonnage the country reports", "n_countries": "Number of countries"},
)
fig.update_traces(textposition="outside", cliponaxis=False)
fig.update_layout(
title=dict(text=(
f"By tonnage, most countries report nearly all of their own food trade ({year})<br>"
f"<sup>Countries grouped by the share of their traded tonnage they report.</sup>"
)),
coloraxis_showscale=False,
showlegend=False,
bargap=0.15,
)
fig.add_annotation(
x="0%", y=n_silent,
text=f"<b>{n_silent} countries still report nothing</b><br>(the same silent group as before)",
showarrow=True, arrowhead=2, ax=40, ay=-20, xanchor="left", align="left",
bgcolor="rgba(255,255,255,0.85)",
)
fig.add_annotation(
x="90-100%", y=n_over_90,
text=f"<b>{n_over_90} countries</b> report over 90%<br>of their traded tonnage",
showarrow=True, arrowhead=2, ax=-120, ay=-30, xanchor="center", align="center",
bgcolor="rgba(255,255,255,0.85)",
)
fig.show()
plot_country_reporting_rate_by_tonnage(tb, year=YEAR)
Mapping each country's share of trade reported by tonnage shows where the real gaps are: most of the world reports its own trade, with the fully-silent countries standing out in red.
def plot_country_reporting_rate_map(tb, year):
"""Choropleth of each country's reporting rate by tonnage: the share of its own traded
tonnage it reports, over all items. Most of the world is green; the fully-silent countries
stand out in red. Countries not in the dataset are left grey."""
pc = country_reporting_rate(tb, year)
pc["iso3"] = pc["country"].map(country_to_iso3)
# Bin the rate into five intervals, shaded red (low) to green (high).
eps = 1e-9
bands = ["0-20%", "20-40%", "40-60%", "60-80%", "80-100%"]
pc["band"] = pd.cut(pc["rate_tonnage"], bins=[-eps, 0.2, 0.4, 0.6, 0.8, 1.0 + eps], labels=bands, include_lowest=True)
colors = px.colors.sample_colorscale("RdYlGn", [0.1, 0.3, 0.5, 0.7, 0.9])
fig = px.choropleth(
pc,
locations="iso3",
color="band",
hover_name="country",
custom_data=["rate_tonnage"],
category_orders={"band": bands},
color_discrete_map=dict(zip(bands, colors)),
projection="robinson", # match OWID's world-map projection
)
fig.update_traces(hovertemplate="%{hovertext}: %{customdata[0]:.0%}<extra></extra>")
fig.update_layout(
title=dict(text=(
f"How much of their own food trade each country reports, by tonnage ({year})<br>"
f"<sup>Share of a country's traded tonnage for which it reports its own side, over all items.</sup>"
)),
margin=dict(l=0, r=0, t=70, b=0),
legend=dict(orientation="h", yanchor="top", y=0.05, xanchor="center", x=0.5, title_text="Share reported (by tonnage)"),
)
# Crop Antarctica, as OWID's maps do.
fig.update_geos(showframe=False, showcoastlines=False, lataxis_range=[-58, 85])
fig.show()
plot_country_reporting_rate_map(tb, year=YEAR)
Quantity agreement¶
For item flows that were reported both ways, how closely do the two reported quantities agree? Let's define an agreement ratio as:
100 x min(exported quantity, imported quantity) / max(exported quantity, imported quantity)
This means that 100% is a perfect match (i.e. both countries report exactly the same quantity), and 50% means one country reports twice the other.
The following chart shows the agreement ratio for the top food trading countries.
def plot_quantity_mismatch_by_reporter(tb, year=None, top_n=20, include_unmatched=False, by_tonnage=False):
"""Stacked horizontal bar showing how each country's reports distribute
across quantity-agreement bands, for the top-N reporting countries.
A *matched* flow is one where both the exporter and the importer filed a
report. Agreement = min(exp_qty, imp_qty) / max(exp_qty, imp_qty). 100%
= perfect match, 50% = one side reports 2x the other. Bins into five bands.
The chart aggregates *by the country that filed the report* (the FAOSTAT
`reporter_country` column). Each matched flow therefore contributes two
data points — one to the exporter's distribution and one to the importer's
— both carrying the same agreement value.
When include_unmatched=True, flows reported by only one side land in a
sixth Unmatched band, attributed to the single reporting country."""
if year is None:
rows_per_year = tb.groupby("year", observed=True).size()
year = int(rows_per_year[rows_per_year >= 0.9 * rows_per_year.max()].index.max())
sub = tb[(tb["year"] == year) & tb["element"].isin(["Export quantity", "Import quantity"])][
["reporter_country", "partner_country", "item_code", "element", "value"]
].copy()
for col in ("reporter_country", "partner_country"):
sub[col] = sub[col].astype(str)
# Canonical flow key: (exporter, importer, item_code) for every row.
# Export rows: exporter = reporter, importer = partner.
# Import rows: exporter = partner, importer = reporter.
is_imp = (sub["element"] == "Import quantity").to_numpy()
rep = sub["reporter_country"].to_numpy()
par = sub["partner_country"].to_numpy()
exporter = np.where(is_imp, par, rep)
importer = np.where(is_imp, rep, par)
item_code = sub["item_code"].astype(str).to_numpy()
sub["flow_key"] = [f"{e}|{i}|{c}" for e, i, c in zip(exporter, importer, item_code)]
side_counts = sub.groupby("flow_key").size()
matched_keys = side_counts[side_counts == 2].index
matched_rows = sub[sub["flow_key"].isin(matched_keys)].copy()
# For each matched flow, the agreement is the same for both of its rows
# (since min/max is computed per flow_key).
matched_rows["v_min"] = matched_rows.groupby("flow_key")["value"].transform("min")
matched_rows["v_max"] = matched_rows.groupby("flow_key")["value"].transform("max")
with np.errstate(divide="ignore", invalid="ignore"):
agreement = np.where(matched_rows["v_max"] > 0,
matched_rows["v_min"] / matched_rows["v_max"], 0.0)
matched_rows["agreement"] = agreement
matched_band_labels = ["<25%", "25-50%", "50-75%", "75-90%", ">=90%"]
matched_band_colors = ["#d73027", "#fc8d59", "#fee08b", "#91cf60", "#1a9850"]
matched_rows["band"] = pd.cut(
matched_rows["agreement"],
bins=[-0.001, 0.25, 0.50, 0.75, 0.90, 1.001],
labels=matched_band_labels,
)
if include_unmatched:
unmatched_keys = side_counts[side_counts == 1].index
unmatched_rows = sub[sub["flow_key"].isin(unmatched_keys)].copy()
unmatched_rows["band"] = "Unmatched"
all_rows = pd.concat([
matched_rows[["reporter_country", "band", "value"]],
unmatched_rows[["reporter_country", "band", "value"]],
], ignore_index=True)
band_labels = ["Unmatched"] + matched_band_labels
band_colors = ["#67000d"] + matched_band_colors
else:
all_rows = matched_rows[["reporter_country", "band", "value"]].copy()
band_labels = matched_band_labels
band_colors = matched_band_colors
all_rows["band"] = pd.Categorical(all_rows["band"], categories=band_labels)
# Rank reporters by total tonnes they reported this year (matched + unmatched-by-them rows).
# This matches the natural intuition of "biggest traders" better than report count.
reporter_tonnes = all_rows.groupby("reporter_country", observed=True)["value"].sum().sort_values(ascending=False) # ty: ignore[missing-argument]
top = reporter_tonnes.head(top_n).index.tolist() if top_n is not None else reporter_tonnes.index.tolist()
sub_flat = all_rows[all_rows["reporter_country"].isin(top)]
grouped = sub_flat.groupby(["reporter_country", "band"], observed=True)
shares = (grouped["value"].sum() if by_tonnage else grouped.size()).unstack(fill_value=0)
shares = shares.reindex(columns=band_labels, fill_value=0)
shares = shares.div(shares.sum(axis=1), axis=0)
shares = shares.sort_values(">=90%", ascending=True)
long = shares.reset_index().melt(id_vars="reporter_country", var_name="band", value_name="share")
title = f"Quantity agreement among matched flows{', by tonnage,' if by_tonnage else ''} in {year}"
if by_tonnage:
x_label = "Share of the country's matched-flow tonnage"
else:
x_label = ("Share of all reports filed by the country" if include_unmatched
else "Share of matched-flow reports filed by the country")
fig = px.bar(
long, x="share", y="reporter_country", color="band", orientation="h",
title=title,
labels={"share": x_label, "reporter_country": "Reporter"},
category_orders={"band": band_labels, "reporter_country": shares.index.tolist()},
color_discrete_sequence=band_colors,
)
fig.update_layout(
barmode="stack",
xaxis_tickformat=".0%",
height=max(400, 18 * len(shares)),
yaxis=dict(tickmode="linear", dtick=1),
)
fig.show()
plot_quantity_mismatch_by_reporter(tb, year=2023, top_n=30, include_unmatched=False)
For Brazil, for example, we see that only around 43% of those matched flows agreed with each other well (the reported quantity matched between 75 and 100%).
This disagreement is surprisingly large and common.
# Simple code to double-check that result for 2023 alone.
# Define a unique identifier of each item flow, namely [importer country][exporter country][item]
t = tb[(tb["year"]==YEAR)].reset_index(drop=True)
t["id"] = ""
mask_imports = (t["element"] == "Import quantity")
t.loc[mask_imports, "id"] = t[mask_imports]["reporter_country"].astype("string") + t[mask_imports]["partner_country"].astype("string") + t[mask_imports]["item"].astype("string")
t.loc[~mask_imports, "id"] = t[~mask_imports]["partner_country"].astype("string") + t[~mask_imports]["reporter_country"].astype("string") + t[~mask_imports]["item"].astype("string")
assert t["id"].notnull().all()
# Select item flows reported by both countries in 2023.
t = t[t.groupby("id")["id"].transform("count")==2].reset_index(drop=True)
# Add columns for the minimum and maximum values of each item flow.
t["value_min"] = t.groupby("id")["value"].transform("min")
t["value_max"] = t.groupby("id")["value"].transform("max")
# Define the quantity agreement ratio.
t["ratio"] = t["value_min"] / t["value_max"]
# Verify the claim above: about 43% of Brazil's matched flows agree well (ratio between 75% and 100%).
mask_country = t["reporter_country"] == "Brazil"
brazil_well = 100 * len(t[mask_country & (t["ratio"] >= 0.75)]) / len(t[mask_country])
assert round(brazil_well) == 43, f"Brazil's well-agreeing share is {brazil_well:.0f}%, expected ~43%"
Weighting each flow by its tonnage, the large flows agree far better. So the severe disagreement is concentrated in small shipments; most of the tonnage matches closely.
plot_quantity_mismatch_by_reporter(tb, year=2023, top_n=30, include_unmatched=False, by_tonnage=True)
Estimating production and supply¶
To put trade figures in context, it is convenient to express them in relative terms:
- exports as a share of what a country produces, and
- imports as a share of domestic supply (the amount of tonnes that are available for consumption in the country).
However, production and domestic supply are not available in the TM dataset. So we need to combine it with other FAOSTAT datasets, of which there are three candidates:
- Crops and livestock products (QCL) is FAOSTAT's headline production dataset, but it only contains production-side elements (production, area harvested, yield, animal numbers). It has no trade information, so domestic supply cannot be computed from it at all. It also mostly tracks primary production (raw commodities like wheat or milk): for 2023 it covers only 43% of the items traded in TM, accounting for 63% of traded tonnage.
- Food Balances (FBS) conveniently provides production, domestic supply, imports, and exports directly.
However, its items are aggregated groups that do not map cleanly to TM or QCL item codes.
For example, FBS' "Cocoa Beans and products" bundles several items (
661 Cocoa beans, 662 Cocoa paste, 665 Cocoa powder & cake, 666 Chocolate products), and properly disaggregating FBS commodities is not trivial (see e.g. Zhao et al. (2025)). - Supply Utilization Accounts (SCL) contains production, imports, exports, and stock variation for each item, country and year. It covers 78% of the items traded in TM, accounting for 91% of traded tonnage, and conveniently uses the same item codes as TM.
Of these three, SCL is the best fit, so we explored using it to estimate production and domestic supply.
# SCL and QCL are loaded only here (for this section), read from the published feather directly to
# avoid the fetch() memory spike, and freed at the end of the section so the notebook fits in Colab.
import gc
with contextlib.redirect_stdout(io.StringIO()):
tb_scl = pd.read_feather(f"{CATALOG_URL}/garden/faostat/2026-02-25/faostat_scl/faostat_scl.feather")
tb_qcl = pd.read_feather(f"{CATALOG_URL}/garden/faostat/2026-02-25/faostat_qcl/faostat_qcl.feather")
# Double-check the QCL vs SCL coverage numbers quoted above (kept as silent assertions).
tm_year = tb[tb["year"] == YEAR].copy()
tm_year["item_code"] = tm_year["item_code"].astype(int)
tm_tonnage_by_item = pd.Series(tm_year.groupby("item_code", observed=True)["value"].sum())
item_codes_tm = set(tm_tonnage_by_item.index)
total_tonnage = tm_tonnage_by_item.sum()
item_codes_scl = set(tb_scl[tb_scl["year"] == YEAR]["item_code"].astype(str).astype(int))
# QCL has no trade elements: production-side only (its "Stocks" element is live-animal headcounts).
assert not {"Import quantity", "Export quantity", "Stock Variation"} & set(tb_qcl["element"].astype(str).unique())
item_codes_qcl = set(tb_qcl[(tb_qcl["year"] == YEAR) & (tb_qcl["element"].astype(str) == "Production")]["item_code"].astype(str).astype(int))
def _coverage(item_codes):
share_items = len(item_codes_tm & item_codes) / len(item_codes_tm)
share_tonnage = tm_tonnage_by_item[tm_tonnage_by_item.index.isin(item_codes)].sum() / total_tonnage
return share_items, share_tonnage
qcl_items, qcl_tonnage = _coverage(item_codes_qcl)
scl_items, scl_tonnage = _coverage(item_codes_scl)
# Verify the coverage figures quoted in the text above: QCL ~43% of items / ~63% of tonnage, SCL ~78% / ~91%.
assert 0.40 < qcl_items < 0.46 and 0.55 < qcl_tonnage < 0.70
assert 0.75 < scl_items < 0.81 and 0.85 < scl_tonnage < 0.95
# Verify the country-coverage gap quoted in the text: countries that trade in TM but are absent from SCL.
tm_countries = set(tb[tb["year"] == YEAR]["reporter_country"].astype(str)) | set(tb[tb["year"] == YEAR]["partner_country"].astype(str))
scl_countries = set(tb_scl[tb_scl["year"] == YEAR]["country"].astype(str))
assert len(tm_countries - scl_countries) == 22
# Free SCL and QCL: the remaining cells need only the trade matrix.
del tb_scl, tb_qcl
_ = gc.collect()
With SCL in hand, we estimated each country's production and domestic supply, and showed, alongside the raw trade flows, two derived shares: what fraction of its production a country exported, and what fraction of its domestic supply it imported. The aim was to give a sense of scale, for example whether a country exports most of what it grows, or depends on imports for most of what it consumes.
We ultimately decided not to show these shares. Each share divides a trade figure taken from the TM dataset (prioritizing importer reports) by a production or domestic supply estimated using data from SCL. Although SCL draws on the same underlying trade statistics, its trade figures are country-level totals that prioritize a country's own reporting; so the combination of the two datasets often led to significant discrepancies in trade figures, and the resulting share could not be trusted.
Another limitation of the SCL dataset was country coverage: 22 countries that trade food are entirely absent from SCL, including major importers such as Japan and Singapore.
The shares also frequently came out well above 100%: a country appearing to export more than 100% of its production, or import more than 100% of its domestic supply. This is not necessarily an error: a country that re-exports imported goods can export more than it produces, and, equivalently, it can import more than its domestic supply. But the same thing can arise artificially, simply because the two datasets disagree.
Once we corrected for all these issues, only a relatively small fraction of the flows retained a meaningful share estimate. Therefore, to avoid unnecessary complexity and potential inaccuracies, we decided to avoid combining the TM dataset with SCL, and hence removed the estimated shares of production and domestic supply.
Combining items¶
A certain food item is sometimes traded in different derived forms, e.g. bone-in vs boneless meat, in-shell vs shelled nuts, paddy vs milled rice, raw vs refined sugar. For this reason, the TM dataset often splits one common product, e.g. almonds, across several items, e.g. 'Almonds, in shell', 'Almonds, shelled'.
But this split is often a technicality, rather than a meaningful distinction. We therefore combine these derived forms to be able to have a complete picture of the trade of those food items.
To do that, we sum traded quantities across all their derived forms.
The charts below show the items we have combined, and the relative share of each derived forms across all item flows.
# Group the combined items by commodity family so each chart shares one set of FAO forms.
ITEM_GROUPS = {
"Meat (bone-in vs. boneless)": ["Beef", "Pork"],
"Nuts (in-shell vs. shelled)": ["Almonds", "Cashews", "Walnuts", "Hazelnuts", "Brazil nuts", "Groundnuts"],
"Rice (milled, broken, husked)": ["Rice"],
"Sugar (raw vs. refined)": ["Sugar"],
}
# Global trade per FAO code (2023): the larger of the two reported directions, summed to the world total.
year_t = tb[(tb["year"] == YEAR) & (tb["unit"] == "t")]
world = year_t[year_t["element"].isin(["Export quantity", "Import quantity"])]
by_code = world.groupby(["item_code", "element"], observed=True)["value"].sum().unstack("element").max(axis=1)
def split_frame(items):
rows = []
for item in items:
members = COMBINED_ITEMS[item]
total = sum(float(by_code.get(code, 0.0)) for code in members)
for code, form in members.items():
rows.append({"item": item, "form": form, "share": 100 * float(by_code.get(code, 0.0)) / total,
"mt": float(by_code.get(code, 0.0)) / 1e6, "primary": code == next(iter(members))})
return pd.DataFrame(rows)
for group, items in ITEM_GROUPS.items():
frame = split_frame(items)
# Order items by the primary code's share, so the split reads as a gradient.
order = frame[frame["primary"]].sort_values("share", ascending=False)["item"].tolist()
fig = px.bar(
frame, x="share", y="item", color="form", orientation="h",
category_orders={"item": order},
hover_data={"mt": ":.2f", "share": ":.1f", "primary": False},
labels={"share": "Share of the commodity's traded tonnes (%)", "item": "", "form": "FAO form"},
title=f"{group}: how trade splits across items, {YEAR}",
)
fig.update_layout(height=200 + 55 * len(items), width=850, barmode="stack", legend_title_text="FAO form")
fig.show()
Summary¶
This notebook examined FAOSTAT's Detailed trade matrix, the source behind our food trade visualization, and documented the choices we made along the way.
The trade matrix records bilateral flows for over 500 food items, each reported independently by the importing and the exporting country. Reporting is far from complete or consistent: many flows are reported by only one side, dozens of countries report none of their own trade at all, and even when both sides report the same flow their quantities often disagree. These mismatches are concentrated in small flows, though; a handful of staples make up most of the traded weight, and those large flows are widely reported and tend to agree between partners.
This shaped a few choices:
- we focus on the latest year with reasonably complete reporting (2023) rather than the partially reported latest year;
- for each flow we show a single figure, prioritizing the importer's report and falling back to the exporter's when only one side reports;
- we combine the derived forms of certain items (for example bone-in and boneless meat, or in-shell and shelled nuts) into a single product.
We also explored expressing trade in relative terms, as a share of each country's production or domestic supply estimated from FAOSTAT's Supply Utilization Accounts. We ultimately decided against it, as the combination of datasets often led to unreliable estimates.