Case study · Research · Statistics
Are we doing well? Statistics to evaluate agri-policy
My MSc thesis: what 55 organic farmers say about EU policy — and how I picked the right statistical test for nominal data and small samples.
55
field interviews
R
Statistical analysis
RMarkdown
Reproducibility
My MSc thesis, part of the EU FARMERS project (FORMIT, funded by the European Joint Research Centre), measured how farmers perceive the European Union’s agricultural public policies. The Spanish case study: organic tomato growers in Andalusia and Extremadura.
I ran 55 field interviews (snowball sampling) and asked about six EU policies — Farm to Fork, the pesticides directive, Biodiversity 2030, Nitrates, the Water Framework and Birds/Habitats. Each farmer rated each policy as Useful, Useless, or “I don’t know it.” The underlying question: are we doing well?
The challenge: firm conclusions from hard data#
The data was purely nominal (Useful / Useless / Don’t know it), measured on the same 55 farmers for every policy — paired data. And with a small sample, many categories held very few cases. That detail decides which statistical test is valid, and rules out the usual shortcuts.
A global average says nothing useful here; what matters is who perceives what. So I designed proxy variables to segment the sample: farm size (utilised agricultural area, from micro to large) as a proxy for small vs. large farmer, plus age, origin (rural/neo-rural) and education.
How I solved it with statistics#
To compare ratings across policies (paired nominal data) I used McNemar-Bowker tests, with a 2×2 symmetry post hoc and a Benjamini-Hochberg correction for the multiple comparisons.
To relate each factor (size, education, origin, etc.) to the perception of each policy, chi-square was unreliable because of cells with fewer than 5 cases, so I used Fisher’s exact test, with a post hoc on the chi-square standardized residuals to see which groups drove the difference.
The point wasn’t running a test, it was choosing the right one for the data: paired vs. unpaired, and Fisher instead of chi-square with small samples.
What I found#
The most common answer wasn’t “useful” or “useless” — it was “I don’t know it”: many of these policies simply don’t reach the people working the land. The least-known was the Nitrates Directive; the best-rated, the Biodiversity 2030 Strategy.
And segmenting by size revealed the key pattern: small growers didn’t know the policies (e.g., Farm to Fork or the pesticides directive), while large ones did know them — and often judged them useless. Education and origin shifted perception too. In short: EU agricultural policy isn’t reaching the small farmers it affects most.
Stack#
Frequently asked questions#
Why did you use RMarkdown?
For reproducibility: the analysis and the document live in the same place, so anyone can go into the repository and reproduce the results exactly. Better still, you can download the database, expand it with new interviews, and run the same statistical analysis on your own data.
Why Fisher instead of chi-square?
Because several categories had fewer than 5 responses; under those conditions chi-square is no longer reliable and Fisher’s exact test is the correct choice.
What was the thesis about?
Evaluating EU agricultural policy from the perception of 55 organic tomato farmers in Andalusia and Extremadura, as part of the FARMERS project (JRC).