How to find research about young people, and how to judge it

Anyone who works with young people ends up needing numbers: a funding application, a submission, a policy note, an argument with a colleague. The internet supplies them instantly, and most of what it supplies is a figure that has travelled so far from its source that nobody can say what it measured. Knowing the small number of places evidence actually comes from is the cure.

The families of sources

Population data comes from a national statistical agency and answers questions about how many: how many young people live in a region, in what households, in what kind of housing, in or out of work. It is strong on structure and weak on experience, and it is usually a few years behind the present.

Longitudinal surveys follow the same people over time. They are the only sources that can honestly say what led to what, because they saw the earlier state. They are slow, expensive and precious, and they are routinely misquoted as if they were snapshots.

Administrative data is collected by institutions doing their job: enrolments, benefit claims, hospital presentations, court appearances. It is detailed and current, and its weakness is built in, because it only sees the young people who came into contact with the system that recorded them.

Peer-reviewed research tests specific questions with declared methods. Grey literature, which means reports from agencies, services and advocacy bodies, often reaches the ground faster and covers subjects academia neglects, at the cost of uneven method. A clearinghouse or research digest is the fifth kind: not a source, but a map of the other four.

Five questions that settle most claims

Who was asked, and who could not be? A survey of school students has no view of young people who have left school, which is frequently the group under discussion.

How many were asked, and how are the results broken down? A national sample that looks reassuring becomes a handful of respondents once it is split by age, region and background, and a handful cannot carry a percentage.

What exactly was measured? Words like disengaged, at risk, homeless and unemployed each have several working definitions, and the definition usually explains the disagreement between two studies more than the findings do.

When was the fieldwork done, as opposed to when was the report published? The gap is often years.

Who paid for it, and what did they want it to show? Funding does not invalidate a study, but it tells you which limitations to read carefully.

Using a clearinghouse well

A digest exists so that you do not have to hold the whole field in your head. Use it to find the original, not to replace it: follow the link, read the method section, then decide. If a summary makes a claim the original does not support, the summary is the problem, and passing it on makes you part of the chain.

Keep a note of what you took and from where, in enough detail that you could find it again in a year. This is the least glamorous habit in the whole business and the one that protects you when someone finally asks where the figure came from.

Why the habit outlives the report

Individual studies age quickly. The questions above do not: they worked on the reports that arrived before you started and will work on the ones that arrive after you have moved on. Learning them once is worth more than bookmarking any particular dataset, because they let you judge something you have never seen before, which is the situation you will actually be in.

FAQ

Impacts of the loss of ACYS on youth services and young people