Laura Schneider: Press freedom rankings ‘problematic’

Every year, press freedom bursts into the media limelight when the freedom of press indexes are published. But these international rankings should be treated with caution, warns Laura Schneider, a media freedom expert at the Research Center for Media and Communication in Hamburg. Schneider is analyzing annual press freedom rankings for her PhD thesis. She says that although the rankings are still important, they are not free from bias or based on sound methodologies. She is working on developing an alternative method of measuring press freedom.

DW Akademie’s Christine Harjes talked to Laura Schneider about the problems of, and possible solutions to, the press freedom rankings.

What rankings have your researched?

Freedom House’s Freedom of the Press and the Press Freedom Index from Reporters Without Borders. These are the only two global rankings which are published annually. UNESCO also releases Media Development Indicators which are very comprehensive instruments for the measurement of media freedom but they aren’t evaluated regularly or globally.

What made you start analyzing these rankings?

I went to Mexico to research press freedom and so I took a look at the rankings to get some background information about the media situation. And when I starting digging down a bit, it quickly became obvious that there’s a problem in the methodology used to compute them. On one hand, the rankings enjoy huge respect and are widely reported in the media when they are released. On the other hand, if you are aware of the problems with rankings, the way the media report on the topic is questionable.

In what way is the methodology problematic?

It’s primarily a question of the definition of freedom of the press because that influences the way the measurements are made. Both rankings say they are based on Article 19 of the United Nations Universal Declaration of Human Rights (which states that everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers). But the organizations don’t elaborate on what they actually mean by press freedom.

This leads to the question of what indicators should be used to measure press freedom: what do you look at if you want to know about the press situation situation in a country? Then the question of weighting or scoring is also important. For example, is the constitutional right to press freedom worth six points or two? Then, who assesses the situation? Who do you survey, and how many people do you need to survey in order to arrive at a result?

So, if different methodologies mean the rankings arrive at different results, is there any one country where the difference is particularly obvious?

Togo’s ranking has shown a massive difference in the past eight years. In 2013, Togo ranked 159th (from 197 countries) in the Freedom House rankings whereas Reporters Without Borders ranked Togo at 83 (from 179). This is a difference of nearly 80 places!

How do you account for that difference?

The problem with the Freedom House rankings is that two people set out the indicators about eight years ago without any group discussion with experts. Rather they used their gut feeling to decide which indicators to include and how many points each indicator was worth. It’s a similar situation at Reporters Without Borders – although the decisions are made in a team, they are still very subjective.

Reporters Without Borders focuses on defending press freedom as it affects journalists. As such, it primarily examines violence against journalists as well as general working conditions. In contrast, Freedom House looks at the structural conditions of the entire system and only one question deals with violence against journalists. Because the number of physical assaults against journalists is relatively low in Togo in comparison to other countries, Togo ranks highly in the Reporters Without Borders index. But the structural conditions that Freedom House focus on are very poor in Togo, so this explains the difference.

So the two organizations have a different understanding of press freedom?

You can see that Freedom House has more of an American understanding of media freedom in that public broadcasting is often regarded as state-controlled media. Whereas Reporters Without Borders has more of a European understanding which sees public broadcasting as important and positive.

This Western bias is one of my biggest criticisms of the rankings, which is why I want to develop an alternative instrument. It’s based on surveying 1,000 experts from 124 countries about which indicators are important for press freedom and how important each indicator is. That way I hope the cultural bias can be slightly less.

That sounds like an ambitious undertaking. How did you find 1,000 experts?

I spent several weeks researching who deals explicitly with the issue of press freedom – from experts working at NGOs and major international organizations to journalists, academics, media development experts and other media activists. I emailed these people and asked to participate in my survey.

Did the experts include indicators which don’t appear in the other two rankings?

I didn’t ask them to nominate their own indicators but rather to rank the importance 47 indicators from a list I gave them. In order to compile the list, I did a literature review and analyzed deficits in the existing rankings as well as looked at numerous other initiatives dealing with the measurement of media freedom. In the end, I had a list of 47 indicators plus sub-indicators. Among them were several indicators that hardly receive consideration in the established rankings, such as the subjective feeling of safety and freedom of journalists, which of course affects journalists’ work. Or the use of new media and social media like Facebook and Twitter – both by the media and by the audience.

Do you have any preliminary findings?

Almost all of the experts perceived judicial independence as the most important indicator – but it isn’t used as a criteria by Freedom House or Reporters without Borders.

Alongside her PhD studies, Laura Schneider is also project coordinator of the Latin American Media Program at the International Media Center Hamburg. She also works as a freelance journalist for several German media outlets and has worked as a radio and newspaper reporter in Mexico. She completed her bachelor’s and master’s degrees in communication science and journalism at the universities of Hamburg, Guadalajara (Mexico), and Sydney.