Centre for Law and Democracy

Sheila Coronel’s paper, Measuring Openness: A Survey of Transparency Ratings and the Prospects for a Global Index, is the first serious piece of research about the systems for assessing government openness which have mushroomed in recent years, alongside a corresponding growth in overall interest in openness. It is useful inasmuch as it provides an overview of what is being done around the world, and also in its analysis of the challenging question of the practicality and utility of a global or super index. It largely misses the mark, however, in its assessment of existing systems, mostly because it compares them to the idea of a super index, rather than against their own objectives and the functional utility they actually provide.

A number of IGOs and NGOs have conducted or are developing systems for measuring the quality of government openness. Coronel’s paper divides these into five categories: rating right to information (RTI) laws; measuring transparency through a governance lens; evaluating RTI practice; assessing supply side interventions; and sector-specific initiatives. This mapping exercise is one of the strong points of the paper, providing those working in the field with an overview of what is going on.

This aspect of the paper could, however, be improved by providing a clearer overview of the various initiatives. As it is, the detail is somewhat buried in the text, leaving the reader to try to extract a broader picture on his or her own. A brief introductory statement at the beginning of each section (as in: “This section looks at the five main …”) or a table would help orient readers.

Another drawback with the paper is that it does not define what is meant by “openness” or “transparency”. As a result, it refers to initiatives which do not actually aspire to measure levels of access to information, such as governance indicators, without providing a clear critical analysis of what they assess or how extensive their assessment of the right to information is, in either law or practice.

The paper could also have made a more rigorous assessment of the content of the various indices which it cites, evaluating and comparing more precisely what they actually assess and how that data is obtained. For example, some of the governance indicators are based on very rough assessments of the legal framework for access to information and on a rather unscientific approach to assessing transparency in practice. The paper also qualifies some indicators as not being based on perceptions when a review of the questions and answers shows that they evidently are, at least in part.

By throwing governance indices into the bag without evaluating in any depth their access to information content adds a layer of confusion to the paper because it is not clear which indicators might be included in a possible “super index”. The paper could usefully evaluate what national and international research exists in specific areas of advancing transparency: legal framework, infrastructure, proactive disclosure, and reactive release of information.

A key thrust of the paper is to explore what the value of a global or super transparency index might be, and to assess whether such an exercise would be practical (logistically, in terms of funding, in terms of bringing various actors together). As an initial exploration of this issue, the paper makes a useful contribution here, in particular by exploring the various challenges in preparing such an index. It notes, for example, the enormous likely costs of such a venture, the challenges of bringing together sufficient expertise to prepare it, the many different actors, with sometimes divergent or competing interests, which have been involved in existing initiatives, and the difficulty of identifying what exactly it should measure.

It also addresses the utility of such a tool, recognising that it would be useful to donors and also as an advocacy tool, but also noting that it might oversimplify what is ultimately too complex an issue to be reflected in one index. The paper also tackles the question of whether a set of national or regional indices would be more useful than a global index, again noting pros and cons on both sides.

The paper could, however, be criticised for an undue focus on the idea of a super index, which many might deem to be a straw man. This is reflected in one of the headings: “VIII. The Ratings Paradox: Many Measures But No ‘Super Index’”. While catchy, this heading only makes sense if one starts from the assumption that there should be a super index. This becomes clear by analogy (no one would say: “So many universities but no super university”). As a result, we have not so much a paradox as a begging of the question. We do not believe that the idea of a super index is viable and note that indices and ratings in other sectors do not take this approach. A good example is the UNDP Human Development Index, which relies on only three key indicators: life expectancy, education and wealth. Other measures, including macro-economic indicators such as GDP, are based on agreed indicators which are measured by multiple actors rather than as part of one mega monitoring exercise.

One consequence of the focus on a super index is that the piece fails to explore the issue of what gaps are left by the other initiatives, apart from a very brief section at the end, since it works on the assumption that any gaps would be filled by a super index. This is a hugely important subject, which it would have been natural for a paper along these lines to broach.

We submit that a better approach would have been to assess the research gaps and then consider the best way to fill them. In particular, it would be useful to assess whether more extensive and systematic indices along the lines of or extrapolating from those that have already been conducted might largely satisfy the research needs that exist in this area.

Another consequence is the constant theme running through the paper that there are too many indices, as reflected in the claim that the openness community “suffers from a surfeit, rather than a lack, of indices”. This is not supported by proper analysis, for example showing that there was no cost-benefit correlation to a particular body of work or that there is unnecessary overlap and repetition in the indices. In some cases, highly contestable claims are made. For example, the paper states in several places that there is no dearth of comparative assessments of RTI laws. While it is true that several indices do include a few indicators relating to RTI laws as part of a wider basket of openness measurements, so far only the Access Info Europe and Centre for Law and Democracy RTI Rating claims to be a rigorous assessment of legal quality in this area. Prior to the existence of the harmonised standards proposed by the RTI Rating, national groups had no proper framework for comparing their country’s law with that of other countries.

The focus on a super index leads to the most serious shortcoming in the paper, namely its assessment of the value of the existing indices. Thus, indices are repeatedly criticised for not being global in nature or for only measuring certain types of openness, while in other cases indices are praised simply because they measure more features. In very few cases is reference made to the underlying purpose of the index in question, whether this is a useful purpose, or the extent to which the index satisfies the purpose. This is a bit like criticising a Ferrari for not being able to carry a family of four, on the basis that it does not meet all of the needs one might wish for in an automobile.

This shortcoming is clearly evident in relation to the RTI Rating, developed by our two organisations, which comes in for strong criticism because it only assesses legal protection and not implementation. This fact and its implications have been acknowledged from the beginning. What the paper fails to recognise, however, is that as a tool for legal improvement – a matter of not inconsiderable importance to which enormous energies are being directed in countries around the world – the RTI Rating has consistently proven its value. This is reflected, among other things, in the very high demand for it. Ensuring a strong legal framework for access to information is an indispensable step towards increased government transparency in practice in countries around the world. Many civil society organisations are campaigning for a stronger legal framework and the RTI Rating provides valuable comparative arguments to support such campaigns. No doubt many of those responsible for the other indices covered in the paper would make similar claims to the effect that their own indices also serve useful purposes in the various different sectors they address.

Coronel’s paper will hopefully initiate more of a debate among advocates about existing initiatives to measure government openness, something which would undoubtedly prove useful. Despite the criticisms above, the paper provides us with a good starting point for this debate, which we should take advantage of. A natural next step in this process would be to host a workshop bringing together some key players – people who have been involved in developing existing indices, both on transparency and in other areas, academics and civil society groups who have used the indices – to identify gaps in the existing research and look at ways to address them.

By Helen Darbishire, Executive Director, Access Info Europe, and Toby Mendel, Executive Director, Centre for Law and Democracy

Below is an Expert Review of the ELLA materials on Transparency and Access to Information, specifically the Guide on the Latin American Approach to Transparency and Access to Information, and the Practice Briefs on Building the Legal Framework to Support Transparency and Access to Information in Latin America, Fighting Corruption by Improving Transparency and Access to Information, Civil Society’s Regional Network for Advancing Freedom of Expression and Access to Information, and Using Information for Accountability and Justice: Lessons From Latin American Civil Society.

The ELLA materials can be found here. ELLA seeks to provide a knowledge sharing and learning platform on selected economic, environmental and governance issues from Latin America.

ELLA Expert Reviews: As part of ongoing efforts to ensure quality in our research and communications, the ELLA team asks recognised experts to conduct reviews of the knowledge materials in a given theme and produce a short written response. The purpose is to highlight a range of distinct perspectives amongst experts in the field and link readers with additional research, cases and arguments that the original ELLA materials may not cover. This Expert Review is also available on the ELLA website here.

Overall, these materials provide an excellent overview of the very significant and often innovative developments in the region in terms of transparency. As such, they will be useful for those working on this issue both within the region and globally.

As a general comment, the materials focus on the legal environment, corruption, the regional network and accountability. Another key benefit of access to information is facilitating participation in decision-making. It might be interesting to consider producing a fifth Practice Note on this important issue. Otherwise, in some places the materials seem to focus exclusively on positive developments, although a few sections do also highlight challenges. While the positive developments are impressive, at the same time it would also be useful to include some of the challenges faced in the region.(1) Finally, though the comparative assessments only focus on Africa and South Asia, given they are the target regions of the initiative, other regions, especially Southeast Asia, could also benefit from the work.

Below, observations and suggestions for additional considerations are presented regarding individual knowledge materials.

Guide on the Latin American Approach to Transparency and Access to Information
The Guide opens with the following statement: “Transparency and the right to access information is now a reality in many Latin American countries.” Unfortunately, this is not quite correct. What is true is that a majority of Latin American countries have passed access to information laws but, as the Key Lessons Learned describes, translating this into reality remains a challenge in many countries.

The first Key Lesson Learned notes that regional and national court rulings have helped create a legal framework for access to information even in the absence of freedom of information acts (FOIAs). This is true, but at least as importantly, these court rulings have had another key impact: supporting local campaigns to adopt local laws (as the international rulings have required). This is recognised later in the Guide but not at the front end. Ultimately, access to information can only effectively be protected through a FOIA.

Although the Guide refers to several standard setting initiatives in Latin America, it omits one which is quite important, the Inter-American Juridical Committee’s Principles on the right of access to Information.

The section on national FOIAs claims that many Latin American laws incorporate ‘innovative elements’ in their design, such as establishing oversight bodies and requiring public bodies to disclose information proactively. While these are certainly positive features, and the Mexican oversight body, the Federal Institute for Access to Public Information (IFAI), is one of the strongest in the world, it is not actually correct to describe these as innovations, as they are found in many older access to information laws.

In contrast, the electronic systems in Mexico and Chile are defined as “good practice in the region” but in fact these systems are more than just reference points for the region, they are actually global leaders. The Claude Reyes case was also a global leader, it being the first international court to adopt such a decision. It is fair to go even further and note that the European Court had consistently refused to recognise a right to information, and that recognition by the Inter-American Court played a rule in pushing the European Court towards recognition in the 2009 case, Társaság A Szabadságjogokért v. Hungary.

The Guide discusses ‘targeted transparency policies’ as though they were a new phenomenon but, although the term was only coined in 2007, in fact there has been a reasonably long-standing practice of this in many countries. This is better seen as a process of evolution of the wider practice of proactive disclosure than a specific new development. Indeed, the whole development of e-government is closely linked to this.

Finally, the Guide describes Colombia as a country without a FOIA, but all of the leading lists recognise that Colombia does have a FOIA, albeit a rather weak one.(2)

Practice Brief on Building the Legal Framework to Support Transparency and Access to Information in Latin America
One overall comment is that this Practice Brief regularly refers to ‘citizens’ as the group enabled to make requests but in fact better practice is to allow anyone to make such requests.

The Key Lessons Learned highlights three features of FOIAs: broad scope, sanctions and an oversight body. This seems to be somewhat arbitrary, as there are several other features that are at least arguably equally important, such as good procedures for processing requests, a narrow regime of exceptions and effective promotional measures. The sections on innovative elements are a bit broader, but they also omit the issues of exceptions (at least in the list on page two, although there is a short section on this on page four) and promotional measures. Latin America has contributed some interesting innovations in the latter area, for example in terms of requirements of public education about the right to information, which it would be useful to highlight.

Once again, the claim that Latin American countries have been at the forefront of designing and enacting innovating FOIAs is probably overstated. While there have been strong developments in this region, the same is true of East and Central Europe and Asia, and to a lesser extent Africa.

The section on scope defines this as the range of public bodies covered by the law; this is certainly important but the issue also extends to the range of types of information covered, which is limited in some laws in the region. The section on sanctions and protections refers to protection for whistleblowers, but it is also important –probably even more important in terms of changing the culture of secrecy – to provide protection to civil servants who disclose information in good faith pursuant to the law. In the area of procedures, it would be useful to add in that Latin America is again a world leader in terms of measures to enforce timelines. The section on exceptions refers to the need for a harm test, but it omits to mention the equally important need for a public interest override, unfortunately something that Latin American laws are weak on.

Practice Brief on Fighting Corruption by Improving Transparency and Access to Information
This Practice Brief contains a good description of two key openness systems for reducing corruption, procurement and official asset disclosures. The claim that the region pioneered the former is a bit overstated, although the region has certainly introduced some interesting innovations.

A third transparency approach might also be worth mentioning, namely the proactive publication of information about all beneficiaries of public programmes. This has the potential to significantly reduce corruption. While this information would be considered private in most countries, it is its potential to reduce corruption that justifies making it public (ironically, legally this would be pursuant to the public interest override that is missing in most Latin American laws). It would, therefore, be interesting to assess the way these systems function in practice.

Practice Briefs from the Civil Society Perspective: Civil Society’s Regional Network and Using Information for Accountability and Justice
This Practice Brief contains an excellent description of the work of the Alliance across the region. It might benefit from adding in a bit of a description of the global network which preceded it, FOIAnet, as well as at least a reference to the regional networks that exist in South Asia (SARTIAN) and Africa (AFIC). This would provide readers with a sense of the wider context in which the Alliance works, as well as examples of the work other similar groups are doing.

Once again, this presents a very good overview of some of the ways information has been used for accountability. Two comments here. First, in the absence of a full Practice Brief on participation, it might be useful to discuss here some aspects of how information can foster participation. Second, it would be interesting to add in here a section on challenges Latin American civil society has faced.

1. One useful resource which compares all of the access to information laws in the region is Mendel, Toby, The Right to Information in Latin America: A Comparative Legal Survey (2009, Quito, UNESCO).
2. See, for example, the RTI-Rating, available at: http://www.rti-rating.org/index.html.

Prepared by Toby Mendel, Executive Director, Centre for Law and Democracy