Where angels fear to tread

The New Zealand Drug Harm Index, published in June, was developed by Business and Economic Research Limited as a tool for the Police to assess and quantify social harms resulting from illicit drug use. Alison Ritter offers a review of the index and suggests it remains useful despite some significant flaws.

-----

Evaluating the effectiveness of illicit drug policy is enormously difficult. This is partly because the impacts of drug use span multiple domains: health, crime, social amenity and so on. Measuring the diverse aspects of drug policy involves combining various consequences (such as health and crime outcomes) with prevalence and consumption.

A single index is often a nice solution to the problem of combining different types of consequences across domains, and dollars is a common unit of measurement that can summarise data across them.

Comparisons within a country or region over time, between regions or countries and across domains of policy initiative (such as law enforcement, treatment, harm reduction and prevention) are all difficult without a composite index. It is for these reasons that the development of an index reported in cost terms is useful.

However, the work is fraught (hence this article’s title). It is methodologically complicated, conceptually challenging and the data are limited even when they exist. Furthermore there will always be criticism. The ecological footprint (a single index that measures impact on the environment) has any number of criticisms ranging from conceptual to methodological. Despite this, the ecological footprint as an index has proven useful in environmental policy. My hope is that a drug harm index can also be useful.

The drugs field has been engaged in work across the globe on composite drug indexes. For example, there is the UK Drug Harm Index, the UNODC Illicit Drug Index, the Australian Drug Policy Index (DPMP) and the Australian Federal Police (AFP) Drug Harm Index. Each index is constructed differently and for different purposes.

The recently published New Zealand Drug Harm Index adds to this list. The report identifies three components of the work: firstly, to estimate the social costs of illicit drugs to New Zealand; secondly, to calculate a ‘harm-perkilogram-consumed’ measure; thirdly, the translation of those figures into a single index that can be used for performance monitoring purposes.

The first exercise is useful in demonstrating the extent of financial burden of illicit drugs borne by the community. The results indicate that, in 2005/06, the total social cost burden of illicit drugs in New Zealand was $1.3 billion dollars.

The social costs include direct consequences associated with illicit drug use, such as criminal activity, healthcare and road accidents, and indirect costs such as productivity lost. It is very easy to be highly critical of the method and arithmetic used in the calculations – data are often unavailable so proxy information is used to derive estimates. For example, many of the estimates used information from Australia.

The decision about what data to include is also fraught. Where good data exist such as hospital inpatient services or road accidents, it is used extensively. Where limited data exist such as public amenity, the domain is not included.

Aside from estimating the consequences (such as health and crime), there are substantial data uncertainties regarding the population itself.

The most obvious of these are prevalence of drug use, consumption rates, distinctions between occasional and frequent users and polydrug use. The total social cost estimate cannot be converted to a ‘harm-per-kilogram-consumed’ unit without the population data. All the usual difficulties with population prevalence of illegal behaviours as assessed through random telephone surveys apply here. Likewise, there are significant challenges in estimating consumption levels.

Having established the total social cost of illicit drug use, and the prevalence and consumption levels, these two sets of data can then be combined to estimate a per user social cost and a per kilogram social cost. As should now be apparent, as we work through each aspect of the index (total social cost, unit cost per kilogram and the final index), the degree of confidence in the measure becomes more and more weak.

Once we arrive at the actual index – which compares the social costs of seizures across the years 2000 to 2006, we are on quite shaky ground. The index multiplies the social cost per kilogram of drug by the kilograms seized by law enforcement. A total annual estimate is then derived.

Interestingly, the index shows that the largest seizure social costs were from cannabis. But this is an artefact of the index – the per kilogram social cost of cannabis is $11,790 as compared to the per kilogram social cost of opioids at $1,074,130. But because the police seize substantially more cannabis than opioids, the total index largely reflects cannabis social costs savings. This, in itself, may be misleading from a policy perspective.

More important, however, is the basic assumption underpinning the index that seized drugs are not replaced. Therefore, the social costs associated with the quantity seized are realised savings. The report notes, “The harm per kilogram estimates indicate the gross economic benefit of drug seizures.” This is false, or at least it is only true if those seized quantities are not replaced. In the absence of evidence about changing or reduced consumption, the assumption about seized quantities is unsupported. The authors do point this out and use the term “potential harm avoided” in the last section of the report, but this important point requires constant clarification.

Despite this fundamental flaw in the usage of the index, the work is important for a number of reasons. It provides further indications of what we do not know and can inform data collection activities. The overall social cost estimate for New Zealand is a useful figure, at least for political and funding purposes.

Methodological advances have been made that can inform international efforts at index development. The New Zealand work has made some methodological advances in the area of productivity losses and in handling the problem of polydrug use.

Finally, the development of indexes that can be used either to monitor the effectiveness of government actions over time (like the UK DHI) or that can be used to compare policy investment options (like the DPMP Policy outcome tool) have a role to play in improving the evidence base and making our field accountable.

• Professor Alison Ritter is Director of the Drug Policy Modelling Program, an Australian-based multi-disciplinary initiative to provide big picture analyses of policies relating to illicit drugs, www.dpmp.unsw.edu.au.