Tag Archives: Mark Kleiman

When is subtle manipulation of data a flat out lie? Truth about Chinese prisons [UPDATE]

Share
Send to Kindle

I’ve been on a China kick lately (reading and listening about its history and global position) and a crime public policy kick (reading and listening to Mark Kleiman). I was struck when I heard Mark say in an interview that the US has more people in jail in absolute terms than China. So I went about to looking for some data. I found the most comprehensive source of info in the “World Prison Population List” published by the King’s College London International Centre for Prison Studies. Their top bullet point is alarming:

More than 9.25 million people are held in penal institutions throughout the world, mostly aspre-trial detainees (remand prisoners) or assentenced prisoners. Almost half of these are inthe United States (2.19m), China (1.55m plus pre-trial detainees and prisoners in ‘administrativedetention’) or Russia (0.87m).

But I was surprised by China. The US have a 738  people in prison per 100,000 of population, Russia 611 and China 111. England and Wales has more than China with 158. In fact, more than half of the countries of the world have more than China. I did some numbers in the spreadsheet below what that means with respect to the total population of each countries (throwing in the UK, India and Brazil for good measure):

And the results could not be clearer. China is not in any way comparable to Russia and the US when it comes to prison population. In fact, the UK is a worse offender (pun intended) when it comes to owning a disproportionate chunk of the global prison population. It is just under parity. India is by far the most lenient when it comes to incarceration with only 3.5% of the world prison population to 16% of the world’d global population. The Center provides no estimate of the pre-trial and administrative detainees in China. But even if it was another half-a-million people, it would still only give China a parity. To be as disproportionately prison-happy as the US, China would have to arrest more than 2.5 as many people as it has in jail right now.

But the question arises why did the Centre for Prison Studies choose to include US, Russia and China on the same list? My suggestion is prejudice combined with number magic. The authors were trying to come up with a way to get to say that half of the prisoners are in a small number of countries. And China is “known” for its human rights record, so it must be OK to list it there if it will bump up the numbers. But in effect, they managed to lie about China by saying something numerically true. It didn’t say anything flat out incorrect but it created an implicit category which clearly labels China as a bad country. This is a silly way to affirm Western supremacy where there is none.

There are lots of other things that could be estimated based on these numbers. I couldn’t find a clear estimate of how many people were sent to prison for things they didn’t do (we can’t just extrapolate from death row exonerations) but if we set it at about 0.5%, we get that there may be more unjustly imprisoned people in the US than there are political prisoners in China (estimated at about 5,000) or slightly less if we count the same rate of miscarriage of justice across the rest of China’s prison population. This is, of course, too much guess work for drawing any firm conclusions but it certainly puts the numbers in some perspective.

UPDATE: I have actually interviewed Mark Kleiman (it was a long time ago but I only now remembered to update here) and his estimate is that there are 3-4% of people in US prisons who are there because of something they did not do (often because of police mis-behavior). Now it is important to qualify this by saying that most of these people have done other things for which they deserve to go to prison but were not caught, so the miscariage of justice is more technical than moral. But it shows the massive holes in the US vaunted “rule of law”. It is there, no doubt, when it comes to settling middle-class property and other business disputes (and by all accounts this would be very important thing to have in many countries in the Middle East and China). But it is not evenly distributed. I think it would not be completely outrageous to say that, for many of its citizens, the US is in effect a police state. Just like it could be said that for many of China’s citizens, China is not!

Enhanced by Zemanta
Send to Kindle

Epistemology as ethics: Decisions and judgments not methods and solutions for evidence-based practice

Share
Send to Kindle
Common (rapper) Common Sense (rapper) Tufts Un...

Image via Wikipedia

Show me the money! Or so the saying goes. Implying that talk is cheap and facts are the only thing that matters. But there is another thing we are being asked to do with money and that is put it where our mouth is. So evidence is not quite enough. We have to also be willing to act on it and demonstrate a personal commitment to our facts.

As so often, folk wisdom has outlined two contradictory dictums that get applied based on the parameters of a given situation. On one account, evidence is all that is needed. Wheras an alternative interpretation claims that evidence is only sufficient when backed up by personal commitment.

I’ve recently come across two approaches to the same issue that resonate with my own, namely claiming that evidence cannot be divorced from the personal commitment of its wielder. As such it needs to be constantly interrogated from all perspectives, including the ones that come from “scientific method” but not relying solely on them.

This is crucial for anybody claiming to adhere to evidence-based practice. Because the evidence is not just out there. It is always in the hands of someone caught (as we all are) within a complex web of ethical commitments. Sure, we can point to a number of contexts where lack of evidence was detrimental (vaccinations). But we can equally easily find examples of evidence being the wrong thing to have (UK school league tables, eugenics). History of science demonstrates both incorrect conlusions being drawn from correct facts (ether), and correct conclusions being drawn form incorrect facts (including aircraft construction as Ira Flatow showed). I am not aware of any research quantifying the proportion of these respective positions. Intuitively, one would feel inclined to conclude that mostly the correct facts have led to the correct conclusions and incorrect ones to the wrong ones but we have been burned before.

That’s why I particularly liked what Mark Kleiman had to say at the end of his lecture on Evidence-based practice in policing:

“The notion that we can substitute method for common sense, a very wide-spread notion, is wrong. We’re eventually going to have to make judgments. Evidence-based practice is a good slogan, but it’s not a method in the Cartesian sense. It does not guarantee that what we’re going to do is right.”

Of course, neither is the opposite true. I’m always a bit worried about appeals to “common sense”.  I used to tell participants on cross-cultural training courses: there is nothing common about common sense. It always belongs to someone. And it does not come naturally to us. It is only common because someone told someone else about it. How often, have we heard a myth debunked only to conclude that the debunking had really been common sense. That makes no sense. It was the myth that was common sense. The debunking will only become common sense after we tell everyone about it. How many people would figure out even the simplest things such as looking right and then left before crossing the road without someone telling them? We spend a lot of time (as a society and individuals) maintaining the commonality of our senses. Clifford Geertz showed in “Common sense as a cultural system” the complex processes involved in common sense and I would not want to entrust anything of importance to common sense. But I agree that judgments are essential. We always make judgements whether we realise that or not.

This resonates wonderfully with what I consider to be the central thesis of the “Science’s First Mistake” a wonderful challenge to the assumptions of the universality of science by Ian Angell and Dionysios Demetis.

“…different ages have different perceptions of uncertainty; and so there are different approaches to theory construction and application, delivering different risk assessments and prompting different decisions. Note this book stresses decisions not solutions, because from its position there are no solutions, only contingent decisions. And each decision is itself a start of a new journey, not the end of an old one.

Indeed, there is no grander delusion than the production of a solution, with its linear insistence on cause and effect.”

All our solutions are really decisions. Decisions contingent on a myriad of factors both related to the data and our personal situation. We couch these decisions in the guise of solutions to avoid personal responsibility. Not perhaps often in a conscious way but in a way that abstracts away from the situation. We do not always have to ask qui bono when presented with a solution but we should always ask where from. What is the situation in which a solution was born? Not because we have a suspicious mind but because we have an inquisitive one.

But decisons are just what they are. They are on their own neither good or bad. They are just inevitable. And here I’d like to present the conclusions to a paper on the epistemological basis of case study in educational policy making I co-wrote a few years ago with John Elliott. It’s longer (by about a mile) than the two extracts above. But I highlighted in bold the bits that are relevant. The rest is there just to provide sufficient context to understand the key theses:

from the Summary and Conclusions of “Epistemology as ethics in research and policy: Under what terms might case studies yield useful knowledge to policy makers. by John Elliott and Dominik Lukeš. In: Evidence-based Education Policy: What Evidence? What Basis? Whose Policy?, 2008.”

Overall, we have informed our inquiry with three perspectives on case study. One rooted in ethnography and built around the metaphor of the understanding of cultures one is not familiar with (these are presented by the work of scholars like Becker, Willis, Lacey, Wallace, Ball, and many others). Another strand revolves around the tradition of responsive and democratic evaluation and portrayal represented by the work of scholars such as MacDonald, Simons, Stake, Parlett, and others. This tradition (itself not presenting a unified front) aims to break down the barriers between the researcher, the researched and the audience. It recognizes the situated nature of all actors in the process (cf. Kemmis 1980) and is particularly relevant to the concerns of the policy maker. Finally, we need to add Stenhouse’s approach of contemporary history into the mix. Stenhouse provides a perspective that allows the researcher to marry the responsibility to data with the responsibility to his or her environment in a way that escapes the stereotypes associated with the ‘pure forms’ of both quantitative and qualitative research. A comprehensive historian-like approach to data collection, retention and dissemination that allows multiple interpretations by multiple actors accounts both for the complexity of the data and the situation in the context of which it is collected and interpreted.

However, we can ask ourselves whether a policy-maker faced with examples of one of these traditions could tell these perspectives apart? Should it not perhaps be the lessons from the investigation of case study that matter rather than an ability to straightforwardly classify it as an example of one or the other? In many ways, in the policy context, it is the act of choice, such as choosing on which case study a policy should be based, that is of real importance.

In that case, instead of transcendental arbitration we can provide an alternative test: Does the case study change the prejudices of the reader? Does it provide a challenge? Perhaps our notion of comprehensiveness can include the question: Is the case study opening the mind of the reader to factors that they would have otherwise ignored? This reminds us of Gadamer’s “fusion of horizons” (“Understanding [...] is always the fusion of […] horizons [present and past] which we imagine to exist by themselves.” Gadamer 1975, p. 273).

However, this could be seen as suggesting that case study automatically leads to a state of illumination. In fact, the interpretation of case study in this way requires a purposeful and active approach on the part of the reader. What, then, is the role of the philosopher? Should each researcher have a philosopher by his or her side? Or is it necessary, as John Elliott has long argued, to locate the philosopher in the practitioner? Should we expect teachers and/or policy-makers to go to the philosopher for advice or would a better solution be to strive for philosopher teachers and philosopher practitioners? Being a philosopher is of course determined not by speaking of Plato and Rousseau but by the constant challenge to personal and collective prejudices.

In that case, we can conclude that case study would have a great appeal to the politician and policy maker as a practical philosopher but that it would be a mistake to elevate it above other ways of doing practical philosophy. In this, following Gadamer, we advocate an antimethodological approach. The idea of the policy maker as philosopher and policy maker as researcher (i.e. underscoring the individual ethical agency of the policy maker) should be the proper focus of discussion of reliability and generalization. Since the policy maker is the one making the judgement, the type of research and study is then not as important as a primary focus.

And, in a way, the truthfulness of the case arises out of the practitioners’ use of the study. The judgment of warrant as well as the universalizing and revelatory nature of a particular study should become apparent to anybody familiar with the complexities of the environment. An abstract standard of quality reminiscent of statistical methods (number of interviewees, questions asked, sampling) is ultimately not a workable basis for decision making and action although that does not exclude the process of seeking a shared metaphorical perspective both on the process of data gathering and interpretation (cf. Kennedy 1979, Fox 1982). Gadamer’s words seem particularly relevant in this context: “[t]he understanding and interpretation of texts is not merely a concern of science, but is obviously part of the total human experience of the world.” (1975, p. xi)

We cannot discount the situation of the researcher no more than we can discount the situation of the researched. One constitutive element of the situation is the academic tribe: “[w]e are pursuing a ‘scholarly’ identity through our case studies rather than an intrinsic fascination with the phenomena under investigation” warns Fox (1982). To avoid any such accusations of impropriety social science cultivates a ‘prejudice against prejudice’, a distancing from experience and valuing in order to achieve objectivity whereas the condition of our understanding is that we have prejudices and any inquiry undertaken by ‘us’ needs to be approached in the spirit of a conversation with others; the conversation alerts participants to their prejudices. In a sense, the point of conversation is to reconstruct prejudices, which is an alternative view of understanding itself.

It should be stressed however, that this conversation does not automatically lead to a “neater picture” of the situation nor does it necessarily produce a “social good”. There is the danger of viewing ‘disciplined conversation’ as an elevated version of the folk theory on ideal policy: ‘if only everyone talked to one another, the world would be a nicer place’. Academic conversation (just like any democratic dialectic process) is often contentious and not quite the genteel affair it tries to present itself as. Equally, any given method of inquiry, including and perhaps headed by case study, can be both constitutive and disruptive of our prejudices.

Currently the culture of politeness aimed at avoiding others’ and one’s own discomfort at any cost contributes to the problem. Can one structure research that enables people to reflect about prejudices that they inevitably bring to the situation and reconstruct their biases to open up the possibility of action and not cause discomfort to themselves or others? We could say that concern with generalization and method is a consequence of academic discourse and culture and one of the ways in which questions of personal responsibility are argued away. Abstraction, the business of academia, is seen as antithetical to the process of particularization, the business of policy implementation. But given some of the questions raised in this paper, we should perhaps be asking whether abstraction and particularization are parallel processes to which we ascribe polar directionality only ex post facto. In this sense we can further Nussbaum’s distinction between generalization and universalization by rephrasing the dichotomy in the following terms: generalization is assumed to be internal to the data whereas universalization is a situated human cognitive and affective act.

A universalizable case study is of such quality that the philosopher policy maker can discern its relevance for the process of policy making (similarly to Stake’s naturalistic generalization). This is a different way of saying the same thing as Kemmis (1980, p. 136): “Case study cannot claim authority, it must demonstrate it.” The power of case study in this context can be illustrated by anecdotes from the field where practitioners had been convinced that a particular case study describes their situation and berating colleagues or staff for revealing intimate details of their situation, whereas the case study had been based on research of an entirely unrelated entity. In cases like these, the universal nature of the case study is revealed to the practitioner. Its public aspects often engender action where idle rumination and discontent would be otherwise prevalent. Even when this kind of research tells people what they already “know”, it can inject accountability by rendering heretofore private knowledge public. The notion of case study as method with transcendental epistemology therefore cannot be rescued even by attempts like Bassey’s (1999) to offer ‘fuzzy generalizations’ since while cognition and categorization is fuzzy, action involves a commitment to boundaries. This focus on situated judgement over transcendental rationality in no way denies the need for rigour and instrumentalism. We agree with Stenhouse that there needs to be a space in which the quality of a particular case study can be assessed. But such judgments will be different when made by case study practitioners and when made by policy-makers or teachers. The epistemological philosopher will apply yet another set of criteria. All these agents would do well to familiarize themselves with the criteria applied by the others but they would be unwise to assume that they can ever fully transcend the situated parameters of their community of practice and make all boundaries (such as those described by Kushner 1993) disappear.

Philosophy often likes to position itself in the role of an independent arbiter but it must not forget that it too is an embedded practice with its own community rules. That does not mean there’s no space for it in this debate. If nothing else, it can provide a space (not unlike the liminal space of Turner’s rituals) in which the normal assumptions are suspended and transformation of the prejudice can occur. In this context, we should perhaps investigate the notion of therapeutic reading of philosophy put forth by the New Wittgensteinians (see Crary and Read 2000).

This makes the questions of ethics alluded to earlier even more prominent. We propose that given their situated nature, questions of generalization are questions of ethics rather than inquiries into some disembodied transcendent rationality. Participants in the complex interaction between practitioners of education, educational researchers and educational policy makers are constantly faced with ethical decisions asking themselves: how do I act in ways that are consonant with my values and goals? Questions of warrant are internal to them and their situation rather than being easily resolvable by external expert arbitration. This does not exclude instrumental expertise in research design and evaluation of results but the role of such expertise is limited to the particular. MacDonald and Walker (1977) point out the importance of apprenticeship in the training of case study practitioners and we should bear in mind that this experience cannot be distilled into general rules for research training as we find them laid out in a statistics textbook.

Herein can lie the contribution of philosophy: An inquiry into the warrant and generalization of case study should be an inquiry into the ethics surrounding the creation and use of research, not an attempt to provide an epistemologically transcendent account of the representativeness of sampled data.

 

Enhanced by Zemanta
Send to Kindle

Do science fiction writers dream of fascist dictatorships?

Share
Send to Kindle
This is the cover to the January 1953 issue of...

Image via Wikipedia

Some years ago in a book review, I made an off-the-cuff comment that thriller writers tend to be quite wright-wing in their outlook whereas science fiction authors are much more progressive and leftist. This is obviously an undue generalisation (as most of such comments tend to be) but it felt intuitively right. Even then I thought of Michael Chricton as the obvious counterexample – a thirller writer with distinctly liberal leanings – but I couldn’t think of a science fiction writer that would provide the counterexample. I put this down to my lack of comprehensive sci-fi reading and thought nothing more of it. Now, I’m not even sure that the general trend is there or at least that the implications are very straightforward.

Recently, I was listening to the excellent public policy lectures by Mark Kleiman and remembered that years ago, I’d read some similar suggestions in the Bio of the Space Tyrant by Pierce Anthony. It wasn’t a book (or rather a book series) I was going to reread but I set to it with a researcher’s determination. And frankly, I was shocked.

What I found was not a vision of a better society (Anthony projects the global politics of his day – the early 80s – into the Solar system 600 years hence) but rather a grotesque depiction of what the elites of the day would consider ‘common-sense’ policies: free-market entrepreneurship with social justice with a few twists. It was anti-corporalist and individualist on the surface but with a strong sense of collective duty (and pro-militarism) that was much more reminiscent of fascism than communism. It espoused strong, charismatic leadership with a sense of duty and most of all a belief in the necessity of change led by common sense. The needs of the collective justified the suppression of the individual in almost any way. But all of this is couched in good liberal politics (like the free press, free enterprise, etc.)

It is not clear whether Anthony means this as a parody of a fascist utopia but there are no hints there that this is the case. The overwhelming sense I get from this book is one of frustration of the intellectual elite that nobody is listening to what they have to say and a perverted picture of what the world would be like if they only got to start over with their policies.

Speaking of perverted. Through all of this is woven a bizzare and disturbing mix of patriarchy and progressive gender politics. On the one hand, Anthony is strongly against violence against women and treats women as strong and competent individuals. But on the other hand, his chief protagonist is an embodiment of a philanderer’s charter. All women love him but understand that he cannot love just one! The policies are there but what is to prevent any man from feeling that he is the one exception. So despite the progressive coating one is left feeling slightly unclean.

Now, is Anthony the exception I was looking for years ago? I don’t think so. First, I think he would fall in the liberal to libertarian camp if asked. But second, I don’t think he’s any exception at all. I recently reread some SciFi classics and found hints of to full blown monuments to this rationalist yearning for control over society – the “if they only listened to us” syndrome, also known as the “TED syndrome”. We understand so much about how things work, so now we have the solution for how everything works. That’s why we should never seek to be ruled by philosopher kings (Plato, Hobbes, and any third rate philosopher – more likely to be fascist than liberal). Classics like “Mote in God’s Eye“, “Starship Troupers” (the movie was a parody but the book wasn’t), “Foundation” or less well-known ones like “The Antares Trilogy” or “The Lost Fleet“. They all unwittingly struggle with the dilemma of we know what to do but we know it can’t be achieved unless we have complete control. I found echoes of this even in cyberpunk like Snowcrash.

So am I seeing a trend that isn’t there? I’m not as widely read in SciFi as other genres so it’s possible I just happened on books that confirm my thesis (such as it is). Again, the exception I can think of immediately is Cory Doctorow whose “For the Win” is as beautiful and sincere a depiction of the union movement as any song by Pete Seger. And I’m sure there are many more. But are there enough to make my impression just that? (Of course, there’s SciFi  where this doesn’t come up, at all.)

But this tendency of the extremely intelligent and educated (and SciFi writers are on the whole just as well versed in anthropology as they are in science) to tell stories of how their images of the just society can be projected onto society as a whole is certainly a worrying presence in the genre. It seems to be largely absent From fantasy, which generally deals with journeys of individuals within existing worlds. And while these worlds maybe dystopic, they generally are not changed, only explored. Fantasy has a strong thread of historical nostalgia – looking for a pure world of yore – which can be quite destructive when mis-projected to our own world. But on the whole, I feel, it contains less public policy than the average science fiction novel.

Enhanced by Zemanta
Send to Kindle