My post on the Iraq Body Count project elicited some critical comments. I don't normally go back into the comments section to defend a post, on the grounds that I have generally already said all I wanted to – but in this case an issue has arisen that I am unwilling to let pass.
A critic, an Oxford politics don, makes a serious allegation. He takes exception to what he claims are:
smears [of the Iraq Body Count Project] by association (with Herold or the Oxford Research Group) and the jeers at the thought that music PhD students might not be able to sift through media reports and classify them according to simple rules.
The Iraq Body Count’s association with Marc Herold, of the University of New Hampshire, is not a smear but a matter of record. I know this because the Iraq Body Count says so, and says it proudly:
The project takes as its starting point and builds upon the earlier work of Professor Marc Herold who has produced the most comprehensive tabulation of civilian deaths in the war on Afghanistan from October 2001 to the present, and the methodology has been designed in close consultation with him.
Professor Herold commented: "I strongly support this initiative. The counting of civilian dead looms ever more importantly for at least two reasons: military sources and their corporate mainstream media backers seek to portray the advent of precision guided weaponry as inflicting at most, minor, incidental civilian casualties when, in truth, such is is [sic] not the case; and the major source of opposition to these modern ‘wars’ remains an informed, articulate general public which retains a commitment to the international humanitarian covenants of war at a time when most organized bodies and so-called ‘experts’ have walked away from them".
The association of the project's leader, Professor John Sloboda, with the Oxford Research Group is likewise a matter of record:
He is currently Web Resources Manager for Peace News, and is undertaking consultancy work for the Oxford Research Group.
A survey that by design 'builds upon' Herold's work automatically invalidates itself thereby. It doesn't even matter if the results reported by that study are accurate in every respect: where a methodology is unsound, then the study itself must be rejected. That is how quantitative disciplines proceed.
Pace my critic, no one has asked these researchers to come up with an 'ideal rule'. All that is required is that the design characteristics of their surveys be predetermined and structured. Herold's survey was almost a textbook case – indeed it ought to be, and perhaps is, literally a textbook example – of not paying attention to this elementary requirement. His survey is presented here, and on the final page he inadvertently undermines the entire exercise by stating:
Frequently, different sources mention different civilian casualty figures, and most of the time, I have chosen the lowest number
Most of the time? What rules did he adopt the rest of the time? What was the rationale for choosing a higher estimate in those cases? It's impossible to say whether there was method and what it might have consisted in, because in an egregious violation of scientific protocol Herold refuses to have his working methods examined by those he suspects of not sharing his political opinions – as a devastating account by the neoconservative writer Joshua Muravchik recounts:
I wrote Herold again, asking for the sources I could not find and the method of his own "data compilation." He began his reply by wondering "why you are so interested" and said his failure to give authors and titles was because "I do not have a staff to assist." My other questions about sources and methods went unanswered, but he appended a brief text, explaining that its "purpose . . . is to cast doubt upon both the method and reported results of Human Rights Watch." It cited new sources: the Oman Daily Observer, Al-Ahram, the Hindustan Times, the Jordan Times, and the BBC Online. The only piece I could find in the BBC Online was one citing Herold's own account. I could not find the others through Nexis or Google or the search engines of the individual papers. Presumably they repeat the same unverified assertions that have appeared elsewhere.
Herold provided no further information. He e-mailed that he had learned I am a neoconservative and therefore answering my queries did not justify "the opportunity cost of my time…. I 'owe' you absolutely nothing."
It hardly seems necessary – indeed, it's redundant, because Herold has already discredited his work irretrievably – to point out that choosing a higher estimate in just a few cases out of hundreds can nonetheless have a huge impact on the survey's results. And if, even then, you're minded to take Herold's bogus methods seriously, Iain Murray has usefully summarised the double-counting, confusion of status between civilians and combatants, and factual inaccuracy that the survey exhibits.
I record with stupefaction that as well as being an associate professor of women's studies, Herold teaches economics. I would normally expect an economist to be familiar with methods in descriptive statistics; admittedly there are exceptional cases, but I know of none as incompetent, unscholarly and unprofessional as Marc Herold. And this is a man with whom, recall, there has been 'close consultation' in the design of the methodology for the Iraq Body Count project.
In the circumstances – indeed, in any study of this type, but never more obviously than here – it does indeed matter that not a single participant in the Iraq Body Count, with one possible tangential exception, possesses relevant expertise in techniques of collecting numerical data. The sole possible exception is the project's leader, Professor John Sloboda – but I am deliberately giving him the benefit of the doubt when in fact I have been unable to find any information suggesting he has quantitative expertise. His academic discipline is psychology, and I therefore assume without argument that he is experienced and competent in the design of controlled experiments in that field. But that is not the same as having the requisite skills to handle statistics, nor may it be taken as direct evidence of such skills.
Having tried several Internet searches of the type 'John + Sloboda + statistics', the only results I can come up are those that refer to the Iraq Body Count project, and not to any academic work. The Psychology departmental web site at Keele University lists the specialist fields of staff members available for the supervision of PhDs; statistics is not included as one of Professor Sloboda's interests, which are given as:
Psychology of music; acquisition of skill; counselling in organisations.
I have thus tried – genuinely tried, for I don't wish to be accused of ignoring or misrepresenting publicly-available information – to identify some relevant professional or scholarly competence on the part of the 'researchers' in the Iraq Body Count project. I have found none. But then it's not hard to work out what animates Marc Herold and his disciples.
After the war in Afghanistan, an antediluvian far-left polemicist called William Blum, author of several books denouncing US foreign policy, demonstrated that even an irredeemably trifling author may perpetrate monstrous indecency, with an article entitled Civilian Casualties: Theirs and Ours. His conceit was that the US overthrow of the Taliban was morally equivalent to the acts of the hijackers on September 11. (I would urge you to read the whole of this piece of piffle, though not at mealtimes, lest you doubt that an apparently significant part of the US adversary culture genuinely thinks like that.) And on what possible grounds could he have arrived at so perverse a conclusion? Have a guess.
Who killed more innocent, defenseless people? The terrorists in the United States on September 11 with their crashing airplanes? Or the U.S. Government in Afghanistan the past 10 weeks with their AGM-86D Cruise missiles, their AGM-130 missiles, their 15,000 pound "daisy cutter" bombs, their depleted uranium, and their cluster bombs?
The count in New York and Washington is now 3,062 and going down steadily. The total count of civilian dead in Afghanistan has been essentially ignored by American officials and the domestic media, but a painstaking compilation of domestic and international press reports by University of New Hampshire professor Marc Herold, hunting down the many incidents of 100-plus counts of the dead, the scores of dead, the dozens, and the smaller numbers, arrived at 3,767 through December 6, and still counting.
That, I'm afraid, is the explanation for the venture and the distortions on which it rests. For reasons more easily declaimed than explained, the anti-war campaigners devised their own moral calculus in which the war on terrorism became more culpable than the September 11 atrocities. The flaw in this was noted by the philosopher Michael Walzer, who has written extensively about the ethics of war:
A few left academics have tried to figure out how many civilians actually died in Afghanistan, aiming at as high a figure as possible, on the assumption, apparently, that if the number is greater than the number of people killed in the attacks on the Twin Towers, the war is unjust. At the moment, most of the numbers are propaganda; there is no reliable accounting. But the claim that the numbers matter in just this way-that the 3,120th death determines the injustice of the war-is wrong. It denies one of the most basic and best understood moral distinctions: between premeditated murder and unintended killing. And the denial isn't accidental, as if the people making it just forgot about, or didn't know about, the everyday moral world. The denial is willful: unintended killing by Americans in Afghanistan counts as murder. This can't be true anywhere else, for anybody else.
Walzer's moral judgement is acute; so is his analysis of the motivation of these tendentious studies. They are designed not to discover facts but to arrive at predetermined conclusions. What is true of Herold's exercise applies also to the Iraq Body Count that he has inspired and whose methodology he has advised upon. I have referred to Sloboda's work for the Oxford Research Group, an outfit that disguises its partisan political intent with a veneer of disinterested research. As I commented in my earlier post, the ORG published a 'report'- more accurately, a tract – last October on the consequences of an Iraq war. It confidently asserted:
A civilian death toll of at least 10,000 should be expected but this may be a low estimate, given the experience of urban warfare in Beirut and elsewhere.
Once more, the answer was predetermined. The war in fact proved to be a much swifter affair than the peace movement – and the absurd Oxford Research Group report – claimed. There was no Stalingrad. Not even the Iraq Body Count has yet been able to claim the inflated totals initially mooted, but they have been assiduous in front-running this inevitable conclusion. As The Guardian reported its assertions in June:
At least 5,000 civilians may have been killed during the invasion of Iraq, an independent research group has claimed. As more evidence is collated, it says, the figure could reach 10,000.
You bet it says that.
The Iraq Body Count comprises not researchers with relevant professional or academic training, but a group of music enthusiasts with shared political prejudices. As well as – by their own account - adopting Herold's methods, they have failed to dispel the fundamental flaw in his approach. They claim to have cited 'multiple and independent sources', but have in practice relied on just one, which is far from independent. Iain Murray (again) identifies the problem succinctly:
The main trouble is that, while the conflict is going on [at the beginning of April], there is only one real source of information on civilian casualties, which is the Iraqi government. As is emblazoned on the [Iraq Body Count] web site's front page, General Tommy Franks has said "We don't do body counts." So when it is alleged that a bomb has hit a marketplace, the only real source of a number for civilian casualties is what the Iraqi spokesmen say. Occasionally, a report may get out of Baghdad querying this number. Yet, equally, initial reports may say something higher than the Iraqis later claim. It is therefore likely that the "minimum" figure is likely to be inflated, although the "maximum" figure is probably accurate as an upper range for claims about how many have been killed.
We don't know how many civilians died in our side's military campaigns in Afghanistan and Iraq. We do know that the casualties in Afghanistan were few compared with earlier Afghan wars and with the unspeakable atrocities committed by the regime we deposed. Whereas most reputable sources estimate that between 1000 and 2000 Afghan civilians were killed in the US-led campaign, Human Rights Watch estimated that the Taliban had killed more than that number – deliberately, systematically – of civilians when they massacred the Shi'ah Hazaras at Mazar-I-Sharif in 1998.
Of the Afghan campaign, the moral philosopher Jean Bethke Elshtain has written:
The United States must do everything it can to minimize civilian deaths – and it is doing so. The United States must express remorse for every civilian death in a way that is not simply rote – and it is doing so. The United States must investigate every incident in which civilians are killed – and it is doing so. The United States must make some sort of recompense for unintended civilian casualties, and it may be making plans to do so – an unusual, even unheard of, act in wartime.
I have no doubt, and every reason to believe, that the same may be said of the Anglo-American liberation of Iraq, even though a reliable and disinterested accounting of civilian deaths in the whole campaign remains to be published.
Of other types of accounting, we may say this. The point about Marc Herold and the Iraq Body Count is not that they have political views, but that they have only political views. They are not 'independent researchers', but propagandists. They are not even peace activists: they are mouthpieces for, respectively, Islamist and Baathist tyrannies. Their output should be judged in that light.
UPDATE: A correspondent has forwarded to me a curious message he has received on this subject. I have not included the name of the person who sent it to him, in line with my usual policy. Suffice to say that the message is not a spoof but is intended seriously. I have previously come across the name of the person who sent it; he is, as he says, a musician (he has a site promoting his troupe), and he is a relentless contributor to various far-Left Internet discussion fora, while not obviously - on that evidence - being in possession of a firm grasp of numbers. The message reads as follows:
My name is [...] and I'm a participant in the Iraq Body Count project. I recently saw your rebuttal to Oliver Kamm's piece at the website: http://oliverkamm.typepad.com/blog/2003/09/pilger_truth_an.html. Your rebuttal to this piece was perfectly sound and elementary, which is probably why it has received precious little meaningful response in the thread. You were correct to point out that Oliver's piece was mostly regurgitations of previous claims by writers like Chavetz (claims which have little relevance since the few Iraqi government claims from the early days of the war have mostly been surplanted [sic] by other sources since, as you pointed out), and that really his only new contribution to the debate was implying that since many of us are musicians, we aren't qualified to do this work. Or, even more ludicrous, his implication that many of our members are claiming "authority" on this issue on the basis of our musical training described in the bio page of our members. Or still more ludicrous his "guilt by association" implications, which you alluded to. In any case, I just wanted to write and thank you for taking the time to rebut some of the things written against us and for taking the time to look a bit more closely at our site. I think if more people took a closer look at how our site works and the methods we try to uphold, they'd be less likely to be fooled by pieces like those of Mr. Kamm.
Unfortunately my correspondent's comment on my original post contained none of the remarks attributed to him. It began, "Great post, and badly needed, Oliver. Thanks." It thereafter provided a point of supporting evidence for my criticisms of the Iraq Body Count project. As my correspondent states in some perplexity:
These guys live in an parallel universe.
For the record, I have no problem whatever with musicians' participation in a survey. I am also all in favour of 'holistic critiques' of Mozart operas, especially as my wife (who lately sang the part of Susanna in the Royal Danish Opera production of The Marriage of Figaro) and I are generally familiar only with musical and dramatic ones, doubtless owing to our bourgeois false consciousness. But when a survey contains no one of specialist professional or academic competence in the handling of data, and when it so clearly has been constructed to serve an ideological purpose, it is neither improper nor an ad hominem charge to point this out.