More truth and lies in the war on terror

My post on the Iraq Body Count project elicited some critical comments. I don't normally go back into the comments section to defend a post, on the grounds that I have generally already said all I wanted to – but in this case an issue has arisen that I am unwilling to let pass.

A critic, an Oxford politics don, makes a serious allegation. He takes exception to what he claims are:

smears [of the Iraq Body Count Project] by association (with Herold or the Oxford Research Group) and the jeers at the thought that music PhD students might not be able to sift through media reports and classify them according to simple rules.

The Iraq Body Count’s association with Marc Herold, of the University of New Hampshire, is not a smear but a matter of record. I know this because the Iraq Body Count says so, and says it proudly:

The project takes as its starting point and builds upon the earlier work of Professor Marc Herold who has produced the most comprehensive tabulation of civilian deaths in the war on Afghanistan from October 2001 to the present, and the methodology has been designed in close consultation with him.

Professor Herold commented: "I strongly support this initiative. The counting of civilian dead looms ever more importantly for at least two reasons: military sources and their corporate mainstream media backers seek to portray the advent of precision guided weaponry as inflicting at most, minor, incidental civilian casualties when, in truth, such is is [sic] not the case; and the major source of opposition to these modern ‘wars’ remains an informed, articulate general public which retains a commitment to the international humanitarian covenants of war at a time when most organized bodies and so-called ‘experts’ have walked away from them".

The association of the project's leader, Professor John Sloboda, with the Oxford Research Group is likewise a matter of record:

He is currently Web Resources Manager for Peace News, and is undertaking consultancy work for the Oxford Research Group.

A survey that by design 'builds upon' Herold's work automatically invalidates itself thereby. It doesn't even matter if the results reported by that study are accurate in every respect: where a methodology is unsound, then the study itself must be rejected. That is how quantitative disciplines proceed.

Pace my critic, no one has asked these researchers to come up with an 'ideal rule'. All that is required is that the design characteristics of their surveys be predetermined and structured. Herold's survey was almost a textbook case – indeed it ought to be, and perhaps is, literally a textbook example – of not paying attention to this elementary requirement. His survey is presented here, and on the final page he inadvertently undermines the entire exercise by stating:

Frequently, different sources mention different civilian casualty figures, and most of the time, I have chosen the lowest number

Most of the time? What rules did he adopt the rest of the time? What was the rationale for choosing a higher estimate in those cases? It's impossible to say whether there was method and what it might have consisted in, because in an egregious violation of scientific protocol Herold refuses to have his working methods examined by those he suspects of not sharing his political opinions – as a devastating account by the neoconservative writer Joshua Muravchik recounts:

I wrote Herold again, asking for the sources I could not find and the method of his own "data compilation." He began his reply by wondering "why you are so interested" and said his failure to give authors and titles was because "I do not have a staff to assist." My other questions about sources and methods went unanswered, but he appended a brief text, explaining that its "purpose . . . is to cast doubt upon both the method and reported results of Human Rights Watch." It cited new sources: the Oman Daily Observer, Al-Ahram, the Hindustan Times, the Jordan Times, and the BBC Online. The only piece I could find in the BBC Online was one citing Herold's own account. I could not find the others through Nexis or Google or the search engines of the individual papers. Presumably they repeat the same unverified assertions that have appeared elsewhere.

Herold provided no further information. He e-mailed that he had learned I am a neoconservative and therefore answering my queries did not justify "the opportunity cost of my time…. I 'owe' you absolutely nothing."

It hardly seems necessary – indeed, it's redundant, because Herold has already discredited his work irretrievably – to point out that choosing a higher estimate in just a few cases out of hundreds can nonetheless have a huge impact on the survey's results. And if, even then, you're minded to take Herold's bogus methods seriously, Iain Murray has usefully summarised the double-counting, confusion of status between civilians and combatants, and factual inaccuracy that the survey exhibits.

I record with stupefaction that as well as being an associate professor of women's studies, Herold teaches economics. I would normally expect an economist to be familiar with methods in descriptive statistics; admittedly there are exceptional cases, but I know of none as incompetent, unscholarly and unprofessional as Marc Herold. And this is a man with whom, recall, there has been 'close consultation' in the design of the methodology for the Iraq Body Count project.

In the circumstances – indeed, in any study of this type, but never more obviously than here – it does indeed matter that not a single participant in the Iraq Body Count, with one possible tangential exception, possesses relevant expertise in techniques of collecting numerical data. The sole possible exception is the project's leader, Professor John Sloboda – but I am deliberately giving him the benefit of the doubt when in fact I have been unable to find any information suggesting he has quantitative expertise. His academic discipline is psychology, and I therefore assume without argument that he is experienced and competent in the design of controlled experiments in that field. But that is not the same as having the requisite skills to handle statistics, nor may it be taken as direct evidence of such skills.

Having tried several Internet searches of the type 'John + Sloboda + statistics', the only results I can come up are those that refer to the Iraq Body Count project, and not to any academic work. The Psychology departmental web site at Keele University lists the specialist fields of staff members available for the supervision of PhDs; statistics is not included as one of Professor Sloboda's interests, which are given as:

Psychology of music; acquisition of skill; counselling in organisations.

I have thus tried – genuinely tried, for I don't wish to be accused of ignoring or misrepresenting publicly-available information – to identify some relevant professional or scholarly competence on the part of the 'researchers' in the Iraq Body Count project. I have found none. But then it's not hard to work out what animates Marc Herold and his disciples.

After the war in Afghanistan, an antediluvian far-left polemicist called William Blum, author of several books denouncing US foreign policy, demonstrated that even an irredeemably trifling author may perpetrate monstrous indecency, with an article entitled Civilian Casualties: Theirs and Ours. His conceit was that the US overthrow of the Taliban was morally equivalent to the acts of the hijackers on September 11. (I would urge you to read the whole of this piece of piffle, though not at mealtimes, lest you doubt that an apparently significant part of the US adversary culture genuinely thinks like that.) And on what possible grounds could he have arrived at so perverse a conclusion? Have a guess.

Who killed more innocent, defenseless people? The terrorists in the United States on September 11 with their crashing airplanes? Or the U.S. Government in Afghanistan the past 10 weeks with their AGM-86D Cruise missiles, their AGM-130 missiles, their 15,000 pound "daisy cutter" bombs, their depleted uranium, and their cluster bombs?

The count in New York and Washington is now 3,062 and going down steadily. The total count of civilian dead in Afghanistan has been essentially ignored by American officials and the domestic media, but a painstaking compilation of domestic and international press reports by University of New Hampshire professor Marc Herold, hunting down the many incidents of 100-plus counts of the dead, the scores of dead, the dozens, and the smaller numbers, arrived at 3,767 through December 6, and still counting.

That, I'm afraid, is the explanation for the venture and the distortions on which it rests. For reasons more easily declaimed than explained, the anti-war campaigners devised their own moral calculus in which the war on terrorism became more culpable than the September 11 atrocities. The flaw in this was noted by the philosopher Michael Walzer, who has written extensively about the ethics of war:

A few left academics have tried to figure out how many civilians actually died in Afghanistan, aiming at as high a figure as possible, on the assumption, apparently, that if the number is greater than the number of people killed in the attacks on the Twin Towers, the war is unjust. At the moment, most of the numbers are propaganda; there is no reliable accounting. But the claim that the numbers matter in just this way-that the 3,120th death determines the injustice of the war-is wrong. It denies one of the most basic and best understood moral distinctions: between premeditated murder and unintended killing. And the denial isn't accidental, as if the people making it just forgot about, or didn't know about, the everyday moral world. The denial is willful: unintended killing by Americans in Afghanistan counts as murder. This can't be true anywhere else, for anybody else.

Walzer's moral judgement is acute; so is his analysis of the motivation of these tendentious studies. They are designed not to discover facts but to arrive at predetermined conclusions. What is true of Herold's exercise applies also to the Iraq Body Count that he has inspired and whose methodology he has advised upon. I have referred to Sloboda's work for the Oxford Research Group, an outfit that disguises its partisan political intent with a veneer of disinterested research. As I commented in my earlier post, the ORG published a 'report'- more accurately, a tract – last October on the consequences of an Iraq war. It confidently asserted:

A civilian death toll of at least 10,000 should be expected but this may be a low estimate, given the experience of urban warfare in Beirut and elsewhere.

Once more, the answer was predetermined. The war in fact proved to be a much swifter affair than the peace movement – and the absurd Oxford Research Group report – claimed. There was no Stalingrad. Not even the Iraq Body Count has yet been able to claim the inflated totals initially mooted, but they have been assiduous in front-running this inevitable conclusion. As The Guardian reported its assertions in June:

At least 5,000 civilians may have been killed during the invasion of Iraq, an independent research group has claimed. As more evidence is collated, it says, the figure could reach 10,000.

You bet it says that.

The Iraq Body Count comprises not researchers with relevant professional or academic training, but a group of music enthusiasts with shared political prejudices. As well as – by their own account - adopting Herold's methods, they have failed to dispel the fundamental flaw in his approach. They claim to have cited 'multiple and independent sources', but have in practice relied on just one, which is far from independent. Iain Murray (again) identifies the problem succinctly:

The main trouble is that, while the conflict is going on [at the beginning of April], there is only one real source of information on civilian casualties, which is the Iraqi government. As is emblazoned on the [Iraq Body Count] web site's front page, General Tommy Franks has said "We don't do body counts." So when it is alleged that a bomb has hit a marketplace, the only real source of a number for civilian casualties is what the Iraqi spokesmen say. Occasionally, a report may get out of Baghdad querying this number. Yet, equally, initial reports may say something higher than the Iraqis later claim. It is therefore likely that the "minimum" figure is likely to be inflated, although the "maximum" figure is probably accurate as an upper range for claims about how many have been killed.

We don't know how many civilians died in our side's military campaigns in Afghanistan and Iraq. We do know that the casualties in Afghanistan were few compared with earlier Afghan wars and with the unspeakable atrocities committed by the regime we deposed. Whereas most reputable sources estimate that between 1000 and 2000 Afghan civilians were killed in the US-led campaign, Human Rights Watch estimated that the Taliban had killed more than that number – deliberately, systematically – of civilians when they massacred the Shi'ah Hazaras at Mazar-I-Sharif in 1998.

Of the Afghan campaign, the moral philosopher Jean Bethke Elshtain has written:

The United States must do everything it can to minimize civilian deaths – and it is doing so. The United States must express remorse for every civilian death in a way that is not simply rote – and it is doing so. The United States must investigate every incident in which civilians are killed – and it is doing so. The United States must make some sort of recompense for unintended civilian casualties, and it may be making plans to do so – an unusual, even unheard of, act in wartime.

I have no doubt, and every reason to believe, that the same may be said of the Anglo-American liberation of Iraq, even though a reliable and disinterested accounting of civilian deaths in the whole campaign remains to be published.

Of other types of accounting, we may say this. The point about Marc Herold and the Iraq Body Count is not that they have political views, but that they have only political views. They are not 'independent researchers', but propagandists. They are not even peace activists: they are mouthpieces for, respectively, Islamist and Baathist tyrannies. Their output should be judged in that light.

UPDATE: A correspondent has forwarded to me a curious message he has received on this subject. I have not included the name of the person who sent it to him, in line with my usual policy. Suffice to say that the message is not a spoof but is intended seriously. I have previously come across the name of the person who sent it; he is, as he says, a musician (he has a site promoting his troupe), and he is a relentless contributor to various far-Left Internet discussion fora, while not obviously - on that evidence - being in possession of a firm grasp of numbers. The message reads as follows:

My name is [...] and I'm a participant in the Iraq Body Count project. I recently saw your rebuttal to Oliver Kamm's piece at the website: http://oliverkamm.typepad.com/blog/2003/09/pilger_truth_an.html. Your rebuttal to this piece was perfectly sound and elementary, which is probably why it has received precious little meaningful response in the thread. You were correct to point out that Oliver's piece was mostly regurgitations of previous claims by writers like Chavetz (claims which have little relevance since the few Iraqi government claims from the early days of the war have mostly been surplanted [sic] by other sources since, as you pointed out), and that really his only new contribution to the debate was implying that since many of us are musicians, we aren't qualified to do this work. Or, even more ludicrous, his implication that many of our members are claiming "authority" on this issue on the basis of our musical training described in the bio page of our members. Or still more ludicrous his "guilt by association" implications, which you alluded to. In any case, I just wanted to write and thank you for taking the time to rebut some of the things written against us and for taking the time to look a bit more closely at our site. I think if more people took a closer look at how our site works and the methods we try to uphold, they'd be less likely to be fooled by pieces like those of Mr. Kamm.

Unfortunately my correspondent's comment on my original post contained none of the remarks attributed to him. It began, "Great post, and badly needed, Oliver. Thanks." It thereafter provided a point of supporting evidence for my criticisms of the Iraq Body Count project. As my correspondent states in some perplexity:

These guys live in an parallel universe.

For the record, I have no problem whatever with musicians' participation in a survey. I am also all in favour of 'holistic critiques' of Mozart operas, especially as my wife (who lately sang the part of Susanna in the Royal Danish Opera production of The Marriage of Figaro) and I are generally familiar only with musical and dramatic ones, doubtless owing to our bourgeois false consciousness. But when a survey contains no one of specialist professional or academic competence in the handling of data, and when it so clearly has been constructed to serve an ideological purpose, it is neither improper nor an ad hominem charge to point this out.

Comments

[David Gillies]

Any simple-minded calculus of human life is morally dubious at best, and repugnant at worst (and I'll be the first to admit I've erred in the past in using it as a justification for 'our' sons-of-bitches killing people in order to stave off a far worse hypothetical calamity should 'their' sons-of-bitches have prevailed). But Herold and his acolytes are, as you say, doing little more than a naive zero-sum calculation of civilian casualties. Why don't we play them at their own game? As has been pointed out in this post and elsewhere, the removal of the Taliban and Saddam has dropped the death rate in Afghanistan and Iraq by huge amounts (the much-touted Chomskyite prediction of millions of dead Afghan refugees has been proven to be as much of a fantasy as the 'brutal Afghan winter', and if there's one thing that's sure, it's that Uday Hussein's industrial shredder has been standing idle since the occupation of Baghdad). Let's run the odometer backwards. How many 'innocent, defenceless civilians' today live and breathe because the regimes under which they toiled have been overthrown? The anti-war crowd's refusal to acknowledge this question goes beyond simple-minded cultural relativism to embrace a truly noxious racism in which any action by the West is morally suspect, and any action by the non-West is excused by reference to the West's much-touted iniquities of the past. If an incinerated New York stockbroker equals a Baghdad greengrocer dismembered by an errant bomb, then that same greengrocer alive and kicking due to the Tikrit thugocracy's fall is an entry in the credit side of our moral balance sheet. If we take the IBC people at their word, then the invasion of Iraq has resulted in a _negative_ number of civilian casualties. If nothing else, this serves to point out the vacuity of their position.

A significant factor in the unhinged nature of the anti-Bush sentiment that infests the fringes is ignorance. I had the opportunity to tear a strip off a young American lady in the pub the other night. She averred that George Bush was every bit as bad as Saddam Hussein. I pointed out (with a few beer-fuelled expletives) that until a mass grave containing the corpses of 1200 children was found outside Des Moines she was, not to put to fine a point on it, talking drivel. She retracted her allegation, but I think she was a) surprised to encounter someone of a differing viewpoint and b) not fully in command of her brief (she had never heard of Halabjah, for example). Herold and his ilk, by virtue of their positions in the academy, have no such excuse.

[Howard Shaw]

If you agree with Jean Bethke Elshtain that "the United States must investigate every incident in which civilians are killed" then is it not a little surprising that we are still only able to "estimate that between 1000 and 2000 Afghan civilians were killed in the US-led campaign"?

[Colin MacLeod]

One other argument that I think most of us understand intuitively, but that we too infrequently make explicit, is that the primary objectives of war in Afghanistan and Iraq, as of any just war, is the prevention of future calamities. Regarding casualties, the question isn't how many died on September 11, 2001, but how many more would be expected to be die in the future both from direct acts of terrorism and, inescapably, amidst the wars of succession that would likely follow a general US retreat from its security role in the Mideast and elsewhere.

If a "body count" comparison has to be made, then the proper one would be between the casualties and other costs of a war fought mainly on our terms versus those, including indirect casualties of misrule and oppression, of a war fought mainly on theirs.

[Chris Brooke]

Since this post presents itself as being inspired by my comment on your earlier post, perhaps I should respond.

My reference to "smears" was intended as a description of part of the argument of your post, not as an "allegation" in any strong sense. Glancing at the OED, I see that the verb "to smear" is defined as "to attempt to discredit (a reputation...)", which seems to me to be a fair description of what you were trying to do in your previous post, and one which you could happily accept but for the word "smear"'s inescapably pejorative connotations.

What I was trying to do in those opening remarks was to suggest that when we removed the ad hominems against the individuals involved in the IBC for their associations, connections and professional background, the actual substantial critique of the IBC's methods which you offered was really quite weak. And since I take the view that it's the quality of their methods which really matters, not their friends, associates, areas of scholarly expertise or interests in rock music, I wrote what I wrote to try to refocus attention on the question of methods. It's clear from this post that you still think these other issues matter quite a bit. On that point, we continue to disagree.

What's the nature of the association of the IBC with Herold? I'm not entirely sure. But one thing seems clear, that the methods being used by the IBC have been drawn up partly in order to meet the more cogent objections directed against Herold's earlier Afghanistan project -- and learning from previous errors is usually considered a good part of scientific practice. Safeguards against double-counting have been introduced through the ways in which incidents are identified in the database, for example, and while you report in this post the criticism that Herold was then making too many poorly-grounded subjective judgments in his interpretation of the media reports, the methodology of the IBC -- including the minimum/maximum tallies -- is designed to eliminate the need for the analysts to make judgments of this kind. As I've said already, the rules aren't ideal. But they aren't worthless, either.

One of the frequent charges against the IBC is that they're trying to produce as large a tally as possible. But if the IBC wanted to make their figures larger, there are lots of things they could have done to do this. In line with the British Government and the BBC (cheap shot, cheap shot), for example, they could have been content to rely on single sources for the information in their database. They could have upped the numbers of civilians they count as belonging to a "family". They could have broadened the range of media sources of evidence they were using for their count. They could have accepted evidence from print sources which don't appear on the internet. They could have included reports of fatalities which aren't directly linked to military action (deaths due to epidemics facilitated by the collapsing infrastructure in parts of Iraq, for example). They aren't doing any of these things, and that seems to me to be worthy of remark, and of approval.

For the record, I also think the project has been making some bad mistakes, which do seem to be the result of a bias induced by its political stance. This page, for example -- http://tom.idealog.info/blog/20030417-1050589639.blog.txt -- documents a specific incident in their database and suggests that the fatalities had nothing to do with coalition military action. It's cases like these which tend to discredit the IBC in my eyes, at least, not because of the counting rules being used or any worries about whether too much official Iraqi information is seeping into the database (as I said last time, if that's the worry, just deduct 130 from the counts you come across), but because their inclusion in the database suggests that the project's own practice isn't living up to its carefully-stated aims. Which is a very great shame, and diminishes a worthwhile project.

[marc w.]

I think a clear indication of the political nature of the IBC effort is their counting of 'resistance' bombings and failed attacks on US troops. While they're being somewhat upfront about these attacks, they're all counted in the same statistic, and almost everyone who quotes the IBC tally thinks they're talking about civilians killed by the US/UK.
They've counted ammunition dump explosions, attacks meant for US troops, a car bomb or two, etc. You're right, Chris, that they could inflate the count if they wanted. But by doing just enough to be called 'rigorous' by their ideological allies, they get a lot more than shock value: they get legitimacy.

[Squander Two]

Chris,

How do you reason that a criticism of someone's professional background is an adhominem attack? If someone who has qualifications and experience in the fields of history and archaeology writes a book about neurophysics, it's perfectly reasonable to criticise him and his book on the grounds that he doesn't know what he's talking about. That's not an ad hominem attack.

Of course, if a bunch of neurophysicists all say that, surprisingly enough, the book's pretty good, then fair enough and well done. In the case of IBC, though, experienced statisticians haven't endorsed the work; they've rubbished it.

[Chris Brooke]

Squander Two:

A question: who are the "experienced statisticians" you've come across who have "rubbished" the IBC? I've read lots of criticism of the IBC on right-wing (sorry, Oliver!) blogs, some of it well-reasoned, but the authors of this stuff are no more experienced statisticians than I am. So if there is intelligent, dispassionate or academic criticism of the IBC project out there by professional statisticians (or demographers), I'd be interested in knowing where to read it.

A reply: Since Oliver went well beyond pointing out that the IBCers weren't statisticians to give considerable detail about the particular academic interests of the IBCers, I took the subtext of his remarks to be something like this: "C'mon, these guys are interested in rock music: why should we take anything they say about Iraq seriously?" That seemed to me to have an ad hom flavour to it. If they've got a well organised project and some intelligent methods (something we continue to disagree about), then maybe we should take it seriously -- even if their only interests are golf, strangling animals and masturbating.

If the historian writes a bad book about neurophysics, after all, the thing to do is to point out the mistakes about neurophysics in the book rather than to observe that the author trained as an historian...

[Squander Two]

>> "who are the "experienced statisticians" you've come across who have "rubbished" the IBC?"

Oliver Kamm, Iain Murray, and me. A quick bit of googling could find you many more, I have no doubt. I don't think I've ever met a statistician who wouldn't laugh derisively at IBC's use of the phrase "most of the time", as highlighted by Oliver.

Just to be clear, I supported the war for political reasons and would still have done so if the real body-count had been ten times what IBC say it is. I would have absolutely no ideological or political problem with IBC's claims being completely correct. If I want to criticise their anti-war stance, I'll do so in the same way I criticise any other enemy of liberty. When I criticise their statistical methods, it's because, as a maths & logic graduate and former professional manipulator of statistics, I can spot deliberate number-fudging a mile off and think it's a particularly disgusting way to fool people, particularly over an issue this important.

>> "If they've got a well organised project and some intelligent methods (something we continue to disagree about), then maybe we should take it seriously -- even if their only interests are golf, strangling animals and masturbating."

I agree absolutely.

[Chris Brooke]

Squander Two:

We agree about more than we disagree.

Re: "Most of the time": That's a good point, but it's one about the Afghanistan project, not the IBC. As far as I can tell the methodology of the IBC was designed to avoid this problem: the only numbers that enter the IBC database are numbers in the reports they use, not any guess (sorry, "judgment") they make from rival and conflicting reports.

[Daniel Calto]

One thing missing from the "body-count" reductionism (regardless of the whopping inaccuracies of the IBC's methodology) is the lack of accounting for the value of liberty for the people who actually survive the war. The number of Iraqi civilian dead may or may not exceed the number of people killed in 9/11. How does one account for the lifting of brutal political repression and the increase of political and civil freedoms for close to 50 million people in Afghanistan and Iraq?

For anyone who has seen the effects of such repression on the spirit of people under the yoke, even indirectly, they are profound, and soul-killing. My wife lived through the Cultural Revolution, and even though she was a little girl and thus spared its worst effects, its pernicious consequences continue to affect her well into her adulthood and maturity, in lack of ability to trust, estrangement from her parents, and lack of hope. I am very happy that her nieces and nephews will face a far milder set of consequences than she had to, even growing up in a country that one would classify as "still unfree".

Cervantes said it best:

"Liberty is one of the most precious gifts heaven has bestowed upon Man. No treasures the earth contains or the sea conceals can be compared to it. For liberty one can rightfully risk one's life."

[Squander Two]

>> "That's a good point, but it's one about the Afghanistan project, not the IBC."

So it is. Oops. Er... carry on. Nothing to see here.

[Phil Rodgers]

To comment on your update, regarding the forwarded correspondence: I think the IBC participant has become confused by your blog layout, rather than by living in a parallel universe, as you suspect. His email clearly refers to the comment beginning "Once we remove the smears by association..." rather than the immediately preceding comment beginning "Great post, and badly needed". However, he has erroneously attributed the "smears" comment to Franco Alemán, rather than Chris Brooke, thinking that the "Posted by" lines refer to the following, rather than the preceding article. This is an easy enough mistake to make when scanning through a long sequence of comments. Perhaps you could make the "Posted by:" line appear before the horizontal rule, rather than after it, to reduce the chance of this happening?