Afstan and public opinion polls

This is the text of a lengthy comment just made on an earlier post--I think it bears posting itself:

Sorry, I know this is ages after the fact, but I just wanted to respond to your comments about the CBC/Environics poll, which has raised concern among people who have worked in Afghanistan. I was on a trip to Afghanistan when it was released, hence this timelag.

If you're still interested, here are some points that I think bear some discussion on the poll's methodology:

Concerns with Validity -

Methodology involves entering people's homes and ask people's opinions on the military, especially the Afghan National Army/Afghan National Police. While the ANA/ANP are not quite like the militia in Iraq yet, they (esp the ANP) are very corrupt and often seen as dangerous to civilians.

Poll was conducted from September 17-24th, right at the beginning of the Holy month of Ramadan, which for many Muslims represents a period of charity and goodwill, and when the good that is done by fasting can be considered void is one speaks ill of others behind
their backs.

Afghans' oral culture and hospitable nature makes the linearity, aggressively
direct, and confinement of responses into five categories of intensity (highly agree, somewhat agree, etc) bewildering. My own direct attempts at conducting quantitative research in Afghanistan are written up here (Kish grid, audience research survey):
http://cms.mit.edu/research/theses/SarahKamal2005.pdf, pages 42-3, 81-3. The problems I've listed in my Master's thesis barely skim the surface of the research challenges I've continued to have while conducting my PhD.

I have spent 7 years working in and around Afghanistan as an academic, development practitioner, and "undercover Afghan." As a Dari-speaking Afghan-looking woman, I have tended to find that after you scratch the surface of Afghan discourse, something else comes out that could never adequately be captured in as blunt and culturally unfamiliar a tool as a western poll. I usually find that people from other cultures tend not to appreciate the underlying resentment or suspicion felt by many Muslims towards the powerful West, and how quickly it can bubble up over a quiet discussion over a cup of tea.

Finding a good facilitator for polling is hard in Afghanistan. ACSOR has done polls for organizations like the Asia Foundation (said to have been founded with CIA funding) and the US state department, and their polls tend to have eyebrow raising results which run counter to other research but are advantageous for suggesting the military operations are running well. The Environics poll is not the first strange public opinion poll coming out of Afghanistan by ACSOR.

Sometimes the timing of the release of such polls is telling. I did a survey of publicly available public opinion in Afghanistan in Dec 2005, it is available here: http://c4o.unitycode.org/me/PeaceConditionalities.final.20060413.pdf . The studies that I looked at are listed in the appendix. Shortly after I finished this study (which found sharp pessimism and a downturn in public opinion), a new quantitative survey was released that claimed that Afghans were very pleased with the reconstruction process and international presence, released right before a major donor conference. This was in the same year that friends of mine were chased out of a UN compound in Jalalabad by angry mobs, who set fire to the compound. Also the same year as the Koran riots and Afghan Minister of Planning Bashardoost winning major public support in demanding that NGOs leave the country.

Methodology doesn't state how questions were piloted. Were there ways of triangulating responses? For instance, if people are so positive about the future, why is it that in the Environics poll only 40% think the government and foreigners will prevail in the current conflict? (20% believe the Taliban will win, 40% don't know). 20% believe Al Qaida is a positive force in the country - how does that mesh with other responses?

Concerns with generalizability -

Poor to non-existent communications and road infrastructure in rural areas, inadequate mapping, lack of security, illiteracy, widely divergent population estimates and shifting displaced populations hamper statistical generalizability of their poll of about 1,500 Afghans.

--

I have been in Afghanistan many times in the last 6 years, and in my three visits this year I found the security situation to be the worst I have ever seen. I first entered Afghanistan during the time of the Taliban, and even then did not feel as threatened as I did in my most recent journey in October 2007. There is no sense of safety anywhere, and even longtime Afghan friends of mine now feel uncomfortable entering downtown Kabul. Such fears could only have worsened with the Nov 6th suicide bomb killing children and MPs in Baghlan, formerly considered a "safe" area.

I have been wrong more times than I can count when it comes to Afghanistan, which I find a fascinating and unendingly complicated space. I don't object to surprising research findings, but I do object to bad science that run counter to common sense. The Environics poll runs counter to what I and other longtime development workers have found to be the mood in the country (including a practitioner who has lived for 6 years in Kandahar). The poll is also dangerous, in my opinion, because the word for expressing the public's mood that is more and more being bandied about in expert circles, and among Afghans, is "occupation." I was a panelist at the Middle East Studies Association annual conference this weekend, and everybody there agreed with that framing. So I believe it is particularly important to not allow a poll (which, as we understand, even in the best of situations is just a poll and not reflective of anything other than what people choose to say to a pollster) to be taken as more than it is.

Sorry for the long post. Best regards,

Sarah Kamal
2007 Trudeau Scholar
PhD Candidate, London School of Economics

Babbler's Addendum: A different perspective on whether Afghans want us there, from another lady who's spent some time in country:

Q: Are people glad to see us there, and do they really appreciate what we are doing there to help them?

W.O. Lori Coady: I asked myself that same question everyday. And everyday I had a different answer. I was welcomed into places with open arms and received a cold handshake at other places. However, even with a cold welcome the aid that Canada was offering with me as a vehicle was always accepted.

The individuals I would ask that same question to would be the Language Assistants (translators) that worked in our camp. Their response was that our help is welcomed as long it is what Afghans want. We are very careful to ensure that what help we give is that which the villages request and distributed on behalf of the Government of Afghanistan.

Yes, people do appreciate that we are there. They recognize that it is a tough battle and they know that the Canadian public is worried about the soldiers and that there is a chance that we'll leave. They tell us all the time that they hope that does not happen. People are still afraid.