The problem with surveys (and those who report them)

I shall scrupulously report and interpret the news, taking care not to suppress essential facts or to distort the truth by omission or improper emphasis.

Journalist’s Code of Ethics, No. 1


by Dexter MC

Short lesson on reporting surveys

There are clear-cut rules on how to do news reports that will use survey data as its primary source of information. Any journalist would know that the data must be interpreted correctly, and that this interpretation must properly be delivered to the audience without any distortion.

Also, journalists must keep in mind that not all surveys can be used to make news reports. Obviously, not all surveys are credible, and the media does not have the time and space to report all the surveys being conducted everyday.

Credibility of a survey is first judged by the credibility of its conductor, which should not be perceived as biased. The conductor of the survey should not gain anything from the result of the survey, because even a small interest would taint the result.

Once ascertained that the survey is credible enough, that is, it was conducted by a neutral organization, the journalist can then start making a news report. But even then, the journalist should still be careful because, as previously pointed out, there are clear-cut rules on how to do this.

Elements that should not be missing in a news report based on a survey are:

  1. The dates when the survey was conducted, and the places where it was conducted. – These are important because it would give the audience an idea on how big the survey was.
  2. The number of the people surveyed, and how they were chosen – Was the sample divided by age, race, sex, and other categories?
  3. A short summary on how the survey was conducted – Were the people surveyed given a questionnaire? Were they interviewed face-to-face? Were they interviewed by phone?
  4. Contributing factors that may have affected the survey – For instance, a survey of the ranking of presidential candidates may be affected by news of one candidate’s alleged corruption that is currently being reported in the media.

There are other elements, but those listed above are the most important. Without them, the report comes out without any semblance of credibility.

ABS-CBN’s vague, old news

There’s a problem, therefore, with Timi Nubla’s “exclusive” report on ABS-CBN’s Bandila, aired on October 7. The whole report, titled “More Filipinos use contraceptives,” can be seen here:

According to the report, a women’s health NGO named Likhaan Center conducted the “new” survey in 2008. Survey results said 5 out of 10 Filipino women already use contraceptives. The number is increasing slowly because non-contraceptive users have apprehensions, including the fear of the contraceptives’ side effects, confidence that the “natural method” will work, and that contraceptives are expensive.

Next, Nubla quoted Likhaan Center: “Maliit na porsyento na lamang ng mga Pilipino ang apektado sa opinyon ng Simbahang Katolika sa paggamit ng contraceptives. Pero malaki pa rin ang impluwensiya nito sa mga pulitiko, kung kaya’t hindi ito maisulong o maipatupad.

There are many faults in this report. One, it makes great importance to the data from Likhaan Center, who did the survey obviously for its own gains. Likhaan Center, as the report said, is a “women’s health NGO.” Therefore, any of its surveys should be treated with great doubt.

The lack of survey elements in the report greatly increase this doubt. How was the survey conducted, and who were the subjects? When and where was it conducted? Were the subjects taken from the slums or from affluent families, and were they all women? Of what age? How many were interviewed? What is the survey method used?

The way Bandila handled the report was also malicious. Nubla said the survey is new or “bago,” when it was taken in 2008. The survey data said there were “5 out of 10 Filipino women” who use contraceptives, posing a great disparity to the report’s title “More Filipinos use contraceptives.” One, “Filipino women” cannot be equated to Filipinos in general. Two, it is improper to say there are “more” users of contraceptives than non-users when the data arrived at only half—5 out of 10 is not more, it means there are also 5 out of 10 who do not use contraceptives.

And since the survey method was not explained well, Nubla’s quote from Likhaan Center—“Maliit na porsyento na lamang ng mga Pilipino ang apektado sa opinyon ng Simbahang Katolika sa paggamit ng contraceptives. Pero malaki pa rin ang impluwensiya nito sa mga pulitiko, kung kaya’t hindi ito maisulong o maipatupad.”—smacks of incredulity. Was this conclusion taken from the survey, too? How?

All in all, the report from Bandila should not be taken seriously, because it was based on a survey that is not credible to begin with. In the pursuit of a good scoop or exclusive, journalists should not forget that their main role is to educate the public, not to misinform and malign them.

Dexter MC writes for a living. He is fascinated by social media, web trends, law, literature, and politics.