IT can be psychologically rewarding to write tests evaluating other people’s competence on topics that they should understand (for example, climate, nuclear power, diet, finance). Test writers get to be experts, define the knowledge that matters most, and pass judgment on others. They also get to weigh in on public affairs, as arbiters of citizens’ competence to participate in issues of the day.
It can also be socially destructive to write such tests, if the resulting assessment is off the mark. Citizens whose knowledge is underestimated can be denied legitimate rights. Citizens whose knowledge is overestimated can be denied legitimate protections. Tests can even be written with such errors in mind. An incompetent public strengthens the case for rule by experts and for strong regulatory rules. A competent public strengthens the opposite case. As every teacher knows, it is not hard to write tests that a class passes or fails.
Competence testing can also promote a deficit model of public engagement, positioning scientists as ultimate authorities, with citizens forever scrambling to catch up – and learn the things that experts think that they should know. Deficit thinking can easily lead to confusing ignorance with stupidity, casting those who don’t know as being incapable of learning.
Some of these failings will be familiar to those readers of People & Science who have found themselves grouped with the unwashed on topics outside their own expertise. One such pathology is placing the onus on citizens for not knowing facts that would be easily understood, had they just heard them. For example, late in 2002, we found that most Americans did not know that anthrax was not contagious, that smallpox could be treated before it became symptomatic, or that panic was rare in disasters – three simple, useful facts that would have been happily learned, had they been communicated effectively.
A second pathology is accusing people of hypocrisy, when they fail to adopt every specific behavior that is conceivably consistent with a general attitude. For example, most people have long seen climate change as a real threat. They should not be dismissed as ‘not getting it’, when they still fly long distances (if they have far-flung families) or leave their apartments incompletely insulated (if they are renting).
A third pathology is accepting the accuracy and relevance of expert opinion without question or qualification. For example, before the ongoing debacle, citizens were often criticised for not knowing that stocks produce the best return on investment, over any reasonable time period. Now, citizens are criticised for not knowing the differences between registered and unregistered investment firms or between monthly statements issued by firms and by banks holding securities. Some financial competence can increase exposure to financial chicanery.
Although experts can claim unique ability to know what is really worth knowing, citizens are not completely defenceless. They can put the tests to the following test:
Is each question relevant to a decision that they are facing? If not, then they are being evaluated for their mastery of minutiae. Are they sure of the meaning of each word in each question? If not, then they are being set up to fail. Are facts that they want to know missing from the test? If not, then their needs are being ignored.
Developing relevant tests is straightforward.1 However, it requires listening carefully to citizens, in order to understand the decisions that they face, the facts that they need, and the clarity of questions on successive test drafts. Without such a disciplined approach, competence testers can do bad, while feeling good.
1 B Fischhoff (in press). Risk perception and communication. In R Detels, R Beaglehole, M A Lansang, and M Gulliford (eds), Oxford Textbook of Public Health, Fifth Edition (pp. 940-952). Oxford: Oxford University Press