The Ideological Turing Test is a concept invented by Bryan Caplan to test whether a political or ideological partisan correctly understands the arguments of his or her intellectual adversaries. The partisan is invited to answer questions or write an essay posing as his opposite number; if neutral judges cannot tell the difference between the partisan's answers and the answers of the opposite number, the candidate is judged to correctly understand the opposing side.
Contents
- History
- Attempts at the Ideological Turing Test
- Attempts at the test where the identity and ideological affiliation of the author are known in advance
- Turing test contests religious and others
- Historical precedents
- References
The Ideological Turing Test is so named as to evoke the Turing test, a test whereby a machine is required to fool a neutral judge into thinking that it is human.
History
The idea was first mooted by Caplan in 2011 in response to Paul Krugman's claim that, in the context of US politics, liberals understand conservatives (and libertarians) better than conservatives (and libertarians) understand liberals. Borrowing the idea of the Turing test used to judge whether machines can pass themselves off as human, Caplan suggested the Ideological Turing Test as a way to impartially test Krugman's claim: whichever side understands the other better would perform better on an Ideological Turing Test. He also offered to take the test himself and offered to bet that libertarians could more easily pass themselves off as liberal than liberals could pass themselves off as libertarian.
Caplan's post was praised by Ilya Somin at the Volokh Conspiracy blog.
Attempts at the Ideological Turing Test
Three versions of the Ideological Turing Test have been attempted:
- The identity and ideological affiliation of the author are explicitly known. This may happen if an author who is well known for espousing certain ideological positions writes a piece play-acting as somebody with the opposite ideological position, with a clear indication that this is intended to be an Ideological Turing Test. A disadvantage of this approach is that readers already know the answer, and hence might be affected by hindsight bias.
- The identity and ideological affiliation of the author are not known, but it is made clear that this may be an Ideological Turing Test. Typically, this is done as part of a "contest" where a number of entries are solicited to argue for an ideological position, some from genuine proponents of the position, and some from opponents. People are asked to distinguish genuine proponents from opponents. The proportion of proponents versus opponents may or may not be known in advance. A criticism of this approach was that it made readers overly skeptical compared to what they should be under ordinary circumstances.
- The identity and ideological affiliation of the author are not known, and it is not suggested that the piece may be an Ideological Turing Test. Rather, every attempt is made to deceive readers into believing that they are dealing with a genuine proponent of the position.
Attempts at the test where the identity and ideological affiliation of the author are known in advance
Caplan's blog post inspired a number of tries at the Ideological Turing Test by prominent bloggers including those listed below:
Turing test contests: religious and others
Leah Libresco (who was an atheist at the time, but later converted to Catholicism) set up a contest based on the Ideological Turing Test idea involving atheists and Christians. Interested contestants were given a set of four questions as prompts and asked to write responses to the questions, both as atheists and as Christians. Fifteen people participated in the contest, about half of them Christian and half atheist. In the first round, all the fifteen responses written as atheists were published. In the second round, all the responses written as Christians were published, albeit not with the same numbering. Readers were asked to fill in a survey specifying their own religious identification and giving their view, for each response whether it was by a Christian or an atheist. Later, the identities and affiliations of all the participants, as well as how they did in the survey responses, was revealed in a series of blog posts. The test was repeated in 2012 and a resource page recording all iterations of the test was created on the Patheos website. In 2015, Libresco ran another Christian/Atheist Ideological Turing Test.
Historical precedents
Dawes, Singer and Lemon (1972) conducted an experiment along the lines of the Ideological Turing Test between "hawks" versus "doves" on the Vietnam War in the United States. They found that each side rejected more statements made by the other side during the experiment than by their own side, on the grounds that the statements were too extreme. However, the degree of exaggeration was not symmetric. Doves significantly exaggerated the extremity of the typical hawk, but hawks showed only a weak tendency to exaggerate the extremity of the typical dove.