The surprisingly popular answer is a wisdom of the crowd technique that taps into the expert minority opinion within a crowd.[1] For a given question, a group is asked both "What do you think the right answer is?" and "What do you think the popular answer will be?" The answer that maximizes the average difference between the "right" answer and the "popular" answer is the "surprisingly popular" answer.[2] The term "surprisingly popular" was coined in a 2017 paper published in Nature entitled "A solution to the single-question crowd wisdom problem", which outlined the technique.[2][3]
Example
editSuppose the question to be determined is: Is Philadelphia the capital of Pennsylvania? The two questions asked of the group, and the numbers of responses, are:
- Is Philadelphia the capital of Pennsylvania?
- Yes: 65%
- No: 35%
- What do you think most people will respond to that question?
- Yes: 75%
- No: 25%
The difference between the answers to the right question and the popular question:
- Yes: 65% − 75% = −10%
- No: 35% − 25% = 10%
Thus, the No answer is surprisingly popular (10% > −10%). Because of the relatively high margin of 10%, there can be high confidence that the correct answer is No. (The capital is indeed not Philadelphia, but Harrisburg.)
An illustrative breakdown of this follows. There are four groups of people.
- A – "Philadelphia is the capital, and others will agree." (This group answers yes/yes.)
- B – "Philadelphia is the capital, but most others won't know that". (This group answers yes/no.)
- C – "Philadelphia is not the capital, and others will agree." (This group answers no/no.)
- D – "Philadelphia is not the capital, but most others won't know that." (This group answers no/yes.)
This technique causes groups A and C to be eliminated from consideration and measures the difference in size between groups B and D.
Both groups B and D think they know something others do not, but B is wrong and D is right. In cases where people feel like they have "inside" knowledge, it is more often the case that it is because they are correct and knowledgeable (group D), not because they are misled (group B).[3]
Surprisingly Popular Algorithm for ground-truth ranking recovery
editFor m>2 candidates, the Surprisingly Popular Algorithm requires votes from an infinite number of voters on all possible ranked permutations (m!) of the alternatives to recover the ground-truth ranking with complete certainty, as discussed in the Nature article. Hosseini et al. (2021) extended the previous work to recover rankings using various elicitation formats.[4]
See also
edit- Keynesian beauty contest
- Guess 2/3 of the average
- Family Feud
- Focal point (game theory), also known as Schelling point
References
edit- ^ Akst, Daniel (February 16, 2017). "The Wisdom of Even Wiser Crowds". The Wall Street Journal. Retrieved 16 May 2018.
- ^ a b Dizikes, Peter (January 25, 2017). "Better wisdom from crowds". MIT News. Retrieved 16 May 2018.
- ^ a b Prelec, Dražen; Seung, H. Sebastian; McCoy, John (2017). "A solution to the single-question crowd wisdom problem". Nature. 541 (7638): 532–535. Bibcode:2017Natur.541..532P. doi:10.1038/nature21054. ISSN 1476-4687. PMID 28128245. S2CID 4452604.
- ^ Hosseini, Hadi; Mandal, Debmalya; Shah, Nisarg; Shi, Kevin (2021). "Surprisingly Popular Voting Recovers Rankings, Surprisingly!". Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. pp. 245–251. arXiv:2105.09386. doi:10.24963/ijcai.2021/35. ISBN 978-0-9992411-9-6.
Further reading
edit- Prelec, Dražen; Seung, H. Sebastian; McCoy, John (25 January 2017). "A solution to the single-question crowd wisdom problem". Nature. 541 (7638): 532–535. Bibcode:2017Natur.541..532P. doi:10.1038/nature21054. PMID 28128245. S2CID 4452604.