Do Truth-Telling Oaths Improve Honesty in Crowd-Working?
Download 14.68 Kb.
|
admin, Echols
|
Intro In the article “Do Truth-Telling Oaths Improve Honesty in Crowd-Working?”, found in PLoS ONE, Jan 15, 2021, volume 16, authors Nicolas Jacquemet, Alexander G. James, Stéphane Luchini, James J. Murphy, and Jason F. Shogren investigate dishonesty in online labor markets, and test the efficacy of truth-telling oaths in reducing dishonest behavior in participants of online surveys. Using a basic coin-flip test, Nicolas Jacquemet et all prove that certain dishonest behavior has become commonplace in online, non-centralized workplaces such as Amazon MTurk: a crowdsourcing platform that helps aggregate HITs or Human Intelligence Tasks such as audio transcription and data collection. While it was found that a voluntary pledge not to lie had a marked effect on displayed dishonesty in the coin flip test, it did not affect the tendency to “shirk”, or to make up a number without flipping the coin. Methodology The experiment took the form of an anonymous job listing posted to MTurk under the survey category. Participants who responded to the survey were instructed to flip a coin and report how many heads they flipped. For every head, they would receive an extra ten cents added to the one-dollar survey payout. The average hourly wage for MTurk workers (at the time the experiment took place) was two dollars an hour, making the extra ten cents per head a substantial incentive for dishonesty. Participants were also required to fill out a brief demographic questionnaire and consistency question (a “Are you a robot” test). Due to the nature of the survey, dishonesty could not be measured (conclusively) on an individual level and could only be measured against the average normal distribution of 5 with a variance of ¼. Dishonesty was broken into two categories: lying and shirking. Lying was defined as the intentional misreporting of heads flipped, and shirking was defined as ignoring the task entirely and making up a number. Shirking was determined by comparing the time it took for each participant to flip the coin to an “average” flip time. This average was the product of another survey, this time done in person to ensure accuracy. The possibility of MTurks using a digital coin flipper was also accounted for. Participants in the survey were broken into two categories: oath (experimental group) and non-oath (control group). Before being prompted to flip the coin, oath participants were first presented with the following: “Before we begin, do you wish to swear upon your honor to answer the following questions truthfully? (You will be allowed to continue with this survey regardless of your answer to this question)”. Those who wished to take the oath pressed “yes”, while those who refused pressed “no”. Only two MTurk workers opted not to take the oath. The non-oath participants or “control group” were not presented with this option and proceeded directly to the coin-flipping task. After the flip task, the non-oath group was asked whether they answered truthfully regarding the number of heads they reported, while the oath group was asked whether their choice to swear upon their honor affected the number of heads they reported flipping. 1,366 MTurks were surveyed in total: 681 were given the choice to take the oath, 685 were not. Results The results of the experiment not only support the hypothesis that MTurk workers lie, but also provide insight into how prevalent this behavior is on a mass scale. When compared to the truthful distribution, the mean of reported heads averaged at 6.33: a statistical impossibility if each participant answered truthfully. Even when disregarding obvious felonious responses (MTurk workers who responded too quickly, reported 10 heads, etc.), the mean still varied substantially from the truthful mean (5.5 compared to 5). Nicolas Jacquemet et all also found that 42.6% of MTurk workers completed the task in under 30 seconds: a statistical impossibility. Comparing the oath and non-oath groups proves that the oath is marginally effective at reducing “big” lies. Those who opted to take the oath flipped an average of 6.06, 4.2% closer to the truthful average. The oath also reduced reports of 10 heads by 27%: 88 from those who took the oath compared to 122 from those who did not. The oath had less of an effect on shirking: the time it took the oath and non-oath participants to complete the coin-flipping task were identical. Download 14.68 Kb. Share with your friends:
|