Did you read this part-
One weakness of the study is that the control question, which should show 0% of people rejecting the statement “I do not believe that 2 + 2 is less than 13”, actually gave an estimate of 34%. The authors note this, showing that they are careful about the data:
Says a lot about their methodology.
Says more about people’s math word problem abilities
And problems with parsing chains of negation, I think. (They have to reject the statement “I do not believe that 2+2=13”).
Or the question was framed in a way that confused whoever they were surveying… i.e a problem with the methodology.
In fact, this is one reason direct answers/self reporting gives more reliable results.
A direct question wouldn’t have gotten 34% false positives.
I agree. If the point is to test the people’s arithmetical skill, that can be done by asking them to comment on a positive statement, not on a nearly double negative (“I do not believe that 2 + 2 is less than 13”). Why not ask them to comment on “I believe that 2 + 2 is less than 13”? Or “I believe that 2 + 2 is more than 13”? If the point is to find out what people believe rather than how good they are at answering tricky questions, a more straightforward question is desirable. But you can’t tell these survey “experts” anything. They are convinced they know all about how to do a proper survey.