Forum

Interesting data. Questions about obtaining it are:

Where were the wallets dropped?

How often was this "test" done?

First and foremost, if I recall my statistics courses right, no panel with less than 500 individuals can be considered representative. That is the main problem in surveys like this, and it leads to other problems - many people who do these experiments use methods like one would with a large (representative) panel, namely count and compute, and then make statements out of it like 'old people are four times more honest than young people'.
But just that is what you are not allowed to do in a serious experiment with a non-representative sample. Either you have a representative sample and treat it as such, or you don't and then use different methods. What was done here was 'mixing pears and apples' as we say here.

But anyway...you see a lot of 'tests' like this on TV and in newspapers. None of them tell us about the scientific background, and they mostly aim to sell the 'results' as a proven fact. This one - I think - is particularly bad, because it fortifies prejudices...and there are so unbelievably many people out there who are willing to believe anything the media tell them...and even more so if they feel supported in their world view.

But well, that's the way it is...

hugs

...Naysayers have re-hashed the same arguments so many times that I think I can safely list just about every variation of some one crying "foul!"...

Many have said that the results of the wallet test are invalid because...

1) ...the number of black people tested is not the same as the number of white people tested.

It is true that more white people were tested then black people, but that was not by design. The wallets were dropped at random in public places. The 100 people tested were, therefore, randomly selected from the population as a whole. Even numbers, however, are not necessarily needed to compare different groups to each other - that is what percentages are for.

2) ...economic considerations were not included in the test.

True. But the nature of the test is such that affluence of individuals is not something that can be measured. You can look at the films of the people picking up the wallets and can SEE their race, gender and approximate age - but you can't really SEE how affluent they are. Although a lot of people have pointed out this short coming - none have yet been able to come up with a practical method of adding affluence measurment to the test.

3) ...black people have less money then white people.

Many have suggested that the reason that fewer black people returned the wallets was because... "people with less money steal more then people with more money - black people have less money - so that is why black people were more likely to keep the wallets." That is a valid theory, but that does not mean the results of the wallet test was any less valid - it is just a good explanation for the racial discrepancy.

4) ...the person conducting the test is white, therefore black people might be slower to return a wallet to him.

Most of the time when people found wallets - they had no idea what the race of the person that had dropped it. Of the 74 people who found and returned wallets - only 4 were people who saw me drop them (3 white, 1 black).

5) ...the motivation for the test was racist.

Not true. My motivation was curiosity and a desire to create an informative website based on my findings. I was determined to post my results truthfully and accurately and I have done so to the best of my ability.

6) ...race is a factor that should never have been included in a test like this because acknologing racial differences, in any context, is wrong.

That is a debatable point and one that I do not agree with. There is a big difference between acknologing differences between people and bigotry.

7) ...the number of people tested was not large enough.

Large enough for whom? I spent a full month conducting the wallet test experiment. Each of the 100 people tested took an hour of driving time, observation time, movie editing time, etc... I also spent a lot of time and money on the wallets themselves and on the website. As far as I know no one else has conducted a simular test with this much detail and with this large of a group of people before. While I'll admit that a larger number of people would have been better - I still feel that 100 was a rather "significant" (if not "ideal") number of people to test. Many harsh critics have said I should have tested 1,000 people! But if I had, I'd be willing to bet that most of those same critics would have found something else to complain about or would have insisted that I should have done 10,000! LOL!

...the tester was not qualified to conduct the test.

You don't need to be an expert movie maker to film people stealing wallets. You don't need to be a criminologist to figure out that wallets that were never returned were probably stolen. You don't need to be an anthropologist to tell if a person is a man or woman, young or old, black or white. You don't need to be a statistician to figure out that 43% is over twice as likely as 21%... This in not rocket science or brain surgery - this is careful observation, good record keeping and high-school-level math.

9) ...the results were falsified.

The results are the product of a great deal of time and hard work. I can back up my findings with detailed notes and hours of recorded video and audio.

10) ...the results are not politically correct.

This is perhaps the most honest reason for people's dislike of the wallet test. It is difficult to accept results that are counter to what you believe are true or what you WANT to believe is true. And I know exactly how it feels. When I started the wallet test experiment - I did not expect any significant difference between the men and women tested. I was, of course, aware that more crimes were committed my men. But I had always chalked it up to "aggressiveness" and "guts" not "honesty". My reasoning was that women WANTED to rob liquor stores just as much as men - they were just too scared to do it. I now see that was wishful thinking on my part. After the test I was shocked by the gender differences; men were over twice as likely to steal wallets as men?!? To be honest - I was a little angry about it. But I could not deny it. I was faced with the ugly truth of my own test. I understand from where anger toward the "wallet test" comes from because I have felt it myself.

11) ...the results could be used by racist to justify their beliefs.

That is true and that is unfortunate. But I feel that suppressing the truth would be even more unfortunate. Covering up unpleasant test results will not help further the worthy goal of racial equality. A better strategy would be to use this research as a starting point for discussing the extent and causes of racial inequality and to explore possible solutions. Ignorance and denial will aid bigotry far better then logical examination and critical thinking.

How would you guys have done it?
What, specifically, should the researcher/webmaster have done differently?