Here is a proposed agenda for the current Mturk issue. The document also includes some updates about how many researchers have seen duplicates in their data so far. I invite comments and collaboration http://z.umn.edu/MturkIssue
Crowd-sourced work force platforms have always interested me. As I read about the alarm that you have raised for the potential about bad data on account of bots, I can't help but to wonder about what sort of financial impact that your posts have had on the income of workers on Amazon Mechanical Turk as some researchers opt to not post surveys on Amazon Mechanical Turk on account of the doubts that you have raised, but not sufficiently proven. Reading through the worker forums, many are commenting on the recent drop in work available following your posts. Several of the more seasoned workers are beginning to mention "lawsuit".
Have no doubt, I am certainly worried about surveys being flooded with bad data from bots of malicious workers.
However, in considering the fact that you raised the alarm before conducting a truly systematic study/survey of the data has been completed, and considering that many legitimate workers meticulously track statistics about work available and their own earnings from Amazon Mechanical Turk, are you potentially at risk for any sort of legal action on the behalf of these workers who can reasonably demonstrate a loss of wages on account of your public blog posts?
If these surveys were unpaid and completed strictly by volunteers, then I wouldn't be concerned. However, since your postings could potentially have an impact on the wages of thousands of workers shouldn't that require you to be more thorough and cautious about the validity of your claims before they are made public?
THANK YOU! YES! [APPLAUSE]
I too think that before you published or stated your opinion that you should of had some kind of proof. There are many honest hard working people that put in their time on Mturk to assist requesters with their surveys and most of it for underpaid wages. I have also looked at the forums and see the struggle they are having and the reports of less well paying surveys and other work since your article. For many of the workers this is their main source of income. I encourage any requesters, students, etc. that before making a decision not to use mturk that they themselves go to a couple of the different forums and get to know the people that work on Mturk and not to just take Jason's unproven opinion on bots.