Proposed agenda and updates on the Mturk issue

8/15/2018

Here is a proposed agenda for the current Mturk issue. The document also includes some updates about how many researchers have seen duplicates in their data so far. I invite comments and collaboration http://z.umn.edu/MturkIssue

4 Comments

Jason

9/17/2018 07:32:05 pm

Crowd-sourced work force platforms have always interested me. As I read about the alarm that you have raised for the potential about bad data on account of bots, I can't help but to wonder about what sort of financial impact that your posts have had on the income of workers on Amazon Mechanical Turk as some researchers opt to not post surveys on Amazon Mechanical Turk on account of the doubts that you have raised, but not sufficiently proven. Reading through the worker forums, many are commenting on the recent drop in work available following your posts. Several of the more seasoned workers are beginning to mention "lawsuit".

Have no doubt, I am certainly worried about surveys being flooded with bad data from bots of malicious workers.

However, in considering the fact that you raised the alarm before conducting a truly systematic study/survey of the data has been completed, and considering that many legitimate workers meticulously track statistics about work available and their own earnings from Amazon Mechanical Turk, are you potentially at risk for any sort of legal action on the behalf of these workers who can reasonably demonstrate a loss of wages on account of your public blog posts?

If these surveys were unpaid and completed strictly by volunteers, then I wouldn't be concerned. However, since your postings could potentially have an impact on the wages of thousands of workers shouldn't that require you to be more thorough and cautious about the validity of your claims before they are made public?

Helen

9/20/2018 11:13:12 am

THANK YOU! YES! [APPLAUSE]

Sally

9/20/2018 11:24:39 am

I too think that before you published or stated your opinion that you should of had some kind of proof, not just what you think. There are many honest hard working people that put in their time on Mturk to assist requesters with their surveys and most of it for underpaid wages. I have also looked at the forums and see the struggle they are having and the reports of less well paying surveys and other work since your article. For many of the workers this is their main source of income. I encourage any requesters, students, etc. that before making a decision not to use mturk that they themselves go to a couple of the different forums and get to know the people that work on Mturk and not to just take Jason's unproven opinion on bots.

ultius link

12/1/2019 01:18:54 am

Before conducting a certain survey, you must know how to deal with the right process and make sure that you are doing it the right way. Committing mistake should never be an option because you are afters consistent and systematic result. You will not achieve it if you are going to give mistakes the chance to be committed. Accuracy is what we are talking about here and we should know that it's really important! Making surveys is challenging, so make sure you are prepared before you do it!

Proposed agenda and updates on the Mturk issue

Leave a Reply.

Author

Archives

Categories