Waxy.org is the sandbox of When you experiment with Amazon's Mechanical Turk, it feels like magic. You toss 500 questions into the ether, and the answers instantly start rolling in from anonymous workers around the world. It was great for getting work done, but who are these people? I've seen the demographics, but that was too abstract for me.
Last week, I started a new Turk experiment to answer two questions: what do these people look like, and how much does it cost for someone to reveal their face?
Answer #1. This is what Mechanical Turk looks like (click for full-size):
Answer #2. About $0.50.
Results
Here's my original request:
Upload a photo of yourself holding a handwritten sign that says "I Turk for ...", filling out why you turk. For example, "I Turk for Cash," "I Turk for My Kids," "I Turk to Kill Time," or whatever else you like. Be honest, be funny, be whatever you like.As a good faith gesture, here's my photo.
If you have a webcam, you can simply go to Cameroid to snap a photo from your web browser, download the JPG, and upload it below. (Don't worry if the text is backwards, I can fix that myself.) DON'T provide any identifiable information, like your name or email, since that's a violation of MTurk policy.
The result will be used in a collage that can be found on my personal weblog, http://waxy.org. By uploading your image and accepting payment for the image, you give permission to me, Andy Baio, to use your image in all forms and media for any lawful purposes. (That's just cover-my-ass language. I'm almost certainly only going to restrict it to this one project.) The collage will show up there shortly after the HIT is complete. Thanks, everybody!
I started the task at $.05, but only two people responded in the first 24 hours. (And one of those was Joshua Schachter, who I'd told about the project.) Clearly, that was too low, so I increased it to $.25, receiving only eight submissions in 48 hours. (For reference, all 500 of my Girl Talk tasks were done in about an hour.) Increasing it to $.50 got me 20 more submissions in about 48 hours, after which it started to drop off quickly. I wasn't about to give dollar bills to random people for their photos, so I ended the experiment there. People aren't willing to give up their anonymity for cheap.
The final results: 30 people total — 10 women, 20 men. Almost all were white, mostly in their 20s and 30s. 21 said they turked for money, 9 for fun or boredom.
Thanks for pulling back the curtain, Turkers.
2:23 PM
Love that more than one person said "for the lulz"
2:38 PM
I'd love to see that broken down by fee. Who was willing to do it for $0.05? for $0.25?
2:42 PM
Did you have to reject many for not following the right procedure? I could see people faking it with pictures of other people's faces.
2:58 PM
Kevin: On the big image, you can see a white border separating the three groups... Two people at the top for $.05, the next eight for $.25, and the rest for $.50.
Dave: That's why I required a handwritten sign, which is harder to fake (but far from impossible). There were only two people that I rejected. One accidentally sent a digital photo of someone's back (I suspect this was an accident) and the other photoshopped his sign in with type.
3:24 PM
You should try the same experiment with ChaCha (chacha.com) Guides.
3:36 PM
This is pretty funny :) Turk is interesting, to say the least.
3:47 PM
Seems like it would actually be more work to photoshop a sign into another photo than just to take one of your own (and I'm pretty quick with photoshop). In a way, I continue to be stunned that people will do these tasks for so little money. I'd have to be getting at least $5-10 to bother with something like this (not that it's a lot of work, it's just the annoyance factor). In this particular case, you may have gotten some people who participated at a lower price than they otherwise would have because they liked the idea of it. But even with mundane tasks, I'm amazed by the low prices.
3:52 PM
Great post. As a researcher I've been wanting to use Mechanical Turk for some of my machine learning tasks. This clears away the fog about who are the real turkers and reduces my anxiety about using it.
Your post is a great validation for this wonderful tool that matches research needs with human needs.
3:58 PM
Sean: I think a big part of the appeal for turkers motivated by money is the flexibility. These people can work for five minutes at a time in between classes, during commercials, or during their lunch breaks. It's simple, mindless work that can be done in spare time for spare change.
4:27 PM
Wow, nicely done. This is great.
5:17 PM
Great!
This is already hanging outside my office door, and the photos will be definitely in a slide of my presentation when I talk about Mechanical Turk!
7:02 PM
@Panos: of course the people still retain the copyright in the images they submitted. Clearly they've consented for the authors use of those images but you'd have to (legally) contact them to request redistribution and/or reproduction rights.
You may also need model releases including (it appears in at least one case) parental consent.
FWIW.
A lawyer who wanted the fight could also defend an infringement suit for the rights of the 3 people shown in 2 collages, the agreement says "used in a collage that can be found on my personal weblog" and that 'sample' is not a subsection of the collage.
8:25 PM
pbhj: I used standard language for a model release form. The agreement stated clearly, "By uploading your image and accepting payment for the image, you give permission to me, Andy Baio, to use your image in all forms and media for any lawful purposes."
Since I have a model release, I'm placing them under the same Creative Commons license as the rest of my original content.
As a result, it's not necessary for Panos to try to obtain his own model releases... which would be completely impossible anyway, since the Mechanical Turk submitters are all anonymous. As long as his use is non-commercial, which I suspect it is, he's golden.
Actually, even if the images were copyrighted, Panos could claim fair use for using the image for educational purposes at his school.
8:25 PM
One thing -- the fact that $0.50 was required to get anyone to do it, maybe not necessarily be just the price of revealing their face. I bet a big part of that 50 cents is the hassle of making a sign and figuring out how to take the picture.
Or maybe making a sign is fun -- a task to just take a picture might cost more?
11:42 PM
This is really fantastic! Putting a face to a "turk" is quite reassuring in some ways.
I've had such great success with the Turk community though, that I've wondered less about who they are. I also have 300 replies from Turkers stating their professions, which I really should post on my blog to add another dimension to these mysterious people. For now here's a small sampling of their occupations (much more than simply students and part-time workers): spanish translator; United Nations worker; webmaster of dogtoys.com; researcher at Duke Medical Center; paralegal at Murray Law Office; manager of a Bait & Tackle shop in Florida...
5:24 AM
I've never used Turk, but always wondered about creative things to do with it, this far outdoes what I was thinking. Very interesting.
9:33 AM
As a Turk worker, I loved getting a glimpse of some of my "co-workers!"
What I find fascinating about most Turkers is the fact that, like me, they will work for what seems like a pittance, but give each assignment their best.
Andy, you hit the nail on the head. Flexibility is the key. Add that to the fact that, with thousands of HITs to choose from, everybody can find something that interests them.
Great HIT, btw!
12:47 PM
Loved this! Though I still don't know quite how to set up the job I want done, I have signed up with Turk and intend to give it a try now! Thanks for showing that we are outsourcing to some darn cute people!
2:06 PM
Haha, well done. I was at an Amazon Web Services demo in Toronto when I first heard about the mechanical turk service, and wondered the same things.
BTW come back and make some gifs, we've been updating the site for a while.
10:46 AM
I would have done it for 50 cents, I saw the HIT up, but I was really sick and I didn't want to take a picture of myself that made me look horrible!
2:30 AM
why the turks?
10:40 AM
Brilliant stuff.
10:40 AM
great idea.
and, why it's name is turk? what does it's mean?
10:52 AM
It's a reference to The Turk, a chess-playing automaton from the 18th century, that turned out to have been secretly powered by a human hiding inside. Likewise, Amazon Mechanical Turk is designed to handle large, tedious tasks in a programmatic fashion with human labor.
3:36 AM
I am the girl with the Ramen sign, I'm not "white", don't be so quick to judge and label races.
You said it's mindless work? Some of it is, but some of it takes time, effort, and thought (typing articles and that kind of thing). Those are jobs we should be getting paid way more for, but we have to take what we can get.
That one guy sooo isn't turking for lolz. But I think I was thinking about putting that. I turk for spare money pretty much and had barely bought some ramens with money I made. =P This hit was entertaining to do, so thanks!
4:59 AM
I love the Mechanical Turk posts, please keep them coming!!
11:18 PM
Adrián Lamo, ChaCha Guides get asked for their photos often. But I'm pretty sure they'd get in trouble for doing so. I've never seen any actually give their photo out.
(Including me, sorry horny 12 year old texters!)
Very interesting experiment though! :)
6:13 AM
I Turk to see what it's like.
7:12 AM
Very cool, I'm surprised to see this made it through fine because MTurk doesn't allow personally identifying info — someone's WHOLE FACE would certainly be that.