The Faces of Mechanical Turk

Posted by Andy Baio

When you experiment with Amazon’s Mechanical Turk, it feels like magic. You toss 500 questions into the ether, and the answers instantly start rolling in from anonymous workers around the world. It was great for getting work done, but who are these people? I’ve seen the demographics, but that was too abstract for me.

Last week, I started a new Turk experiment to answer two questions: what do these people look like, and how much does it cost for someone to reveal their face?

Answer #1. This is what Mechanical Turk looks like (click for full-size):

Answer #2. About $0.50.

Results

Here’s my original request:

Upload a photo of yourself holding a handwritten sign that says “I Turk for …”, filling out why you turk. For example, “I Turk for Cash,” “I Turk for My Kids,” “I Turk to Kill Time,” or whatever else you like. Be honest, be funny, be whatever you like.

As a good faith gesture, here’s my photo.

If you have a webcam, you can simply go to Cameroid to snap a photo from your web browser, download the JPG, and upload it below. (Don’t worry if the text is backwards, I can fix that myself.) DON’T provide any identifiable information, like your name or email, since that’s a violation of MTurk policy.

The result will be used in a collage that can be found on my personal weblog, http://waxy.org. By uploading your image and accepting payment for the image, you give permission to me, Andy Baio, to use your image in all forms and media for any lawful purposes. (That’s just cover-my-ass language. I’m almost certainly only going to restrict it to this one project.) The collage will show up there shortly after the HIT is complete. Thanks, everybody!

I started the task at $.05, but only two people responded in the first 24 hours. (And one of those was Joshua Schachter, who I’d told about the project.) Clearly, that was too low, so I increased it to $.25, receiving only eight submissions in 48 hours. (For reference, all 500 of my Girl Talk tasks were done in about an hour.) Increasing it to $.50 got me 20 more submissions in about 48 hours, after which it started to drop off quickly. I wasn’t about to give dollar bills to random people for their photos, so I ended the experiment there. People aren’t willing to give up their anonymity for cheap.

The final results: 30 people total — 10 women, 20 men. Almost all were white, mostly in their 20s and 30s. 21 said they turked for money, 9 for fun or boredom.

Thanks for pulling back the curtain, Turkers.

Comments

    Did you have to reject many for not following the right procedure? I could see people faking it with pictures of other people’s faces.

    Kevin: On the big image, you can see a white border separating the three groups… Two people at the top for $.05, the next eight for $.25, and the rest for $.50.

    Dave: That’s why I required a handwritten sign, which is harder to fake (but far from impossible). There were only two people that I rejected. One accidentally sent a digital photo of someone’s back (I suspect this was an accident) and the other photoshopped his sign in with type.

    Seems like it would actually be more work to photoshop a sign into another photo than just to take one of your own (and I’m pretty quick with photoshop). In a way, I continue to be stunned that people will do these tasks for so little money. I’d have to be getting at least $5-10 to bother with something like this (not that it’s a lot of work, it’s just the annoyance factor). In this particular case, you may have gotten some people who participated at a lower price than they otherwise would have because they liked the idea of it. But even with mundane tasks, I’m amazed by the low prices.

    Great post. As a researcher I’ve been wanting to use Mechanical Turk for some of my machine learning tasks. This clears away the fog about who are the real turkers and reduces my anxiety about using it.

    Your post is a great validation for this wonderful tool that matches research needs with human needs.

    Sean: I think a big part of the appeal for turkers motivated by money is the flexibility. These people can work for five minutes at a time in between classes, during commercials, or during their lunch breaks. It’s simple, mindless work that can be done in spare time for spare change.

    @Panos: of course the people still retain the copyright in the images they submitted. Clearly they’ve consented for the authors use of those images but you’d have to (legally) contact them to request redistribution and/or reproduction rights.

    You may also need model releases including (it appears in at least one case) parental consent.

    FWIW.

    A lawyer who wanted the fight could also defend an infringement suit for the rights of the 3 people shown in 2 collages, the agreement says “used in a collage that can be found on my personal weblog” and that ‘sample’ is not a subsection of the collage.

    pbhj: I used standard language for a model release form. The agreement stated clearly, “By uploading your image and accepting payment for the image, you give permission to me, Andy Baio, to use your image in all forms and media for any lawful purposes.”

    Since I have a model release, I’m placing them under the same Creative Commons license as the rest of my original content.

    As a result, it’s not necessary for Panos to try to obtain his own model releases… which would be completely impossible anyway, since the Mechanical Turk submitters are all anonymous. As long as his use is non-commercial, which I suspect it is, he’s golden.

    Actually, even if the images were copyrighted, Panos could claim fair use for using the image for educational purposes at his school.

    One thing — the fact that $0.50 was required to get anyone to do it, maybe not necessarily be just the price of revealing their face. I bet a big part of that 50 cents is the hassle of making a sign and figuring out how to take the picture.

    Or maybe making a sign is fun — a task to just take a picture might cost more?

    This is really fantastic! Putting a face to a “turk” is quite reassuring in some ways.

    I’ve had such great success with the Turk community though, that I’ve wondered less about who they are. I also have 300 replies from Turkers stating their professions, which I really should post on my blog to add another dimension to these mysterious people. For now here’s a small sampling of their occupations (much more than simply students and part-time workers): spanish translator; United Nations worker; webmaster of dogtoys.com; researcher at Duke Medical Center; paralegal at Murray Law Office; manager of a Bait & Tackle shop in Florida…

    As a Turk worker, I loved getting a glimpse of some of my “co-workers!”

    What I find fascinating about most Turkers is the fact that, like me, they will work for what seems like a pittance, but give each assignment their best.

    Andy, you hit the nail on the head. Flexibility is the key. Add that to the fact that, with thousands of HITs to choose from, everybody can find something that interests them.

    Great HIT, btw!

    Loved this! Though I still don’t know quite how to set up the job I want done, I have signed up with Turk and intend to give it a try now! Thanks for showing that we are outsourcing to some darn cute people!

    Haha, well done. I was at an Amazon Web Services demo in Toronto when I first heard about the mechanical turk service, and wondered the same things.

    BTW come back and make some gifs, we’ve been updating the site for a while.

    I would have done it for 50 cents, I saw the HIT up, but I was really sick and I didn’t want to take a picture of myself that made me look horrible!

    It’s a reference to The Turk, a chess-playing automaton from the 18th century, that turned out to have been secretly powered by a human hiding inside. Likewise, Amazon Mechanical Turk is designed to handle large, tedious tasks in a programmatic fashion with human labor.

    I am the girl with the Ramen sign, I’m not “white”, don’t be so quick to judge and label races.

    You said it’s mindless work? Some of it is, but some of it takes time, effort, and thought (typing articles and that kind of thing). Those are jobs we should be getting paid way more for, but we have to take what we can get.

    That one guy sooo isn’t turking for lolz. But I think I was thinking about putting that. I turk for spare money pretty much and had barely bought some ramens with money I made. =P This hit was entertaining to do, so thanks!

    Adrián Lamo, ChaCha Guides get asked for their photos often. But I’m pretty sure they’d get in trouble for doing so. I’ve never seen any actually give their photo out.

    (Including me, sorry horny 12 year old texters!)

    Very interesting experiment though! 🙂

    Very cool, I’m surprised to see this made it through fine because MTurk doesn’t allow personally identifying info — someone’s WHOLE FACE would certainly be that.

    Haha! I didn’t expect to see my face through StumbleUpon, but here we are, and there I am. I’m glad to finally meet one of the mysterious “requesters”. 😀 I love Mechanical Turk like crazy, and I’ll definitely be following you now. I’ve been thinking about setting up a discussion group on the possible creative and beneficial uses of Mechanical Turk as a tool.

    Anyways, again, thanks for the CASH!

    Hey Andy,

    This is a great idea. I’d like to try something similar for my school classes. How did you get them to upload the photo’s? Where where they uploaded to?

    thx.

    Mechanical Turk has support for attaching files, but you need to use the older web-based HIT creator to do it. Use the “Create HITs individually” link on the Requester homepage to find it.

    I signed up on Mturk today, as a worker, to see if it is the sort of thing I can use as a buyer. What really surprised me is the really, really, low prices paid for what is often not that mindless…someone wants a 500-word article rewritten TWICE for $4!! But yeah, I guess if I am really bored I will do it and donate the proceeds to kiva.org where even $5 can change a life.

    This is an old post, but I’m just floored by the creativity! What a super unique use of the mturk system, great idea and lots of fun.

    I’ve been outsourcing for years and just started using mturk a couple weeks ago. In fact I did a similar experiment to this (just not as sexy)where I created an HIT asking that the provider write a 500 word article on why they use mturk and their good / bad experiences. It was fun and I posted one of the articles submitted here: http://www.outsourcedmylife.com/why-i-work-from-home.php

    I’m really enjoying testing out the Amazon Mechanical Turk system and using it for super repetitive and mundane tasks that I don’t even want to give to my virtual assistant.

    I continue to be stunned that people will do these tasks for so little money. I’d have to be getting at least $5-10 to bother with something like this (not that it’s a lot of work, it’s just the annoyance factor). In this particular case, you may have gotten some people who participated at a lower price than they otherwise would have because they liked the idea of it. But even with mundane tasks, I’m amazed by the low prices.

    people may do tasks for others for very little money but then again, every drop makes an ocean. i worked on it for about a week now and have already earned 65$. but not only does it pay, but it’s a good learning experience to be writing articles, reviewing sites and participating in surveys. moreover its fun. it helps me utilize my time instead of wasting time on social networking websites. mturk is unique. its great !

Comments are closed.