Mturk Master Qualification
“How do I get the master qualifications?” Ah, a question that has plagued every mturk forum, WAH site and message board alike. How DOES one go about getting this sacred qualification? What ARE the steps that one needs to take to join the extraordinary ranks? Do I request the qualification? Are there time constraints on how long you have to turk?
At the start of whole Amazon Mechanical Turk special master qualification I really did think it was for the good of the requesters and workers. The qualification was handed out to proven workers that all had experience under their belts. These were turkers that had completed enough “gold” hits properly and really had a good grasp on what turking was all about. That is, all of the turkers that I knew that had received the qualification. It was a badge of honor that only the highest held turkers received. Sure there were those that probably should have received it but did not, trust me I thought I was one of them at the time, but it was still a qualification that I held in high prestige.
On the requesters side of the deal there was a special group of highly qualified turkers that, for a small price, they could use to complete their HITs. Sure the cost was a bit more, roughly 20%, but they were using the best turkers so the quality of work would be superb. A quote from the AMT faq for requesters states:
“Masters are elite groups of Workers who have demonstrated accuracy on specific types of HITs on the Mechanical Turk marketplace. Workers achieve a Masters distinction by consistently completing HITs of a certain type with a high degree of accuracy across a variety of Requesters.”
Which to me, states that master workers are the best of the best. This was all good and true in my eyes when the program started. However, since then my feelings have changed quite a bit.
During the progression of the program it was found that many requesters were not getting their work completed. These “master” hits would sit on the board not being completed because those that were granted the qualification really were the best of the best. The turkers that, in my opinion, deserved to be paid what they were worth. Masters were those that had demonstrated a high accuracy in their HIT completion. In fact, it is even stated on that same AMT faq page that,
“Because Masters have demonstrated accuracy, they can command a higher reward for their HITs. You should expect to pay Masters a higher reward.”
Yet the only one that was receiving a higher reward was Amazon. That 20% hike in pay was not going to the worker at all. It was lining the pockets of AMT on the backs and good hard work that the masters would provide. In fact, it seemed that most of the requesters that were using the master qualification were actually paying less to the worker because they had to pay so much more to Amazon for the service.
So what is the next step? What is Amazon to do if their master qualification hits are not being done? Well they soon had an answer. If the master hits that are lining their pockets with a 20% premium are not getting done then they must need more masters. There must be too much work for the few masters they have to complete. And that is when it happened. People with the master qualification started coming out of the woodwork. Turkers that had only been turking for a few months started receiving the honorable email… “Welcome to Mechanical Turk Masters“. It soon became clear that workers were being assigned the “Master” qualifications when they had hardly been vetted at all. Surely a few of these rookie workers were completing work at a very high quality on some tasks but the numbers just don’t support such a lofty tag when there has been that brief of a track record. I really feel it has turned into more of a ploy for AMT to get more money out of the requesters pockets and into theirs.
As for, “how do I get the master qualification?” ….. I am sorry to say that I cannot fully answer that question. Amazon will not say what you have to do or how you have to do it. There is no special amount of HITs to complete, no time constraints and no special experience needed. There are only vague hints to the workers and requesters alike. There does seem to be a very strong correlation between the masters qualification being granted and doing low priced hits that AMT graders “know” the answers to, but that is about it. Maybe Amazon is just looking to promote those workers who will do those low paying hits with the masters qualification no matter what their experience is. I suppose it is just like a real life job in that aspect. Some people work their asses off and don’t get ahead but there are always those who do as little as possible and live by the skin of their teeth yet somehow end up getting the best job in the house.
5,982 total views, 21 views today