Optimizing Worker Performance in Crowdsourcing Platforms

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Optimizing Worker Performance in Crowdsourcing Platforms"

By

Miss Ting WU


Abstract

Recently, the popularity of crowdsourcing has brought a new opportunity for 
engaging human intelligence into the process of data analysis. Crowdsourcing 
provides a fundamental mechanism for enabling online workers to participate 
tasks that are either too difficult to be solved solely by computers or too 
expensive to employ experts to perform. Though human is intelligent, meanwhile, 
human is erroneous and greedy, which causes the quality of crowdsourcing 
results quite questionable. In this thesis, we discuss three novel approaches 
to optimize the worker performance in Crowdsourcing platforms. They are 
Diversity-Based Worker Selection, Pay-As-You-Go Scheme and Panel Training.

In the field of social science, four elements are required to form a wise crowd 
- Diversity of opinion, Independence, Decentralization and Aggregation. 
Diversity- Based Worker Selection addresses the algorithmic optimizations 
towards the “diversity of opinion” of crowdsourcing marketplaces. We propose 
Similarity-driven Model(SModel) and Task-driven Model(T-Model) for two basic 
paradigms of worker selection. Pay-As-You-Go-Scheme is a new crowdsourcing 
paradigm for Object Identification tasks. In this paradigm, requester 
interactively evaluate each detected object from the crowd, and a worker is 
paid unit of reward for each detected object if it is verified by the 
requester. Such a paradigm not only resolves the difficulty for requester to 
evaluate the performance of the worker, but also avoids same objects being 
detected by many workers and ending up being meaningless workload. Panel 
Training focus on one of the most common and natural practice of crowdsourcing 
- collecting ratings of items. We design a sample-driven rubric to train 
workers, so they would standardized understanding of the rating criteria.


Date:			Thursday, 17 November 2016

Time:			9:00am - 11:00am

Venue:			Room 4472
  			Lifts 25/26

Chairman:		Prof. Man Yu (ISOM)

Committee Members:	Prof. Lei Chen (Supervisor)
 			Prof. Pan Hui (Supervisor)
  			Prof. Dimitris Papadias
  			Prof. Yangqiu Song
  			Prof. Jiheng Zhang (IELM)
  			Prof. Jian Pei (Comp. Sci., Simon Fraser Univ.)
 			Prof. Jianliang Xu (Comp. Sci., Baptist U.)


**** ALL are Welcome ****