Logo image
Back
Worker-job recommendation for mixed crowdsourcing systems: algorithms, models, metrics and service-oriented architecture
Dissertation   Open access

Worker-job recommendation for mixed crowdsourcing systems: algorithms, models, metrics and service-oriented architecture

Julian Jarrett
Doctor of Philosophy (Ph.D.), Drexel University
Jun 2017
DOI:
https://doi.org/10.17918/etd-7815
pdf
Jarrett_Julian_20173.58 MBDownloadView
PDF Open Access Open Access (License Unspecified)

Abstract

Information science Human computation Human beings Machinery Recommender systems (Information filtering) Computer Science Information Technology
Crowdsourcing is used as model to distribute work over the Internet via an open call to anonymous human workers, who opt to take up work offerings sometimes for some small compensation. Increasingly, crowdsourcing systems are integrated into workflows to provide human computation capabilities. These workflows consist of machine-based workers that work harmoniously on different phases of a task with their human counterparts. This body of work addresses workflows where machines and human workers have the capacity to fulfill the requirements for same tasks. To maximize performance through the delegation of work to the most competent worker, this work outlines a collaborative filtering based approach with a bottom up evaluation based on workers' performance history and their inferred skillsets. Within the model, there are several algorithms, formulae and evaluative metrics. The work also introduces the notion of an Open Push-Pull model; a paradigm that maximizes on the services and strengths of the open call model, while seeking to address its weaknesses such as platform lock-in that affects access to jobs and availability of the worker pool. The work outlines the model in terms of a service-oriented architecture (SOA). It provides a supporting conceptual model for the architecture and an operational model that facilitates both human and machine workers. It also defines evaluative metrics for understanding the true capabilities of the worker pool. Techniques presented in this work can be used to expand the potential worker pool to compete for tasks through the incorporation of machine-oriented workers via virtualization and other electronic services, and human workers via existing crowds. Results in this work articulate the flexibility of our approach to support both human and machine workers within a competitive model while supporting tasks spanning multiple domains and problem spaces. It addresses the inefficiencies of current top-down approaches in worker-job recommendation through use of a bottom-up approach which adapts to dynamic and rapidly changing data. The work contrasts the shortcomings of top-down approaches' dependency on professed profiles which can be under-represented, over-represented or falsified in other ways with evaluative metrics that can be used for the individual and collective assessment of workers within a labor pool.

Metrics

54 File views/ downloads
45 Record Views

Details

Logo image