UC Santa Cruz

Reputation systems in most marketplaces involve two parties, reviewers and reviewees,

e.g., employers and employees, buyers and sellers, consumers and advertisers. In this bipartite system, reviewers explicitly or implicitly assign reviewees ratings of their quality, value, or performance, e.g., job competence, product value, level of interest drawn. These ratings are aggregated by the system through a certain mechanism to give reviewees a score for their "reputation" in the community, an indication of their trustworthiness or reliability according to their reviewers. This reputation score can then be used by other people in "choosing" a reviewee (e.g., hiring a worker, buying a product). On the other hand, bias scores are computed for the reviewers, based on the reputation of the items they have rated. This bias can then be used when our interest is in representing

the reviewer and his judgements.

In this work, we study the existing reputation systems, and propose new ones, in two

different marketplaces: labor and advertising.

In the labor context, we study how reputation systems contribute to smart hiring. In this

process, the employer posts a job in the marketplace to receive applications from interested workers.

After evaluating the suitability of applicants for the job, the employer hires one or more of them via an online contract. Once the job is completed and the contract ended, the employer can provide the worker with a rating, which becomes visible on the worker's online profile. This explicit feedback can guide future hiring decisions because it (supposedly) indicates the worker's true ability.

However, reputation systems based on end-of-contract ratings have several shortcomings, such as data sparsity, review bias and skewness, and latency in acquiring review signals. Our study builds reputation mechanisms that use Bayesian updates to combine employers' implicit feedback signals of actual application evaluation, e.g., hiring, interviewing, and rejecting, in a link-analysis approach and thus address such shortcomings while yielding better signals of worker quality to inform hiring decisions.

In the advertising context, we study how reputation systems benefit smart targeting. Audience selection is one of the determining factors in bidding success in all forms of advertising. Typically, advertisers select their audiences either by manually targeting rules or by running machine-learning models on users' past performance data. Here, the reputation setting is based on the decision of a user (reviewer) to convert to an ad posted by an advertiser (reviewee).

However, traditional reputation systems based on explicit user preferences (clicks, actions, or conversions) on the advertised content (opinion about the advertised product or advertiser) mainly accumulate such opinions, explicit or implicit (such as search queries or website visits), by representing them in user profiles. This system has several shortcomings, such as poor representation of interest, along with data sparsity and latency in acquiring user review signals.

Therefore, we create a reputation system that estimates the quality of the items preferred

by users, on top of which items they prefer, to accurately represent user interest while automating the audience selection process. This system also produces immediate results for the cold start problem of new campaigns by accounting for crowd insights from a pool of similar auctioneers, in a privacy- preserving environment.

Overall, the proposed reputation systems show how challenges regarding transparency,

availability, and privacy are overcome in the two marketplaces, and provide a general framework for addressing cold start in typical reputation environments.