De-biasing user preference ratings in recommender systems

Gediminas Adomavicius, Jesse C Bockstedt, Curley Shawn, Jingjing Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

Prior research has shown that online recommendations have significant influence on users' preference ratings and economic behavior. Specifically, the self-reported preference rating (for a specific consumed item) that is submitted by a user to a recommender system can be affected (i.e., distorted) by the previously observed system's recommendation. As a result, anchoring (or anchoring-like) biases reflected in user ratings not only provide a distorted view of user preferences but also contaminate inputs of recommender systems, leading to decreased quality of future recommendations. This research explores two approaches to removing anchoring biases from self-reported consumer ratings. The first proposed approach is based on a computational post-hoc de-biasing algorithm that systematically adjusts the user-submitted ratings that are known to be biased. The second approach is a user-interface-driven solution that tries to minimize anchoring biases at rating collection time. Our empirical investigation explicitly demonstrates the impact of biased vs. unbiased ratings on recommender systems' predictive performance. It also indicates that the post-hoc algorithmic debiasing approach is very problematic, most likely due to the fact that the anchoring effects can manifest themselves very differently for different users and items. This further emphasizes the importance of proactively avoiding anchoring biases at the time of rating collection. Further, through laboratory experiments, we demonstrate that certain interface designs of recommender systems are more advantageous than others in effectively reducing anchoring biases.

Original languageEnglish (US)
Title of host publicationCEUR Workshop Proceedings
PublisherCEUR-WS
Pages2-9
Number of pages8
Volume1253
StatePublished - 2014
EventJoint Workshop on Interfaces and Human Decision Making for Recommender Systems, IntRS 2014, Co-located with ACM Conference on Recommender Systems, RecSys 2014 - Foster City, United States
Duration: Oct 6 2014 → …

Other

OtherJoint Workshop on Interfaces and Human Decision Making for Recommender Systems, IntRS 2014, Co-located with ACM Conference on Recommender Systems, RecSys 2014
CountryUnited States
CityFoster City
Period10/6/14 → …

Fingerprint

Recommender systems
User interfaces
Economics
Experiments

Keywords

  • Anchoring effects
  • Rating de-biasing
  • Recommender systems

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Adomavicius, G., Bockstedt, J. C., Shawn, C., & Zhang, J. (2014). De-biasing user preference ratings in recommender systems. In CEUR Workshop Proceedings (Vol. 1253, pp. 2-9). CEUR-WS.

De-biasing user preference ratings in recommender systems. / Adomavicius, Gediminas; Bockstedt, Jesse C; Shawn, Curley; Zhang, Jingjing.

CEUR Workshop Proceedings. Vol. 1253 CEUR-WS, 2014. p. 2-9.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Adomavicius, G, Bockstedt, JC, Shawn, C & Zhang, J 2014, De-biasing user preference ratings in recommender systems. in CEUR Workshop Proceedings. vol. 1253, CEUR-WS, pp. 2-9, Joint Workshop on Interfaces and Human Decision Making for Recommender Systems, IntRS 2014, Co-located with ACM Conference on Recommender Systems, RecSys 2014, Foster City, United States, 10/6/14.
Adomavicius G, Bockstedt JC, Shawn C, Zhang J. De-biasing user preference ratings in recommender systems. In CEUR Workshop Proceedings. Vol. 1253. CEUR-WS. 2014. p. 2-9
Adomavicius, Gediminas ; Bockstedt, Jesse C ; Shawn, Curley ; Zhang, Jingjing. / De-biasing user preference ratings in recommender systems. CEUR Workshop Proceedings. Vol. 1253 CEUR-WS, 2014. pp. 2-9
@inproceedings{b688846c68ac43bdbca1be3c49d11e66,
title = "De-biasing user preference ratings in recommender systems",
abstract = "Prior research has shown that online recommendations have significant influence on users' preference ratings and economic behavior. Specifically, the self-reported preference rating (for a specific consumed item) that is submitted by a user to a recommender system can be affected (i.e., distorted) by the previously observed system's recommendation. As a result, anchoring (or anchoring-like) biases reflected in user ratings not only provide a distorted view of user preferences but also contaminate inputs of recommender systems, leading to decreased quality of future recommendations. This research explores two approaches to removing anchoring biases from self-reported consumer ratings. The first proposed approach is based on a computational post-hoc de-biasing algorithm that systematically adjusts the user-submitted ratings that are known to be biased. The second approach is a user-interface-driven solution that tries to minimize anchoring biases at rating collection time. Our empirical investigation explicitly demonstrates the impact of biased vs. unbiased ratings on recommender systems' predictive performance. It also indicates that the post-hoc algorithmic debiasing approach is very problematic, most likely due to the fact that the anchoring effects can manifest themselves very differently for different users and items. This further emphasizes the importance of proactively avoiding anchoring biases at the time of rating collection. Further, through laboratory experiments, we demonstrate that certain interface designs of recommender systems are more advantageous than others in effectively reducing anchoring biases.",
keywords = "Anchoring effects, Rating de-biasing, Recommender systems",
author = "Gediminas Adomavicius and Bockstedt, {Jesse C} and Curley Shawn and Jingjing Zhang",
year = "2014",
language = "English (US)",
volume = "1253",
pages = "2--9",
booktitle = "CEUR Workshop Proceedings",
publisher = "CEUR-WS",

}

TY - GEN

T1 - De-biasing user preference ratings in recommender systems

AU - Adomavicius, Gediminas

AU - Bockstedt, Jesse C

AU - Shawn, Curley

AU - Zhang, Jingjing

PY - 2014

Y1 - 2014

N2 - Prior research has shown that online recommendations have significant influence on users' preference ratings and economic behavior. Specifically, the self-reported preference rating (for a specific consumed item) that is submitted by a user to a recommender system can be affected (i.e., distorted) by the previously observed system's recommendation. As a result, anchoring (or anchoring-like) biases reflected in user ratings not only provide a distorted view of user preferences but also contaminate inputs of recommender systems, leading to decreased quality of future recommendations. This research explores two approaches to removing anchoring biases from self-reported consumer ratings. The first proposed approach is based on a computational post-hoc de-biasing algorithm that systematically adjusts the user-submitted ratings that are known to be biased. The second approach is a user-interface-driven solution that tries to minimize anchoring biases at rating collection time. Our empirical investigation explicitly demonstrates the impact of biased vs. unbiased ratings on recommender systems' predictive performance. It also indicates that the post-hoc algorithmic debiasing approach is very problematic, most likely due to the fact that the anchoring effects can manifest themselves very differently for different users and items. This further emphasizes the importance of proactively avoiding anchoring biases at the time of rating collection. Further, through laboratory experiments, we demonstrate that certain interface designs of recommender systems are more advantageous than others in effectively reducing anchoring biases.

AB - Prior research has shown that online recommendations have significant influence on users' preference ratings and economic behavior. Specifically, the self-reported preference rating (for a specific consumed item) that is submitted by a user to a recommender system can be affected (i.e., distorted) by the previously observed system's recommendation. As a result, anchoring (or anchoring-like) biases reflected in user ratings not only provide a distorted view of user preferences but also contaminate inputs of recommender systems, leading to decreased quality of future recommendations. This research explores two approaches to removing anchoring biases from self-reported consumer ratings. The first proposed approach is based on a computational post-hoc de-biasing algorithm that systematically adjusts the user-submitted ratings that are known to be biased. The second approach is a user-interface-driven solution that tries to minimize anchoring biases at rating collection time. Our empirical investigation explicitly demonstrates the impact of biased vs. unbiased ratings on recommender systems' predictive performance. It also indicates that the post-hoc algorithmic debiasing approach is very problematic, most likely due to the fact that the anchoring effects can manifest themselves very differently for different users and items. This further emphasizes the importance of proactively avoiding anchoring biases at the time of rating collection. Further, through laboratory experiments, we demonstrate that certain interface designs of recommender systems are more advantageous than others in effectively reducing anchoring biases.

KW - Anchoring effects

KW - Rating de-biasing

KW - Recommender systems

UR - http://www.scopus.com/inward/record.url?scp=84914681919&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84914681919&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84914681919

VL - 1253

SP - 2

EP - 9

BT - CEUR Workshop Proceedings

PB - CEUR-WS

ER -