Multimodal classification of moderated online pro-eating disorder content

Stevie Chancellor; Yannis Kalantidis; Jessica A. Pater; Munmun De Choudhury; David A. Shamma

doi:10.1145/3025453.3025985

Multimodal classification of moderated online pro-eating disorder content

Stevie Chancellor, Yannis Kalantidis, Jessica A. Pater, Munmun De Choudhury, David A. Shamma

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

63 Scopus citations

Abstract

Social media sites are challenged by both the scale and variety of deviant behavior online. While algorithms can detect spam and obscenity, behaviors that break community guidelines on some sites are difficult because they have multimodal subtleties (images and/or text). Identifying these posts is often regulated to a few moderators. In this paper, we develop a deep learning classifier that jointly models textual and visual characteristics of pro-eating disorder content that violates community guidelines. Using a million Tumblr photo posts, our classifier discovers deviant content efficiently while also maintaining high recall (85%). Our approach uses human sensitivity throughout to guide the creation, curation, and understanding of this approach to challenging, deviant content. We discuss how automation might impact community moderation, and the ethical and social obligations of this area.

Original language	English (US)
Title of host publication	CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems
Subtitle of host publication	Explore, Innovate, Inspire
Publisher	Association for Computing Machinery
Pages	3213-3226
Number of pages	14
ISBN (Electronic)	9781450346559
DOIs	https://doi.org/10.1145/3025453.3025985
State	Published - May 2 2017
Externally published	Yes
Event	2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017 - Denver, United States Duration: May 6 2017 → May 11 2017

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings
Volume	2017-May

Other

Other	2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017
Country/Territory	United States
City	Denver
Period	5/6/17 → 5/11/17

Bibliographical note

Publisher Copyright:
© 2017 ACM.

Keywords

Computer vision
Content moderation
Deep learning
Deviant behavior
Pro-eating disorder
Social media
Tumblr

Access

10.1145/3025453.3025985

OpenUrl availability

Full text

Cite this

Chancellor, S., Kalantidis, Y., Pater, J. A., De Choudhury, M., & Shamma, D. A. (2017). Multimodal classification of moderated online pro-eating disorder content. In CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems: Explore, Innovate, Inspire (pp. 3213-3226). (Conference on Human Factors in Computing Systems - Proceedings; Vol. 2017-May). Association for Computing Machinery. https://doi.org/10.1145/3025453.3025985

Multimodal classification of moderated online pro-eating disorder content. / Chancellor, Stevie; Kalantidis, Yannis; Pater, Jessica A. et al.
CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems: Explore, Innovate, Inspire. Association for Computing Machinery, 2017. p. 3213-3226 (Conference on Human Factors in Computing Systems - Proceedings; Vol. 2017-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Chancellor, S, Kalantidis, Y, Pater, JA, De Choudhury, M & Shamma, DA 2017, Multimodal classification of moderated online pro-eating disorder content. in CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems: Explore, Innovate, Inspire. Conference on Human Factors in Computing Systems - Proceedings, vol. 2017-May, Association for Computing Machinery, pp. 3213-3226, 2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017, Denver, United States, 5/6/17. https://doi.org/10.1145/3025453.3025985

Chancellor S, Kalantidis Y, Pater JA, De Choudhury M, Shamma DA. Multimodal classification of moderated online pro-eating disorder content. In CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems: Explore, Innovate, Inspire. Association for Computing Machinery. 2017. p. 3213-3226. (Conference on Human Factors in Computing Systems - Proceedings). doi: 10.1145/3025453.3025985

Chancellor, Stevie ; Kalantidis, Yannis ; Pater, Jessica A. et al. / Multimodal classification of moderated online pro-eating disorder content. CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems: Explore, Innovate, Inspire. Association for Computing Machinery, 2017. pp. 3213-3226 (Conference on Human Factors in Computing Systems - Proceedings).

@inproceedings{fa9593e6fdb44408a6029064525330a9,

title = "Multimodal classification of moderated online pro-eating disorder content",

abstract = "Social media sites are challenged by both the scale and variety of deviant behavior online. While algorithms can detect spam and obscenity, behaviors that break community guidelines on some sites are difficult because they have multimodal subtleties (images and/or text). Identifying these posts is often regulated to a few moderators. In this paper, we develop a deep learning classifier that jointly models textual and visual characteristics of pro-eating disorder content that violates community guidelines. Using a million Tumblr photo posts, our classifier discovers deviant content efficiently while also maintaining high recall (85%). Our approach uses human sensitivity throughout to guide the creation, curation, and understanding of this approach to challenging, deviant content. We discuss how automation might impact community moderation, and the ethical and social obligations of this area.",

keywords = "Computer vision, Content moderation, Deep learning, Deviant behavior, Pro-eating disorder, Social media, Tumblr",

author = "Stevie Chancellor and Yannis Kalantidis and Pater, {Jessica A.} and {De Choudhury}, Munmun and Shamma, {David A.}",

note = "Publisher Copyright: {\textcopyright} 2017 ACM.; 2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017 ; Conference date: 06-05-2017 Through 11-05-2017",

year = "2017",

month = may,

day = "2",

doi = "10.1145/3025453.3025985",

language = "English (US)",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

pages = "3213--3226",

booktitle = "CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems",

}

TY - GEN

T1 - Multimodal classification of moderated online pro-eating disorder content

AU - Chancellor, Stevie

AU - Kalantidis, Yannis

AU - Pater, Jessica A.

AU - De Choudhury, Munmun

AU - Shamma, David A.

PY - 2017/5/2

Y1 - 2017/5/2

N2 - Social media sites are challenged by both the scale and variety of deviant behavior online. While algorithms can detect spam and obscenity, behaviors that break community guidelines on some sites are difficult because they have multimodal subtleties (images and/or text). Identifying these posts is often regulated to a few moderators. In this paper, we develop a deep learning classifier that jointly models textual and visual characteristics of pro-eating disorder content that violates community guidelines. Using a million Tumblr photo posts, our classifier discovers deviant content efficiently while also maintaining high recall (85%). Our approach uses human sensitivity throughout to guide the creation, curation, and understanding of this approach to challenging, deviant content. We discuss how automation might impact community moderation, and the ethical and social obligations of this area.

AB - Social media sites are challenged by both the scale and variety of deviant behavior online. While algorithms can detect spam and obscenity, behaviors that break community guidelines on some sites are difficult because they have multimodal subtleties (images and/or text). Identifying these posts is often regulated to a few moderators. In this paper, we develop a deep learning classifier that jointly models textual and visual characteristics of pro-eating disorder content that violates community guidelines. Using a million Tumblr photo posts, our classifier discovers deviant content efficiently while also maintaining high recall (85%). Our approach uses human sensitivity throughout to guide the creation, curation, and understanding of this approach to challenging, deviant content. We discuss how automation might impact community moderation, and the ethical and social obligations of this area.

KW - Computer vision

KW - Content moderation

KW - Deep learning

KW - Deviant behavior

KW - Pro-eating disorder

KW - Social media

KW - Tumblr

UR - http://www.scopus.com/inward/record.url?scp=85030310646&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030310646&partnerID=8YFLogxK

U2 - 10.1145/3025453.3025985

DO - 10.1145/3025453.3025985

M3 - Conference contribution

AN - SCOPUS:85030310646

T3 - Conference on Human Factors in Computing Systems - Proceedings

SP - 3213

EP - 3226

BT - CHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

T2 - 2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017

Y2 - 6 May 2017 through 11 May 2017

ER -

Multimodal classification of moderated online pro-eating disorder content

Abstract

Publication series

Other

Bibliographical note

Keywords

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this