Team



Roshini Deva, Chia-Yun Tsai, Sukayna Hamka, Katie Teasley, Jamie Scribner, Abigail Lorincz, Aishwary Pawar,
Aditya Deshmukh



Role



UX Researcher



Academic Advisor : Feng Zhou, PhD, Assistant Professor at University of Michigan-Dearborn
Period : January 2022 - May 2022



Research Problem



The HCDE process is necessary to develop comment removal explanations for misinformation that consider the role and intentions of human moderators alongside content contributors. The conditions under which automated removal explanations satisfy user needs are not well understood in the current social media environment, particularly when content contributors are not consulted on their attitudes toward possible removal explanations.



The rapid adoption of social media presents a growing challenge for moderators to combat comments containing misinformation. Historically, the explanations for comment removal on social media have been opaque and variable. When users become educated of community norms through removal explanations, recurrence of future content violations has been shown to decrease. This raises the question of how automated tools and bots, that emulate human moderators, can be designed for improved explainability following comment removal. By leveraging human-centered design, we can elicit transparency in removal explanations by involving the perspectives of moderators and social media users throughout the problem-solving process. The HCDE approach promotes system usability and utility through an iterative cycle of empathizing with users to define their needs, ideate solutions, construct prototypes, and test for feedback and refinement of the design.



Purpose of the Study





The purpose of this study is to identify the conditions that affect users’ perception of explainability and transparency in automated comment removal explanations. The objective is to reduce ambiguity in explanation design so that users can form more accurate mental models of how online content is curated, helping curb the spread of misinformation. Additionally, we seek to identify patterns in comment removal explanations that affect users’ trust in moderation tools and practices.



Research Questions





How does the manner in which social media users receive feedback, on misinformation from comment removal explanations, cause a behavior shift, and to what extent does this impact the user experience?



Sub Questions





1) How do removal explanations influence users’ trust in social media platforms, other site members, and moderation tools?

2) How do removal explanations affect users’ perceptions of security, safety, liberty, legitimacy, and utility?

3) How does the spread of misinformation become impacted with AI-generated removals and explanations?



Audience and Benefits





We propose a design framework for automated comment removal explanations intended to support human moderators as their online communities scale. From this, we anticipate greater transparency and explainability of removal guidelines for users. Our framework will provide value to social media communities whose moderators seek to foster more productive contributors that submit approved content with greater consistency.



Designing the study



Research



Secondary Research :

Analyzed 10 - 15 research papers.

Primary Research :

1) Quantitative data :

Data Collection - Surveys & Polls

Analysis - Linear Regression Model

2) Qualitative data :

Data Collection - Interviews

Analysis - Open Coding



data Analysis



Analyzed qualitative and quantitative data received so far to categorize observations and create patterns through NVIVO. We will conduct interviews until categories are well developed and the relationships between them can be validated



findings



Major coding themes identified on understanding how users perceive the use of bots for moderating content on Reddit on behalf of humans were : Hesitance, Mixed, Laziness, Efficient, Exploitative, Helps with the Workload, Necessary, and Human Reviews



Next steps



we continue to interview participants, saturate our data, conduct coding analyses, and allow Reddit users to guide future data collection methods until our findings or design solutions can be validated



Get In Touch