This project aimed to create accurate and fair misinformation detection algorithms that can identify false or misleading information in online political news articles. Misinformation detection is a challenging task that requires not only technical skills but also ethical, social, and journalistic skills. Hence, this project yielded an interdisciplinary framework involving experts from information science, computer science, journalism, and communication to develop novel methods for creating fair and accurate misinformation detection algorithms. 

The project had the following main objectives:

• To study existing misinformation detection approaches for the existence of bias based on political leaning or other factors.

• To understand the mechanics and rationale behind the results found and the potential implications for public trust and democratic discourse.

• To identify ways to create fairer algorithms that can account for different perspectives and values.

The project carried out the following major activities:

• Experimented with multiple algorithms for testing the accuracy and fairness of machine learning algorithms for misinformation detection.

• Developed reliable coding procedures for manually labeling news articles for veracity and political leaning. This included training graduate students in coding procedures. One thousand (1000) news articles have been labeled following journalistic practices and shared as a public resource.

• Compared and contrasted the article labels for veracity and political leaning as identified by the journalistic labeling process versus source based attribution.

• Developed new methods for countering bias in misinformation detection algorithms using the above journalistically labeled dataset.

• Extended and validated above-mentioned ideas for algorithmic bias reduction in multiple domains e.g., misinformation detection, news sentiment detection, and cyberbullying detection.

The project achieved the following outcomes:

• The project contributed to the intellectual merit of the field by advancing the state-of-the-art in misinformation detection and algorithmic fairness. The project produced novel insights into the sources and effects of bias in misinformation detection and proposed methods to mitigate it. The project also developed a new dataset of news articles labeled for veracity and political leaning that can serve as a valuable resource for future research in computational journalism and fairness in machine learning literature. 

• The project also had broader impacts on society by raising awareness of the importance and challenges of misinformation detection and algorithmic fairness. The project engaged with various stakeholders such as journalists, researchers, and the general public through publications, presentations, workshops, and media outreach. The project also trained graduate students in interdisciplinary skills and fostered collaboration among researchers from different disciplines and institutions.

The project demonstrated that misinformation detection and algorithmic fairness are complex and interrelated issues that require careful consideration and collaboration. The project also showed that it is possible to create more accurate and fair algorithms that can help combat misinformation and promote informed citizenship.

The project disseminated its findings through various publications, presentations, workshops, and media outreach. Some of the notable ones are:

• One article on fairness in misinformation detection published in the proceedings of ACM Web Science Conference, 2023.

• One article on fairness in misinformation detection published in the proceedings of ICWSM- MEDIATE, 2022.

• One journal article on misinformation detection algorithms published in the Journal of Association for Information Science and Technology, 2020.

• Six additional conference proceedings articles and one journal resulted from the related work around fairness audit and bias reduction in related information-processing domains.

• Results from the work shared in multiple invited talks and panels at NSF-, NIST- and ACM- sponsored events.

The articles have been shared in the NSF Public Access Repository and the codebook and the labeled dataset on misinformation and political leaning are available at https://osf.io/qwnsf/

Overall, this project has created new approaches for creating fair and accurate misinformation detection algorithms that can support improved well-being of individuals in society and improved national security. This research has been contributing practical strategies for detecting and countering misinformation online.

Last Modified: 08/21/2023
Modified by: Vivek K Singh