Detecting Abuse

Yahoo’s Abuse-Detecting Algorithm Works 90% of the Time & It’s A “Major Step Forward” in Its Field

Maddy Myers | July 29, 2016

Yahoo’s news articles have plenty of unsavory comments, much like the rest of the internet, so the Yahoo team decided to use their comments section in order to develop an algorithm that could successfully identify the worst offenders. Their new abuse-detecting algorithm works 90 percent of the time, which they say makes it more effective than other organizations’ attempts at taking on similar feats, and described as a “major step forward” in the field. 90 percent does sound pretty good, I admit.

Yahoo also crowdsourced abuse ratings using Amazon's Mechanical Turk, a website where anyone can sign up to perform tasks which require a degree of human intelligence, typically sorting images or analysing language. In this study, untrained people were paid the equivalent $0.02 for every online comment they attempted to categorise as abusive or non-abusive. Compared with Yahoo's own trained staff, the Mechanical Turk workers were much worse at detecting abuse, suggesting that well-trained staff are a vital part of abuse detection.

Source: http://www.wired.co.uk/article/yahoo-online-abuse-algorithm

[Click to Enlarge] Source: https://webscope.sandbox.yahoo.com/

[Click to Enlarge] Source: https://www.mturk.com/mturk/welcome

<more at http://www.themarysue.com/yahoo-anti-abuse-algorithm/; related articles and links: http://www.wired.co.uk/article/yahoo-online-abuse-algorithm (Yahoo's anti-abuse AI can hunt out even the most devious online trolls. The machine-learning algorithm was trained on a million Yahoo article comments. July 29, 2016) and http://www2016.net/proceedings/proceedings/p145.pdf (Abusive Language Detection in Online User Content. Chikashi Nobata, Joel Tetreault, Achint Thomas, Yshar Mehdad and Yi Chang. WWW 2016, April 11–15, 2016, Montréal, Québec, Canada. ACM 978-1-4503-4143-1/16/04. http://dx.doi.org/10.1145/2872427.2883062 [Abstract: Detection of abusive language in user generated online content has become an issue of increasing importance in recent years. Most current commercial methods make use of blacklists and regular expressions, however these measures fall short when contending with more subtle, less ham-fisted examples of hate speech. In this work, we develop a machine learning based method to detect hate speech on online user comments from two domains which outperforms a state-ofthe-art deep learning approach. We also develop a corpus of user comments annotated for abusive language, the first of its kind. Finally, we use our detection tool to analyze abusive language over time and in different settings to further enhance our knowledge of this behavior.])>