Enhanced Detection of Hate Speech in Dravidian Languages in Social Media Using Ensemble Transformers
This study aims to propose an efficient implementation of identifying and detecting offensive and hate comments on social media platforms for Dravidian languages.
There has been a notable increase in hate comments on social media platforms in recent years. Hate language and hate speech are expressions of conflicts that arise between different groups, both within and across civilizations. This toxic behavior has a detrimental impact on individuals, resulting in disagreements, arguments, political conflicts, and even mental health issues such as cyberbullying, depression, and anxiety. In recent years, research on offensive detection has expanded beyond English and Hindi languages to include other languages such as Urdu, Spanish, Arabic, and more. An ongoing trend in modern research involves gathering data from prominent social media platforms such as YouTube, Instagram, and Twitter to train models in proposed studies.
The objective of this work is to identify and detect offensive or hateful comments on social media platforms dedicated explicitly to the Dravidian languages - Tamil, Kannada, and Malayalam. The dataset, HASOC-Offensive Language Identification track in Dravidian Code-Mix FIRE 2021, undergoes several proposed preprocessing procedures on the YouTube comments. Tokens were created to represent the attributes. The pre-trained models utilizing transformers were trained using tokenized data. The efficacy of different binary classifiers has been assessed and analyzed using the embedded vectors derived from the models. The mBert with the classifier model CATBOOST with GSCV achieved F1 scores for Tamil, Malayalam, and Kannada are 0.94, 0.98, and 0.82, respectively. Precision and recall values for Tamil, Malayalam, and Kannada are 0.94, 0.98, and 0.82, and 0.94, 0.98, and 0.83, respectively.
This research produced an approach to improve the results and F1-score by performing an improved preprocessing method using stemming and stopword removal. The native and English-coded comment data used in this work were not discussed much in the previous works.
The findings of this study indicated that the preprocessing work combined both the native language data and the English-coded language. It describes the improved models and performance of the results in order to detect hate speech.
This study is expected to be valuable for social media platforms and other review sites to separate offensive comments from good and bad ones. This is also valuable for content creators and users in improving their ideas and content and also helps in preventing cyber harassment and bullying.
This study discussed the use of stemming and stopword removal and how it improved the detection of hate comments for both native and codemix comments in social media.
This work helps the users and community to explore the positive side of the internet and content creators to share their ideas without undermining the benefits of online interactions.
Future research will explore the uniqueness of the linguistics and slang of these three languages. Apart from these languages, future research will be conducted on North-Indian Indian languages like Bengali, Marathi, and Gujarati.
Back