NDLI: Robust detection of comment spam using entropy rate

Please wait, while we are loading the content...

Machine learning meets social networking security: detecting and analyzing malicious social networks for fun and profit

Improving malware classification: bridging the static/dynamic gap

Learning stateful models for network honeypots

Robust detection of comment spam using entropy rate

Tracking concept drift in malware families

Early detection of malicious behavior in JavaScript code

Nonparametric semi-supervised learning for network intrusion detection: combining performance improvements with realistic in-situ training

Understanding the time-series behavioral characteristics of evolutionally advanced email spammers

Autonomous learning for detection of JavaScript attacks: vision or reality?

An information theoretic framework for web inference detection

On the effectiveness of using state-of-the-art machine learning techniques to launch cryptographic distinguishing attacks

Robust detection of comment spam using entropy rate

Content Provider	ACM Digital Library
Author	Tygar, J. D. Afroz, Sadia Joseph, Anthony Ma, Justin Huang, Ling Kantchelian, Alex
Abstract	In this work, we design a method for blog comment spam detection using the assumption that spam is any kind of uninformative content. To measure the "informativeness" of a set of blog comments, we construct a language and tokenization independent metric which we call content complexity, providing a normalized answer to the informal question "how much information does this text contain?" We leverage this metric to create a small set of features well-adjusted to comment spam detection by computing the content complexity over groupings of messages sharing the same author, the same sender IP, the same included links, etc. We evaluate our method against an exact set of tens of millions of comments collected over a four months period and containing a variety of websites, including blogs and news sites. The data was provided to us with an initial spam labeling from an industry competitive source. Nevertheless the initial spam labeling had unknown performance characteristics. To train a logistic regression on this dataset using our features, we derive a simple mislabeling tolerant logistic regression algorithm based on expectation-maximization, which we show generally outperforms the plain version in precision-recall space. By using a parsimonious hand-labeling strategy, we show that our method can operate at an arbitrary high precision level, and that it significantly dominates, both in terms of precision and recall, the original labeling, despite being trained on it alone. The content complexity metric, the use of a noise-tolerant logistic regression and the evaluation methodology are thus the three central contributions with this work.
Starting Page	59
Ending Page	70
Page Count	12
File Format	PDF
ISBN	9781450316644
DOI	10.1145/2381896.2381907
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2012-10-19
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Logistic regression Spam filtering Content complexity Comment spam Noisy label
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in