| Abstract |
The usefulness of the World Wide Web as a digital library ofprecise and reliable information is reduced by the increasingpresence of advertising on Web pages. But no one is required toread or see advertising, and this cognitive censorship can beautomated by software. Such filters can be useful to the U.S.government which must permit its employees to use the Web but whichis prohibited by law from endorsing commercial products. While thetask would seem at first simpler than filtering of pornography orgeneral firewalls, subtleties in recognizing advertising make fullsuccess daunting.Our work is evaluating the quality of methods and products forautomatic ad censorship. Commercial products include AdKiller(www.adkiller.com), Ad Subtract Pro (www.adsubtract.com),Advertising Killer (www.buypin.com), AdWiper (www.adwiper.com),FilterGate (www.adscience .co.uk), and WebWasher(www.webwasher.com). Other products prevent the annoying popupwindows that are usually ads. Things these products do includeremoval of ad-like images embedded in the page, prevention of popupwindows and Javascript alert boxes, prevention of blinking text,and prevention of playing of embedded audio. Or so the vendorsclaim. Not a single one provides any statistics on the accuracy oftheir product (e.g. recall and precision) to support their grandclaims of removing ads.We are experimenting to determine how effective varioustechniques are in ad censoring. We constructed a censor of our own("Big Head") with manipulable features. We use Java servletsoftware to implement a page server that fetches HTML source textand edits it to create a modified page for display. The modifiedpage has blanks in the places of inferred ads, and substituteslocal links for remote links to permit further censoring. Blanksare made the same shape as the censored ads so that meaningful pagelayout can be preserved.We initially examined a variety of Web sites to develop a set ofclue properties for both image ads and their associated text,considering the text within a fixed-size window around the HTMLimage reference. We defined ads as information intended to arouse adesire to purchase or patronize something. It became clear thatidentification methods need to include both logical andprobabilistic methods to achieve high recall (fraction of adsremoved from pages), although high precision (fraction of ads amongthe items eliminated) was easy by simply picking the popup windowsand narrow banner-size images. Certain image dimensions are strongclues for ads, especially 480 by 60 banners and 150 by 500, 120 by600, and 160 by 600 images along the sides of the page. Imagesstored on sites different from the page's site (i.e. with differentfirst part of their URL) were also very likely to be ads, as wereimages whose file names contained long integers. These criteria aresufficiently strong to give 95% precision in identifying imageads.Additional weaker criteria used included the words of the imagefile name (the image URL), words of any directly associated text("alt" string), and other words within a fixed-size window aroundthe image reference. Good clue words and phrases were obtained froma study of random commercial Web pages. Examples are "ad", "buy","shop", "free", "join", "click", and "now". The strength of eachclue was estimated as the fraction of the time that the image wasan ad when the word was associated with it. In addition, image adswere usually larger than 2500 pixels, and "alt" text for ads wasusually less than 100 characters long; both tendencies can bemodeled by probability distributions derived from statistics fromexample pages. Evidence from these weaker clues was combined usinga linear model (or weighted average), and items were eliminated iftheir weighted sum exceeded a fixed threshold. In a quick test, ourprogram examined representative pages, and correctly recognized 19of 20 ads and 153 of 156 non-ads where ads were manually identifiedin advance. Public access is fromhttp://triton.cs.nps.navy.mil:8080/rowe/rowedemos.html.Text ads also have exploitable syntax. Analogously to what wedeveloped for finding image captions in our MARIE-4 Web crawler,incitements to purchase typically use a limited range ofgrammatical expressions recognizable by a partial parser. Goodexamples are expressions of the imperative form of verb indicatingacquiring ("buy", "get", "join", "click", etc. followed by a nounindicating a purchasable quantity (a physical object or a service),with optionally a qualifying adjective on the noun or an adverbindicating a desirable property of the acquisition ("now", "free","soon", etc.) Such a parser can approach semantic understanding ofadvertising text and improve precision of ad identification.Our research is ongoing. Future work will obtain more reliableperformance statistics on representative Web pages, and willinvestigate methods of identifying more difficult kinds of ads.Though we did not consider it yet, elimination of popup windows andJavascript applets is usually straightforward from analysis of theHTML source code. We hope to publish performance comparisons ofdifferent methods and different vendor products soon. |
