stopthehacker.com » norton http://www.stopthehacker.com Jaal, LLC Wed, 01 Sep 2010 18:08:05 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Website-Reputation Services Agree to Disagree http://www.stopthehacker.com/2010/01/17/website-reputation-services-agree-to-disagree/ http://www.stopthehacker.com/2010/01/17/website-reputation-services-agree-to-disagree/#comments Sun, 17 Jan 2010 21:17:34 +0000 anirban http://www.stopthehacker.com/?p=1014 We have recently published statistics comparing various website reputation services and have received good feedback over private channels regarding our article. In this sequel we add Microsoft’s Bing, malware filter along with comparison to other website reputation services.

At StopTheHacker.com (Jaal LLC) we have conducted tests of 721 URLs, all of which have been reported as malicious by volunteers of various blacklists. We follow a similar format for presentation of results as in the last post.

Website Reputation services: agree to disagree.

Website Reputation services: agree to disagree.

Note: All 721 domains/URLs, were reported as malicious, and were collected from malware.com.br on January 14, 2010. The blue column (maximum 100) indicates the percentage of sites that the website-reputation service correctly identified as unsafe. The orange column (maximum 100) indicates the percentage of sites that the website-reputation services incorrectly identified as safe.

The aim of the test:

  1. Identify the accuracy of the website reputation service
  2. Identify the overlap in terms of safe/unsafe websites

We present the most interesting results in this article. First we detail the parameters of the testing procedure to provide an idea of how the test was set up.

First, 721 URLs were collected from malware.com.br (mbr) on January 14, 2010. These URLs are reported for listing by one or more of the following: individuals, organizations, agencies and software products or services.  For the purposes of this test we assume that all the URLs obtained from the “regular” list on mbr are malicious and hence deemed “unsafe” to visit.

We compare the reputation provided by each website-reputation service and observe how many websites are marked unsafe, safe, untested, maybe-unsafe/caution/potentially-unsafe, and unreachable.

Website-reputation services tested:

Note, that when analyzing a domainname/URL, for checking with the Google safebrowsing API, we have calculated the MD5 hash of the website name to match with the malware hash list. The date that we conducted this test was: January 15, 2010. The list of domain names tested are presented below and a graph representing the statistics for the 721 sites tested is above.

We identify the most interesting results below:

  1. McAfee SiteAdvisor marked 36.75% of domains as Unsafe, 27.18% as Safe, 32.32% as Untested and 3.74% as Potentially-Unsafe.
  2. Norton Safe Web marked 41.75% of domains as Unsafe, 45.49% as Safe, 4.3% as Untested and 8.32% as Potentially-Unsafe.
  3. Google Safe Browsing marked 5.96% of domains as Unsafe, 94.04% as Safe.
    Note: The presence of the hash of the domain name  being tested, on the google malware hash list, is interpreted as “unsafe” while the absence is interpreted as “safe.”
  4. Microsoft Bing marked 0.69% of domains as Unsafe, 34.26% as Safe, and 65.05% as Untested
  5. Comodo SiteInspector marked 0.19% of domains as Unsafe, 95.82% as Safe, and 4.08% as Unreachable.

This follow-up experiment also shows that the variance between website reputation services that are currently being offered by large Internet-services/security companies continues to be very large indeed.

After discussions with representatives of the companies mentioned in this article, and getting a better idea of their behind the scenes methodologies. It seems that these website reputation services will continue to “agree to disagree.” We welcome their comments.

A note on differences between website reputation services:

Some of the services scan pages and some scan parts of a site. Some scan for potential “signs” of an infection, while others scan for the “postmortem” effect of an infection, such as an exploit being launched. Furthermore, the time difference between one of the services testing a web page or site versus when another one tests the same web page can also complicate issues. At StopTheHacker.com we recognize the current limitations of website reputation services that being offered by the industry.

In conclusion, while website reputation services have come a long way, they still have an even longer path to tread in order to become something that users should trust implicitly.

]]> http://www.stopthehacker.com/2010/01/17/website-reputation-services-agree-to-disagree/feed/ 0 How Good Are Website-Reputation Services? http://www.stopthehacker.com/2009/12/21/how-good-are-website-reputation-services/ http://www.stopthehacker.com/2009/12/21/how-good-are-website-reputation-services/#comments Tue, 22 Dec 2009 01:17:13 +0000 anirban http://www.stopthehacker.com/?p=859 Websites on the Internet have now become the standard modus operandi for spreading malicious software to infect personal and corporate environments. A large number of benign and well-meaning websites are compromised everyday by hackers inserting malicious code to, in turn, infect the computers used by visitors to the hacked site. One of the ways to combat this is to develop a website reputation mechanism which can warn of potential threats before visiting a compromised site.

Website-reputation services vary wildly in their opinions

Website-reputation services vary wildly in their opinions.

Note that all 350 domains, were reported as malicious, and were collected from malware.com.br on December 18, 2009. The blue column (maximum 350) indicates the number of sites that the website-reputation service correctly identified reported bad sites. The orange column (maximum 350) indicates the number of sites that the website-reputation services incorrectly identified reported malicious sites as safe.

Website reputation services have been around for nearly 5-7 years now. Initially developing as a niche product line which could serve to provide an opinion of a site’s reputation to full fledged offerings which provide advisories about websites, whether they are distributing malware, and if they are, what kind, and using which Autonomous Systems.

At StopTheHacker.com (Jaal LLC) we have conducted tests with 350 domain names, all of which have been reported as malicious by volunteers of various blacklists.

The aim of the test is to:

  1. Identify how accurate the website reputation services are
  2. What is the overlap in terms of safe/unsafe websites

We have found some interesting results which we present in this article. First we detail the parameters of the testing procedure to provide an idea of how the test was set up.

350 URLs were collected from malware.com.br (mbr) on December 18, 2009. These URLs are reported to this website for listing by one or more of the following: individuals, organizations, agencies and software products or services.  We assume for the purposes of this test that all the URLs obtained from the “regular” list from mbr are malicious and hence deemed “unsafe” to visit.

We compare the reputation provided by each website-reputation service and observe how many websites are marked as unsafe, safe, untested, maybe-unsafe/caution/potentially-unsafe, unreachable.

Note, that when analyzing a domain name, for checking with the Google safebrowsing API, we have had to calculate the MD5 hashes of the website names to match with the malware hash list. The date that we conducted this test was: December 21, 2009. The list of domain names tested are presented below and a graph representing the statistics for the first 350 sites tested is above.

We have identified some of the most interesting results below:

  1. McAfee Siteadvisor marked 32.5% of Domains as Unsafe, 22% as Safe, 43% as Untested and 1.7% as Potentially-unsafe.
  2. Norton Safeweb marked 50.86% of Domains as Unsafe, 43.71% as Safe, 2.29% as Untested and 3.14% as Potentially-unsafe.
  3. Google SafeBrowsing marked 10.86% of Domains as Unsafe, 89.14% as Safe. Note: the presence of the hash of the domain name  being tested, on the google malware hash list, is interpreted as “unsafe” while the absence in interpreted as “safe”.
  4. Comodo Siteinspector marked 0.29% of Domains as Unsafe, 98.86% as Safe and 0.86% as Unreachable. Note: after feedback from Comodo, a retest was conducted, accuracy changed from 0.29% -> 1.2%.

This limited test is a first step towards showing how much variance there is website reputation services that are currently being offered by large Internet-services/security companies. To highlight this point we present immediately below the relatively few domains (~6% of the total domains tested) that were marked as bad by all three major services, Norton, McAfee, and Google.

In brief:

  • 6% of domains tested were marked as “unsafe” by all 3, McAfee, Norton and Google
  • 10% of domains tested were marked as “unsafe” by Norton and Google
  • 22% of domains tested were marked as “unsafe” by Norton and McAfee
  • 5.7% of domains tested were marked as “unsafe” by Google and McAfee

Update: December 28, 2009

After receiving helpful feedback from representatives at Comodo, we were informed that Comodo’s service could provide more accurate answers if complete web page locations were checked instead of just the domain name. We followed the advice and saw a definite increase in Comodo’s accuracy. Comodo marked 1.2% of the website/pages as malicious. Prior to this re-test, the same service marked 0.2% of the websites as unsafe. The graph at the beginning of this article does not represent the results of this re-test.

Below we list the websites from which we extract the statistics above

Websites marked as “unsafe” by Norton, McAfee and Google

219.148.34.10
219.148.34.9
4gameranking.com
77.245.61.232
aiongamemeca.com
durantilumi1cao.com.br
golary.cn
hagnuor.cn
igivor.cn
igoudix.cn
igouhxe.cn
ihaegup.cn
ihaerxi.cn
ihagoin.cn
ihoekag.cn
ihouvi.cn
ihuere.cn
ihuqoyr.cn
ijaheuw.cn
ikyigy.cn
iloefe.cn

Websites marked as “unsafe” by Google and Norton

212.99.87.130
219.148.34.10
219.148.34.9
4gameranking.com
61.164.108.213
77.245.61.232
aimeblog.com
aiongamemeca.com
bhactuant.com
durantilumi1cao.com.br
findreaso1ble.org
for23.3322.org
golary.cn
gyfvuxe.cn
hagnuor.cn
ifueme.cn
igivor.cn
igoudix.cn
igouhxe.cn
iguyzmo.cn
ihaegup.cn
ihaerxi.cn
ihagoin.cn
ihoekag.cn
ihogedi.cn
ihouvi.cn
ihuere.cn
ihuqoyr.cn
ijaheuw.cn
ijakony.cn
ijazofy.cn
ijeife.cn
ijelodi.cn
ikyigy.cn
iloefe.cn

Websites marked as “unsafe” by McAfee and Google

219.148.34.10
219.148.34.9
4gameranking.com
77.245.61.232
aiongamemeca.com
durantilumi1cao.com.br
emes.com.br
golary.cn
hagnuor.cn
igivor.cn
igoudix.cn
igouhxe.cn
ihaegup.cn
ihaerxi.cn
ihagoin.cn
ihoekag.cn
ihouvi.cn
ihuere.cn
ihuqoyr.cn
ijaheuw.cn
ikyigy.cn
iloefe.cn

Websites marked as “unsafe” by McAfee and Norton

163.fuckunion.com
206.161.127.72
208.75.230.43
209.205.196.16
218.93.205.250
219.148.34.10
219.148.34.9
4gameranking.com
61.235.117.72
70.148.212.252
77.245.61.232
82.98.235.173
85.92.157.141
91.213.126.100
97feihu.com
adobeflashupdates.com
adwareprotectionsite.com
aiongamemeca.com
amforum.lua.pl
antivirus-live.com
artistinove.it
centralspa.ca
comerciocentral.net
densmail.com
diadoamigo0.myartsonline.com
dimorphothec.com
dl.get-torrent.com
dl.targetsaver.com
dudi11.off.co.il
durantilumi1cao.com.br
ebestsite.co.kr
elogios0.myartsonline.com
exeype.cn
fuck-celebrities-movie.com
gclass.it
generalantivirus.com
ghterwa.com
gokzed.cn
golary.cn
google.netcdn.com
gorazyn.cn
hagnuor.cn
hahdyti.cn
hgtr3.com
hiqtacy.cn
hjyuw2.com
icepot.cn
idoafy.cn
idoape.cn
igafep.cn
igakuot.cn
igeuvat.cn
igivor.cn
igoudix.cn
igouhxe.cn
igycoat.cn
ihaegup.cn
ihaerxi.cn
ihagoin.cn
ihoekag.cn
ihouvi.cn
ihuere.cn
ihuqoyr.cn
ijaheuw.cn
ijepiyq.cn
ijesiam.cn
ijobuaw.cn
ijuebka.cn
ikoiwe.cn
ikorate.cn
ikuaxge.cn
ikyadeh.cn
ikyigy.cn
ileufby.cn
ilixyeq.cn
ilodux.cn
iloefe.cn
iluefot.cn
i1gyve.cn

Interestingly, Comodo’s service marked only 1 website, 218.146.255.156 as malicious. This domain was also marked malicious by Norton, “Untested” by McAfee and was not found on the Google malware hash list. Below follows the complete list of domains that were tested.

Complete list of domains tested

001.bbexe.cn
113.105.175.138
114.207.112.169
119.147.114.163
12.10.157.6
12.24.238.229
12.25.151.68
121.12.127.230
121.205.91.142
121.205.91.145
123.244.30.118
123.244.30.66
123.bbexe.cn
147.163.1.77
148.208.196.2
163.fuckunion.com
174.36.233.59
192.220.110.228
193.104.27.139
193.169.234.27
200.111.155.122
200.242.43.250
200.63.5.78
200.67.103.187
200.69.124.17
202.105.183.104
202.114.181.5
204.12.43.43
204.232.131.12
206.161.127.72
208.75.230.43
209.131.200.246
209.172.35.144
209.205.196.16
209.43.123.143
210.166.220.240
210.206.8.254
210.51.166.217
211.39.130.196
211.78.87.42
212.31.234.155
212.63.132.215
212.88.178.22
212.97.63.156
212.99.87.130
216.24.165.4
216.240.148.175
217.116.46.139
218.146.255.156
218.16.120.253
218.188.0.5
218.6.15.135
218.63.200.196
218.86.118.98
218.93.202.115
218.93.205.250
219.146.128.242
219.146.128.245
219.148.34.10
219.148.34.9
220.90.213.158
220.95.232.28
221.1.204.243
221.143.43.200
222.66.209.98
222.76.243.53
24.1188d.cn
24.65.70.52
3.1188d.cn
3310.net.cn
38.99.91.47
3s.8i9i.com
46.1188d.cn
46.3388a.cn
4gameranking.com
5.1188d.cn
53.1188d.cn
58.147.27.69
58.215.79.176
6.1188d.cn
60.191.39.6
61.108.173.3
61.110.21.192
61.164.108.213
61.235.117.72
62.193.229.83
64.160.216.20
65.109.240.130
65.183.178.92
66.116.229.233
66.152.93.119
66.220.17.157
66.45.235.228
67.19.9.234
67.43.224.77
68.153.57.9
70.148.212.252
72.10.166.195
72.20.6.106
72.237.212.57
72.35.84.6
72.64.146.16
731273265.520815.com
76.162.68.70
76.73.42.43
77.245.61.232
77.92.158.122
78.159.127.254
78.46.151.179
80.153.182.80
81.223.40.244
81.252.31.148
82.114.87.46
82.98.235.173
83.103.59.84
83.206.113.161
83.240.174.136
83.245.62.87
84.20.251.223
85.17.136.139
85.25.81.140
85.92.157.141
91.207.7.116
91.213.126.100
93.174.95.140
95.211.98.136
97feihu.com
98.126.34.250
a.amg777.com
a1964.g.akamai.net
absi2008.netfirms.com
acripino7878.110mb.com
admin.bbexe.cn
adobeflashupdates.com
adwareprotectionsite.com
aha-autoimage.com
aimeblog.com
aiongamemeca.com
album.pagi1s.sapo.pt
alison.wz.cz
alkeichah.com
amforum.lua.pl
amoravela.com.sapo.pt
antivirus-live.com
antivirusadvanced.com
arathas.de
arcade.ya.com
arkbroadcasters.org
artdeli.co.kr
artistinove.it
atencaousuario.webcindario.com
atualizaca-juridica.sitesled.com
ausamedia.berepublic.com
avr-download.com
b.amg777.com
backstaroup.home.sapo.pt
bb.bbexe.cn
bbs.pxtang.cn
bcfpb.com
bchokies.com
bdesata.com
belezademulher.org
best-sale.us
bevaccine.com
bgcomstock.com
bhactuant.com
blog20fc2.com
blogaofotos8.com.sapo.pt
blogfotos2008.com.sapo.pt
blogpesoalpessoal.com.sapo.pt
bmz.horizon.net.pl
brasilterra.com.sapo.pt
c.amg777.com
caixa-cefinstall.sitesled.com
caixaeconomica-gov.sitesled.com
cancelamentt0.googlepages.com
carbys.no.sapo.pt
card2009.com.sapo.pt
cardamorhtml.no.sapo.pt
cardpaixao.esmartdesign.com
cartao8578.com.sapo.pt
cartaoamizade000.com.sapo.pt
cartaoespecial9.com.sapo.pt
cartaovirtual2006.no.sapo.pt
cartoesnovos.250x.com
cartoesuol.com.sapo.pt
cartoeswebapaxo1do.no.sapo.pt
casasbahia.com.sapo.pt
cau.ac.kr
centralspa.ca
chaiyapruekpethospital.com
chamadavideo-1.my3gb.com
chi1oilfactory.cn
chinesefreewebs.com
ciduninstall.com
cinema-film-4you.ru
club.telepolis.com
comerciocentral.net
comunidade777.110mb.com
config.koreamessenger.com
correiosweb.com.sapo.pt
cprzafra.juntaextremadura.net
d.amg777.com
d.kkkmfdy.com
d4.kkkmfdy.com
damnkt.logi1pp.com
db.ms.kr
denizlisurucukursu.com.tr
densmail.com
diadoamigo0.myartsonline.com
dimorphothec.com
di1r-cs.real-host.ru
dindindopv.bravehost.com
ditto.arpa.org
dl.get-torrent.com
dl.qvodir.cn
dl.targetsaver.com
dl.woyo8g.com
dl02.softdown-load.com.cn
dollardream.ru
donghae.ms.kr
dorota288.w8w.pl
down.1vysoft.org
down.woyo8g.com
down.yellowsoft.org
download.gameztar.com
download.iobit.com
download.leeboo.com
download.softpedia.com
downlopaginvisualiz.com.sapo.pt
dtvprosoft.hotbox.ru
dudi11.off.co.il
durantilumi1cao.com.br
dw.idchecker.co.kr
dx.woyo8g.com
e-airkoryo.com
e.amg777.com
ebestsite.co.kr
edirrelojoeiro.com.br
elogios0.myartsonline.com
emes.com.br
empresarial0001.pisem.su
energy-sol.com
exeype.cn
extex-events.ru
f-forge.com
fhblack.com
fideizm.ru
fileanchor.com
findreaso1ble.org
flashplaginsmirror.com
flashplayer.home.sapo.pt
fondbaybakova.ru
for23.3322.org
forrodotchaka.com.br
forum.factor8guild.com
fotoalbumbr.flog.br
fotoemsg.110mb.com
fotosbalada10x.fileave.com
fotoslinks439856.com.sapo.pt
franciszkankiswklary.ofm.pl
freefilehosting.net
freeweb.siol.net
fuck-celebrities-movie.com
galeon.com
gclass.it
generalantivirus.com
ghterwa.com
gizemguvenfa1tikleri.googlepages.com
glla.net
gokzed.cn
golary.cn
goldeninka.ii1a.net
google.netcdn.com
gorazyn.cn
govsaude.110mb.com
grwww.info
gtpq.info
gtz-legalproject.az
gyfsanimados2009.com.sapo.pt
gyfvuxe.cn
gymarqe.cn
hagnuor.cn
hahdyti.cn
haimadhav.googlepages.com
hakaymobilya.com
hgtr3.com
hiqtacy.cn
hjwx3.com
hjyuw2.com
hohu.spacequadrat.de
homecards11.no.sapo.pt
hosting.free2w.com
hotmailtorpedos2008.com.sapo.pt
humano.ya.com
icepot.cn
idfc2.info
idoafy.cn
idoape.cn
ies.bbexe.cn
ifueme.cn
ifypeod.cn
igafep.cn
igakuot.cn
igayzde.cn
igeuvat.cn
igivor.cn
igoudix.cn
igouhxe.cn
iguyzmo.cn
igycoat.cn
ihaegup.cn
ihaerxi.cn
ihagoin.cn
ihoekag.cn
ihogedi.cn
ihouvi.cn
ihuere.cn
ihuqoyr.cn
ijaheuw.cn
ijakony.cn
ijazofy.cn
ijeife.cn
ijelodi.cn
ijepiyq.cn
ijesiam.cn
ijobuaw.cn
ijuebka.cn
ijyadpi.cn
ijyoxri.cn
ikayvo.cn
ikeuqe.cn
ikeysi.cn
ikioda.cn
ikoiwe.cn
ikorate.cn
ikuaxge.cn
ikyadeh.cn
ikyigy.cn
ildapadilha.110mb.com
ileufby.cn
ilipyw.cn
ilixyeq.cn
ilodux.cn
iloefe.cn
iluefot.cn
img242.imageshack.us
img503.imageshack.us
img522.imageshack.us
i1gyve.cn
]]> http://www.stopthehacker.com/2009/12/21/how-good-are-website-reputation-services/feed/ 0