URL shortening services have become all the rage on the Internet. These services take a long URL as input and produce a short, easy to use, URL as an output. Simple! By virtue of their ease of use, millions of Internet surfers use them to post messages on twitter. In fact, URL Shortening services like bit.ly have garnered so much attention that even giants like Google and Microsoft have jumped onto the URL shortening bandwagon.
Case in point:
These URL shortening services are godsend for Internet surfers tired of copying and pasting long, ugly looking, URLs. But hold on a minute! All is not hunky dory in URL Shortening Land.
Due to processes inherent to “URL Shortening,” the original URL an Internet surfer might like to shorten is, for all purposes, being obfuscated. Is this a problem? Yes. Why, you ask? Consider the fact that people, not even necessarily tech-savvy ones, have learned to double check the links present in their emails and on websites. They even have help from various browser plugins, but in general, users are smartening up. When these same people see “shortened” links, they have no way to make a judgment call on whether visiting the link is safe, or not. For example, you may recognize www.stopthehacker.com as being a benign, safe to visit link, but what about bit.ly/oJMrP or bit.ly/dc38ze?
Articles published from credible sources, like ISC SANS, show that URL shortening services, when compromised, can provide an excellent mechanism for malicious hackers to infect unsuspecting visitors. Criminals use these services to bypass Google’s Safe Browsing service, which is used by popular browsers.
To combat this growing menace, URL shortening services have partnered with security companies to identify malicious URLs and websites. Some of them even use the SURBL blacklists to identify if someone has tried to link to a malicious website.
This article attempts to identify the effectiveness of security measures put in place by the various URL shortening services.
This experiment answers the following questions:
- Do URL shortening services have any kind of security measures in place?
- How effective are these security measures?
The 25 URL shortening services evaluated in this article are listed below:
We compare 25 URL shortening services listed below. Each URL shortening service is analyzed to measure the effectiveness of their security measures. We use a two stage process to evaluate the security implemented by each service.
snipr.com
budurl.com
bit.ly
short.to
twurl.nl
chilp.it
fon.gs
ub0.cc
snurl.com
fwd4.me
short.ie
a.gd
hurl.ws
kl.am
to.ly
hex.io
tr.im
cli.gs
urlborg.com
is.gd
sn.im
ur1.ca
tweetburner.com
tinyurl.com
snipurl.com
Experiment methodology:
An initial corpus of 932 websites was obtained from Malware Patrol a well respected source of information about malware infected websites, which receives nearly 3,500,000 hits/month. This experiment was conducted between February 2nd and February 4th, 2010.
For each URL obtained from Malware Patrol, we attempt to create shortened URLs for each site domain and full URL using each of the 25 services.
We denote a service as Stage 1 Compliant if it appears to use a security service or blacklist to identify malicious domains and does not allow a user to create a shortened link to any infected domain. Does the URL shortening service allow a user to create a URL pointing to a malicious domain (e.g. http://www.badsite.dom)?
We denote a service as Stage 2 Compliant if it uses a security service or blacklist to identify malicious domains and does not allow a user to create a shortened link to any infected domain or malicious full URL hosted on that domain. Does the URL shortening service allow a user to create a URL pointing to a malicious link hosted on a malicious domain (e.g. http://www.badsite.dom/badfolder/badfile)?
We present the most interesting results in brief:
- Approximately 68% of URL shortening services were Stage 1 Compliant.
- Approximately 56% of URL shortening services were exclusively Stage 2 Compliant.
- Approximately 52% of URL shortening services were both Stage 1 Compliant and Stage 2 Compliant (see graph below).
Observations on specific URL shortening services:
- bit.ly seems to favor blocking malicious domains rather than specific links.
- fwd4.me, hurl.ws and urlborg.com seem to favor blocking malicious links rather than specific domains.
- bit.ly failed to qualify as Stage 2 Compliant due to 0.5% of tested URLs.
- fwd4.me failed to qualify as Stage 1 Compliant due to 9.8% of tested URLs.
- hurl.ws failed to qualify as Stage 1 Compliant due to 0.3% of tested URLs.
- urlborg.com failed to qualify as Stage 1 Compliant due to 0.3% of tested URLs.
Venn Diagram depicting URL filtering capabilities of URL shortening services. Only about half of the most popular URL shortening services are effective at blocking malicious URLs.
Stage 1 Compliant and Stage 2 Compliant services:
budurl.com
cli.gs
fon.gs
hex.io
is.gd
kl.am
sn.im
snipr.com
snipurl.com
snurl.com
to.ly
tr.im
ub0.cc
Deeper security issues remain:
It seems that popular services like bit.ly, which do try to use blacklists in order to prevent malicious hackers from using their services and pointing to bad websites, can still be easily fooled by chaining together shortened URLs created by another service. We have found that if a malicious user can create a shortened URL using a service that does not implement blacklist checks or is not effective, then a service like bit.ly can be tricked into redirecting the visitor via the malicious shortened URL to a malicious domain. Effectively, users can be redirected to a malicious site regardless of bit.ly performing all its checks. See the appendix for an example below (wget log).
Conclusion:
This limited experiment shows that URL shortening services have a long way to go before Internet users can trust them to deliver safe links. About half of the most popular URL shortening services seem to be somewhat effective at blocking access to well known malicious URLs that can be found on blacklists. It remains to be seen if these URL shortening services can improve and provide a safer web experience for their users.
Read more…
News, Report, Security
bit.ly, malware, ow.ly, tinyurl, url shorteners
