URL shortening services have become all the rage on the Internet. These services take a long URL as input and produce a short, easy to use, URL as an output. Simple! By virtue of their ease of use, millions of Internet surfers use them to post messages on twitter. In fact, URL Shortening services like bit.ly have garnered so much attention that even giants like Google and Microsoft have jumped onto the URL shortening bandwagon.
Case in point:
These URL shortening services are godsend for Internet surfers tired of copying and pasting long, ugly looking, URLs. But hold on a minute! All is not hunky dory in URL Shortening Land.
Due to processes inherent to “URL Shortening,” the original URL an Internet surfer might like to shorten is, for all purposes, being obfuscated. Is this a problem? Yes. Why, you ask? Consider the fact that people, not even necessarily tech-savvy ones, have learned to double check the links present in their emails and on websites. They even have help from various browser plugins, but in general, users are smartening up. When these same people see “shortened” links, they have no way to make a judgment call on whether visiting the link is safe, or not. For example, you may recognize www.stopthehacker.com as being a benign, safe to visit link, but what about bit.ly/oJMrP or bit.ly/dc38ze?
Articles published from credible sources, like ISC SANS, show that URL shortening services, when compromised, can provide an excellent mechanism for malicious hackers to infect unsuspecting visitors. Criminals use these services to bypass Google’s Safe Browsing service, which is used by popular browsers.
To combat this growing menace, URL shortening services have partnered with security companies to identify malicious URLs and websites. Some of them even use the SURBL blacklists to identify if someone has tried to link to a malicious website.
This article attempts to identify the effectiveness of security measures put in place by the various URL shortening services.
This experiment answers the following questions:
The 25 URL shortening services evaluated in this article are listed below:
We compare 25 URL shortening services listed below. Each URL shortening service is analyzed to measure the effectiveness of their security measures. We use a two stage process to evaluate the security implemented by each service.
snipr.com budurl.com bit.ly short.to twurl.nl chilp.it fon.gs ub0.cc snurl.com fwd4.me short.ie a.gd hurl.ws kl.am to.ly hex.io tr.im cli.gs urlborg.com is.gd sn.im ur1.ca tweetburner.com tinyurl.com snipurl.com
An initial corpus of 932 websites was obtained from Malware Patrol a well respected source of information about malware infected websites, which receives nearly 3,500,000 hits/month. This experiment was conducted between February 2nd and February 4th, 2010.
For each URL obtained from Malware Patrol, we attempt to create shortened URLs for each site domain and full URL using each of the 25 services.
We denote a service as Stage 1 Compliant if it appears to use a security service or blacklist to identify malicious domains and does not allow a user to create a shortened link to any infected domain. Does the URL shortening service allow a user to create a URL pointing to a malicious domain (e.g. http://www.badsite.dom)?
We denote a service as Stage 2 Compliant if it uses a security service or blacklist to identify malicious domains and does not allow a user to create a shortened link to any infected domain or malicious full URL hosted on that domain. Does the URL shortening service allow a user to create a URL pointing to a malicious link hosted on a malicious domain (e.g. http://www.badsite.dom/badfolder/badfile)?
We present the most interesting results in brief:
Observations on specific URL shortening services:
Stage 1 Compliant and Stage 2 Compliant services:
budurl.com cli.gs fon.gs hex.io is.gd kl.am sn.im snipr.com snipurl.com snurl.com to.ly tr.im ub0.cc
Deeper security issues remain:
It seems that popular services like bit.ly, which do try to use blacklists in order to prevent malicious hackers from using their services and pointing to bad websites, can still be easily fooled by chaining together shortened URLs created by another service. We have found that if a malicious user can create a shortened URL using a service that does not implement blacklist checks or is not effective, then a service like bit.ly can be tricked into redirecting the visitor via the malicious shortened URL to a malicious domain. Effectively, users can be redirected to a malicious site regardless of bit.ly performing all its checks. See the appendix for an example below (wget log).
This limited experiment shows that URL shortening services have a long way to go before Internet users can trust them to deliver safe links. About half of the most popular URL shortening services seem to be somewhat effective at blocking access to well known malicious URLs that can be found on blacklists. It remains to be seen if these URL shortening services can improve and provide a safer web experience for their users.
Wget log example:
In this example, a malicious link (hxxp://wywg.ccsfyb.cn/wywg/txer) has been shortened using ow.ly (hxxp://ow.ly/Zyv3). Then, this shortened URL is fed to bit.ly. The shortened bit.ly URL (hxxp://bit.ly/5s4YhP) is created successfully and blacklist checks are no longer effective.
$ wget -O demonstrate_bit.ly_exploit http://bit.ly/5s4YhP --scrubbed-- http://bit.ly/5s4YhP Resolving bit.ly... 18.104.22.168, 22.214.171.124, 126.96.36.199, ... Connecting to bit.ly|188.8.131.52|:80... connected. HTTP request sent, awaiting response... 301 Moved Location: http://ow.ly/Zyv3 [following] ---scrubbed-- http://ow.ly/Zyv3 Resolving ow.ly... 184.108.40.206 Connecting to ow.ly|220.127.116.11|:80... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: http://wywg.ccsfyb.cn/wywg/txer [following] ---scrubbed-- http://wywg.ccsfyb.cn/wywg/txer Resolving wywg.ccsfyb.cn... 18.104.22.168 Connecting to wywg.ccsfyb.cn|22.214.171.124|:80... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: http://wywg.ccsfyb.cn/wywg/txer/ [following] ---scrubbed-- http://wywg.ccsfyb.cn/wywg/txer/ Reusing existing connection to wywg.ccsfyb.cn:80. HTTP request sent, awaiting response... 403 Forbidden -scrubbed-- ERROR 403: Forbidden.