- Posted December 16, 2013 by
Guess What? Most Visitors to Your Website Are Not Human
If the latest buzz on the news sites and blogosphere is correct, most of the visitors to this webpage will not be humans. No – this is not science fiction and we’re not talking about an alien invasion. We are talking about “bots” (short for “robots”) - automated software programs that are taking over the lion’s share of Internet traffic.
According to a new report from website security firm Incapsula, over 61% of Internet traffic is generated by bots. This represents a 10% increase in non-human traffic since their previous report in March 2012. So – from one human to another - is this something we should be worried about?
Not necessarily. This is because according to Incapsula the bulk of this growth can be attributed to bots that perform very friendly and helpful Internet housekeeping tasks, such as crawling websites, collecting analytical data and indexing content. These “good” bots (think Google) can perform such tedious tasks much faster and more efficiently than a human ever could. In fact, good bots have increased 55% over the past year.
In order to gather their data, Incapsula’s security researchers monitored 1.48 billion bot visits, spread across 20,000 of its clients’ websites in 249 countries. "It's a good representation of what's going on out there [in the Internet]," Marc Gaffan, co-founder of Incapsula, told ABC News. "It ranges [in size] from tiny personal sites to large healthcare, financial, and retail sites."
Incapsula sees two main reasons for the proliferation of good bot traffic. One is the increased activity and higher sampling rates of existing bots, while the other is the emergence of bots from companies specializing in search engine optimization (i.e., how websites and content get noticed on Google). This new type of bot can crawl sites at a rate of 30-50 daily visits or more.
Of course, there is also a darker side to bots, which are the tool of choice for spammers, hackers and other assorted bad guys. Some 30% of total bot traffic is of the malicious variety, including scrapers, hacking tools, spammers and impersonators.
While the relative percentage of malicious bots remains unchanged, there is a noticeable reduction in spam bot activity, which Incapsula attributes to Google’s recent updates of its search algorithm (aka Penguin), designed to discourage link spamming as an SEO tactic.
The “Impersonators” bot category bears watching, as it grew 8% since last year and, as the name implies, these malicious bots are particularly stealthy as they try to impersonate real users. According to Incapsula, this group of unclassified bots is at the technological pinnacle of the bot hierarchy, and may take the form of automated spy bots, human-like DDoS agents or a Trojan-activated barebone browser. What’s for sure is that they all have hostile intentions and are more than likely one of the reasons why there's been a noted increase in cyber attacks over the last year.
"The common denominator for this group is that all of its members are trying to assume someone else's identity. For example, they will say #that they're from Google, but they don't behave like the Google bots and they don't come from Google IP addresses,” explained Gaffan. “While the method and techniques may vary, the ultimate goal is always the same -- to infiltrate their way through a website's security measures.”
So the next time you look at your website traffic statistics on Google Analytics, keep in mind that most of your visitors are probably bots.