We have a bot problem. At least half of all web traffic is automated, and some of this traffic is buying our gig tickets, our sneakers, our plane tickets and our games consoles before the rest of us have a chance.
This has been a problem for some time, but the combination of national lockdowns and some high-profile launches has seen it become a major news story. The lack of availability of Nintendo Switch consoles last year, followed by PlayStation 5 and graphics cards going out of stock as soon as they arrive, have all been blamed on bots. In the UK, there have even been moves in parliament to try and protect consumers by outlawing the use of “scalper bots” to buy in-demand goods.
Ecommerce platforms are understandably keen to limit the use of bots on their sites for a number of reasons. Most obviously, if bots are snapping up consumer goods before real consumers can, that means losing sales—while real people can be upsold and are worth building a relationship with, all bots are interested in is buying what they came for. Regular customers will be disappointed by the lack of stock, damaging any existing relationship.
But the problems can be more subtle. If retailers cannot differentiate between bot traffic and real traffic, they’re likely to make poor decisions based on this tainted data. This is true of not just ecommerce sites, but any business with a customer-facing web presence, whether that’s financial service providers, travel brokers, or online gaming. Bots may, for example, increase bounce rate, making a business think that what they are offering isn’t compelling enough. Or their inclusion in the stats may mean that conversion rates are way down, making businesses think there is a problem where none exists.
Any business that is vulnerable to bot attacks will be keen to understand the traffic on its site better. But traditional ways of doing this are becoming less useful as bots become more sophisticated.
One of the hurdles any traffic analytics solution faces is integration—how can the solution get access to the data it needs to understand the traffic on the site and allow the business to make good decisions? Not all bots are bad, of course, with some being crucial to search engine discovery, so the solution is not as simple as banning all bots from the site.
Unfortunately, the big disadvantage of this method of bot mitigation is that, more and more, it is failing to keep out bots.
The rise of sophisticated bots
Bot attacks and bot mitigation is an arms race. We are continually developing tools to identify and keep out bad bots, but the enemy is a moving target. The idea that hackers are kids in their parents’ basement is now decades out of date, but we find that many people don’t quite realise the extent to which hacker groups have professionalised their service. They are constantly updating their software to handle the best that security vendors can throw at them, just as the vendors do with their software. Many advertise their bots as being able to deal with the bot mitigation present on popular sites.
Read the latest edition of PCR’s monthly magazine below: