PCR Tech and IT retail, distribution and vendor news
Andy Still, CTO, Netacea discusses how hackers are taking advantage of websites that use JavaScript.
We have a bot problem. At least half of all web traffic is automated, and some of this traffic is buying our gig tickets, our sneakers, our plane tickets and our games consoles before the rest of us have a chance.
This has been a problem for some time, but the combination of national lockdowns and some high-profile launches has seen it become a major news story. The lack of availability of Nintendo Switch consoles last year, followed by PlayStation 5 and graphics cards going out of stock as soon as they arrive, have all been blamed on bots. In the UK, there have even been moves in parliament to try and protect consumers by outlawing the use of “scalper bots” to buy in-demand goods.
Ecommerce platforms are understandably keen to limit the use of bots on their sites for a number of reasons. Most obviously, if bots are snapping up consumer goods before real consumers can, that means losing sales—while real people can be upsold and are worth building a relationship with, all bots are interested in is buying what they came for. Regular customers will be disappointed by the lack of stock, damaging any existing relationship.
But the problems can be more subtle. If retailers cannot differentiate between bot traffic and real traffic, they’re likely to make poor decisions based on this tainted data. This is true of not just ecommerce sites, but any business with a customer-facing web presence, whether that’s financial service providers, travel brokers, or online gaming. Bots may, for example, increase bounce rate, making a business think that what they are offering isn’t compelling enough. Or their inclusion in the stats may mean that conversion rates are way down, making businesses think there is a problem where none exists.
Any business that is vulnerable to bot attacks will be keen to understand the traffic on its site better. But traditional ways of doing this are becoming less useful as bots become more sophisticated.
Tackling bots with JavaScript-based solutions
One of the hurdles any traffic analytics solution faces is integration—how can the solution get access to the data it needs to understand the traffic on the site and allow the business to make good decisions? Not all bots are bad, of course, with some being crucial to search engine discovery, so the solution is not as simple as banning all bots from the site.
One of the simplest ways of doing this is to insert snippets of JavaScript into each page. That piece of code collects signals from the user, providing information on where they are browsing from, how long they spend on each page, mouse movements, button clicks and many other pieces of behaviour that together helps to build a profile and complete picture of the user.
JavaScript integration has the advantage of being simple to integrate and collecting a wealth of information about those visiting a site. They work a little like a polygraph or, if we’re indulging in a little sci-fi, like Blade Runner’s Voight-Kampff test to distinguish between bots and real people. When you tick a box marked “I am not a robot”, it is not the act of ticking the box that proves you are real, but the other signals being collected about your browsing.
Unfortunately, the big disadvantage of this method of bot mitigation is that, more and more, it is failing to keep out bots.
The rise of sophisticated bots
Bot attacks and bot mitigation is an arms race. We are continually developing tools to identify and keep out bad bots, but the enemy is a moving target. The idea that hackers are kids in their parents’ basement is now decades out of date, but we find that many people don’t quite realise the extent to which hacker groups have professionalised their service. They are constantly updating their software to handle the best that security vendors can throw at them, just as the vendors do with their software. Many advertise their bots as being able to deal with the bot mitigation present on popular sites.
The problem with JavaScript-based solutions is that those code snippets are little calling cards, letting the bot creator and bot user know exactly what tricks you have up your sleeve waiting for them. Any good heist movie has the scene where the gang gets their inside information – details on the locks they’ll need to bypass, the electronic surveillance they’ll need to slip by, the safe that needs to be cracked. But JavaScript means there’s no need for an inside man – there’s a big neon sign telling anyone exactly what they’re up against. Bot operators don’t have to figure out how they can try and bypass the mitigation in place, instead they just need to browse a bot marketplace for the right tool for the job.
Does this mean that JavaScript is no longer useful? Not quite
So, what’s the alternative? Rather than integrating through JavaScript, server-side solutions should instead be implemented via cloud, CDN or API.
As server-side bot management does not expose code, this means that attackers no longer know exactly what they are facing—bot operators have no visibility of bot identification methods and cannot reverse engineer a way around the solution. As well as full visibility of web traffic, mobile and API traffic is monitored too. The solution is easily maintained by the vendor, meaning customers will always automatically have the latest protection. No site-wide JavaScript updates are needed.
So, it’s simple, right? Ditch the old JavaScript methods of bot mitigation and use other server-side methods? We shouldn’t be quite so hasty. JavaScript still has a place in bot mitigation, though it may become less useful over time. JavaScript can still be used to collect a wealth of information on how the user is interacting with the site – the button clicks, how the user is scrolling, the path the user is taking through the site and more. These signals, while they can be subverted by a sophisticated bot, can also be used to gain a better understanding of the user and in combination with the server-side solution, give a clearer picture of who is using the site – not just a simple Voight-Kampff test of “bot or not”, but of intent: good bot or bad bot?
Unfortunately, JavaScript may become increasingly less useful. Major browsers are phasing out fingerprinting, a detection technique that JavaScript-based bot mitigation relies on that is seen as a privacy nightmare, and also a notorious attack vector for other hackers, and so Chrome, Safari and Firefox are all changing how they work.
This means that there is no long-term future for JavaScript-based bot mitigation alone. The bot creators won’t be happy, as it means many businesses will have to find alternative solutions, effectively removing the help they were giving those who were reverse engineering their mitigation techniques. But this doesn’t mean bot mitigation has won. Instead, it’s just a new phase in the arms race.
Read the latest edition of PCR’s monthly magazine below:
Like this content? Sign up for the free PCR Daily Digest email service to get the latest tech news straight to your inbox. You can also follow PCR on Twitter and Facebook.
