Automated Antagonists: The Quest for Better Bot Management

A New Approach to Bot Management

Bots are a part of life on the internet for today’s businesses. In some ways, the internet has made it easier for criminals to steal information or commit fraud – bots are used to automate attacks that would typically be performed manually in the real world. For example, while it’s trivial for a bot to test a large amount of credentials against a poorly secured API, that type of attack would be difficult or impossible in person – imagine a criminal standing there with a stack of fake driver’s licenses – “Does this one work? No? How about this one?” The good news is, just as bot attacks can be automated, we can automate much of the bot management as well.

Good Bots vs. Bad Bots

When you hear the word “bot,” you may think only of malicious automated code built to attack web applications and APIs, but there are good bots as well. Examples of good bots include:

• Search engine bots that crawl and index content on the internet
• Chatbots such as those you see on websites that help visitors answer common questions
• Site monitoring bots that track website uptime, performance, and errors
• AI bots that crawl approved content to train AI models
• Virtual assistant bots like Alexa or Siri, usually with natural language processing

When you’re securing your business, you need to be able to distinguish between the two because their traffic may look similar at first glance, but bad bots can cause serious problems. Here’s a partial list of some of the consequences of bad bots:

• Account takeovers (ATO)
• Fake account creation
• Content scraping
• Sensitive data exposure
• Gift card or loyalty program abuse
• Distributed denial of service (DDoS)
• Traffic spikes driving increased resource use and associated cloud costs

All of these consequences are potentially serious if not caught early and prevented.

Do Traditional Bot Management Techniques Work?

Bot attacks have evolved over the years to evade defenses. In many cases, the defenses that worked previously haven’t kept up with the times. Most organizations have existing security tools that used to help against bots, like web application firewalls (WAFs). WAFs are still a useful tool for protecting web applications, but attackers now often bypass the web application and target mobile clients or back-end APIs directly. Additionally, WAF attack prevention is mainly focused around blocking specific IPs, which attackers easily bypass by distributing attacks across a seemingly endless supply of different IP addresses, available cheaply through bulletproof proxy vendors.

Traditional bot mitigation solutions rely on JavaScript integrated into web applications or SDKs for mobile applications to track user signals such as clicks and navigation, but this has several drawbacks. These integrations require engineering work and ongoing regression testing, and those hurdles alone lead JavaScript-based solutions to be relegated to shelfware in many organizations. In addition, attackers can simply target the APIs or the mobile apps (which do not support JavaScript) directly, as with the WAF solution, bypassing the apps with JavaScript-based defenses. Another important disadvantage that isn’t immediately obvious is that integrating a vendor’s JavaScript can telegraph the defense – if attackers can see the JavaScript, it gives them an idea of what protective measures are in place and therefore how to avoid them.

Bot management techniques that rely on user signals for behavioral analysis also struggle to differentiate good bots from bad bots and can end up blocking all bots rather than just malicious. Behavioral analysis based on actual traffic and API transactions is needed to discern good vs. bad behavior, whether human or bot.

Traditional bot management techniques also struggle due to how much the scale has changed. For example, retailers used to be primarily brick and mortar locations with an online presence. Now, not only are almost all retailers online-first, but entire classes of businesses are online ONLY. The priority and the volume of online traffic, transactions – and therefore attacks – have skyrocketed. Traditional bot management solutions were just not designed to handle the scale.

Successful Bot Management Requirements

To be successful against today’s evolved attackers and their bot armies, organizations need a solution that meets the following four criteria:

• Easy to deploy – The solution needs to deploy in a manner consistent with the organization’s existing infrastructure (e.g. SaaS, on-premises, or hybrid) and it needs to be deployable in a reasonable amount of time without requiring re-engineering effort on existing applications.
• Comprehensive – It must protect ALL web application traffic, including mobile apps and direct API traffic. Solutions that have per-app integrations or only focus on offending IPs will necessarily miss things – organizations need a solution that takes the guesswork out of app protection.
• Effective – This seems like a no-brainer, but this is where the rubber meets the road. The solution must be able to accurately distinguish good bots from bad bots and detect attacks whether they are slow and low or large-scale brute force. The best solutions should be able to block natively, without requiring a third-party defensive solution such as a WAF or API gateway.
• Resilient – As we outlined earlier, attackers continue to evolve their techniques, and organizations need a solution that can evolve with them to identify and block new, novel attacks. This ability to identify attacks and track them through layers of deception is the secret sauce that means the investment you make in a preventative solution today will still be effective in the future.

Find the Right Bot Management Solution

As the internet has evolved and become the focus of where many organizations do business, naturally so have attacks. Finding the right solution is imperative, but a methodical consideration of the available options based on the requirements above will provide a strong foundation. If you’d like to give Cequence a try, contact us for a personalized demo.