Moving from Threat Hunting to Threat Catching

The goal of a Threat Hunter is to find an attacker in the middle of an attack before they can cause damage. This entails hunting through thousands of requests trying to pick out the malicious telemetry emanating from thousands of endpoints that looks like hundreds of different users. The task becomes exponentially more difficult as the volumes increase. As an example, one of our customers was a recent target of an attack that reached 50,000 requests per minute. If we assume that 10% of the traffic is attack-related, the Threat Hunter has to perform the near-impossible task of analyzing 5,000 requests per minute. In this scenario, the term “hunting” is apt as it implies the potential for going home empty-handed.

The only way that a threat hunting team can keep pace with automated attacks is to either add more resources (people), or use automation, much like the attackers do. Automated threat hunting (analytics) tools detect when the traffic is valid and when it isn’t by analyzing hundreds of signals within the application requests such as:

• Which users are compromised or otherwise problematic
• IP addresses the attacker may be using from a proxy network
• Whether or not the user agent or browser is valid
• Attack behavioral factors such as rotating users, IP addresses, user agents, cookies, headers, and anything else that an attacker will use to make the traffic seem legitimate

Using automation to sort through 50,000 requests per minute will only go so far, perhaps surfacing users coming from too many IPs originating from Bulletproof Proxy Networks or an individual user with 55 different browsers. But remember we have to find 5,000 per minute.

From Hunting to Catching

Enter the concepts of Machine Learning and Artificial Intelligence, the foundational elements of our CQAI analytics engine, which enables Threat Hunters to move from hunting to catching. Our standard onboarding process is to configure CQAI to receive and analyze traffic from the applications a customer needs to protect.

No custom code or development efforts are required; CQAI needs to see and analyze either all or some of the traffic. The next step is to begin “training” the ML models with interesting elements inside the traffic. We can point to login flows and identify good and bad login behaviors. We can point to standard requests to train the models to understand where there should be uniqueness and where there won’t be. As CQAI analyzes the traffic, the attacks begin to surface, based on predefined ML models. In many cases, the ML models don’t need training as the defined characteristics being “hunted” are well understood. This means the process of analyzing traffic with CQAI entails both automatic identification of threats as well as “training” to ensure maximum efficacy over time.

The next step is to translate CQAI threat findings into mitigation policies based on a wide range of factors. Stacking policies and enabling mitigation allows API Spartan to take simple steps like marking the traffic for the customer to further analyze. We can block the traffic if desired or we can be a bit more mischievous and trap the traffic in a honey pot and let the attacker think they have succeeded.

Once CQAI and API Spartan are configured and running, the act of Threat Hunting transitions to catching with periodic tuning as attackers retool and watching for anomalous signals that indicate attacker traffic. Often times, we will be challenged with the question of how can you prevent bots when you are not integrating JavaScript or an SDK into the web or mobile app? The answer lies in the power of the 150+ customizable ML models built into CQAI, which allows a minimally staffed team to easily keep pace with the attack example above of 50,000 requests per minute.