Using the power of Machine Learning to detect cyber attacks

By Chandni Naidu, Member Technical Staff, NetApp

As the world becomes increasingly digital, we are unlocking more value and growth than ever before. However, a challenge that governments, enterprises and well as individuals leveraging technology are constantly facing is the growing threat of cyberattacks that looms large over us.

Cyber security solutions provider SonicWall’s 2019 report revealed 10.52 billion malware attacks in 2018, a 217% increase in IoT attacks and 391,689 new variants of attack that were identified. What’s more is that cyber criminals today are evolving with technology and upping their game. Such incidents don’t just have the potential to bring businesses to a standstill but can also inflict serious damages to their resources and repute.

With an increasing number of cyberattacks targeting critical networked resources that cannot be detected by traditional network monitoring tools, it becomes critical to explore and leverage sophisticated tools for detection and reporting of such attacks.

Artificial Intelligence (AI) and Machine Learning (ML) are two of the hottest technology trends that have the potential to transform the modern security architecture landscape. Artificial intelligence is any technique that enables computers to mimic human behavior. Machine Learning is the ability to learn without being explicitly programmed. Both these techniques are widely used in various industries like healthcare, banking and storage.

DoS and DDoS attacks

In this blog, we explore an innovative approach to detect Denial of Service (DoS) and Distributed Denial of Service (DDoS) attacks – two of the major kinds of attacks plaguing organizations – using ML algorithm by mining application specific logs.

In a DoS attack, the hacker tries to prevent genuine users from using a website by maliciously flooding it with traffic, which can cause the system to crash. The origin of the attack is single, i.e. it is made from one computer or internet connection. With hackers getting more innovative, there are multiple ways of carrying out such attacks. Recently, a specially crafted MP4 file which was circulated on WhatsApp, triggered a DoS attack on individual users. Attackers can take advantage of this vulnerability to deploy malware on the user’s device to steal sensitive files and also use it for surveillance purposes.

When an attacker uses multiple machines to send requests with mischievous intent, trying to take over the target machine’s resources, it is a DDoS attack. In what is said to be one of the most powerful DDoS attacks, GitHub in 2018 received a staggering 1.35 terabits/second of traffic on a particular day for 18 minutes. GitHub, along with their DDoS mitigation service provider Akamai Prolexic, handled the situation and resolved it within 20 minutes.

The reasons for such attacks can be varied – from an intent to steal data or defame an enterprise to using it as a decoy to perform another high impactful attack.

Some of the most high profile cases include the DDoS attack on the Telegram messaging app which hampered its day-to-day communication. Large multinationals such as Paypal, Twitter and Spotify, with some of the most advanced security tools, have also been victims to similar attacks.

Machine Learning to tackle attacks

Today, enterprises across are using cloud to build and manage software. Microservices is a widely used software development technique and Application Program Interface (API) is a type of microservice used in various industries such as banking, storage and healthcare. Many instances of microservices automatically start when required. In such a situation, it is not possible for humans to monitor and check if all the instances are genuine. This presents a greater cyber-attack risk.

A system with APIs is designed to fulfill the assumption that each of the routines will be called only limited times per day and this can provide a viable solution to such attacks. But the number of calls might increase due to programmatic retries if the API fails to respond in a timely manner. Also, the number of API calls may increase in situations when debug or trouble-shooting procedures are performed. Even with trouble shooting., the maximum threshold is not expected to go beyond a defined number of calls per day.

Here, we can make a rudimentary assumption – that if an API call is invoked more than 100 times, then it may constitute a DoS/DDoS attack. The ML algorithm can then be trained using logging data to classify if the system is under attack based on certain attributes.

The logs generated by various microservices are continuously monitored using log monitoring tools such as Fluentd. Various attributes, such as client IP address, API request and date and time, are retrieved from the acquired log data.

This information can be fed into a preprocessor in real time, which calculates the number of hits on a certain API for a given date and time, and client IP address. There can be situations where multiple machines are used to attack multiple APIs exposed by a target. Every industry that uses API, especially applications that deal with sensitive information, can be impacted by DoS or DDoS attacks. These attacks are not just used for denying services to a consumer; an attacker can use it for sending malware with the intent of gathering sensitive data.

Machine Learning algorithms can be used to train and detect if there has been a DoS/DDoS attack. As soon as the attack is detected, an email notification can be sent to the security engineers. Any classification algorithm can be used to categorize if it is a DoS/DDoS attack or not. One example of a classification algorithm is Support Vector Machine (SVM) which is a supervised learning method that analyses data and recognizes patterns.

With increase in attacks, early detection is the best solution

According to data by cybersecurity firm Kaspersky, the number of DDoS attacks rose by a third in the third quarter of 2019. In its survey it observed that DDoS attacks are the second most expensive type of cyberattacks targeting small and medium sized businesses, and the average cost of such breaches is estimated to be $138,000.

With cybercrime mushrooming across the world, the players are not just limited to seasoned criminals, and traditional methods are giving way to sophisticated techniques.

Perhaps one of the strongest indicators of the escalation of such activities is the growth of the DoS/DDoS attack solution market, which is estimated to increase from $900 mn in 2019 to $9 billion in 2025.

The most recent DDoS attacks have been observed to hijack connected devices such as webcams, baby phones, routers, vacuum robots, etc. to launch their attacks.
The number of devices remotely controllable via apps is growing exponentially and the Internet of Things (IoT) is expected to easily surpass 20 billion connected devices by the end of 2020.

Current IoT system follows a centralized architecture that makes it more prone to DoS or DDoS attacks. Blockchain technology can be used to enable creation of IoT networks that are peer-to-peer (P2P) and trustless. This removes the possibility of centralized single point of failure. An attacker’s Command & Control server will not be able to gain access to publish the DDoS attack instructions because of the P2P network of blockchain.

Since we cannot control when, where or how an attack may come our way, and absolute prevention against these cannot be guaranteed yet, our best shot for now is early detection which will help mitigate the risk of irreparable damage such incidents can cause.

Organizations can use existing solutions or build their own to detect cyberattacks at a very early stage to minimize the impact. Any system that requires minimal human intervention would be ideal.