New strains of malware are constantly threatening businesses and creating angst for IT. As cyber risks grow in both volume and sophistication, the tools used to find and eradicate them have to get smarter and scale better, too. Malware writers stay a step ahead of traditional security solutions, creating threats that behave differently from system to system, day to day. Some infections now disguise or even partially encrypt themselves so they don’t match known signatures in malware databases.
Newer malware types include adware, botnet loaders and ransomware. The latter growing in popularity in part because of the recent rise of cryptocurrency. Ransomware encrypts data and prevents access until a digital fee is paid within a specified time period.
A Brief History
The earliest threat prevention solutions were antivirus systems based on signature matching. A monitoring system would search for a match to a known malicious software signature. If detected, the system would alert IT and possibly quarantine or block the traffic. These systems turned out to be too narrow in discovery criteria to have a very high success rate at catching malicious software. Their effectiveness mostly limited to known and documented malware.
As Internet use became widespread, malware writing became popular. These early systems evolved to monitor more complex signatures and then added specific rules to supplement or replace those signatures. Rules combine program attributes and logic to identify the features that indicate a malicious file. But because the rules don’t apply to 100% of all situations, exceptions to the rules had to be created and entered into the system.
Malware researchers “weighted” some of these rules to indicate a higher level of suspicion and establish an overall level of risk for a sample. But these were typically too aggressive or not aggressive enough. And due to the increasing complexity and imperfectly calculated weighting, such anti-malware systems have been susceptible to missing threats and generating false-positive alerts about benign software.
A Constant Battle
In any networked organisation, the greater the traffic volumes, the greater the number of alerts. Many turn out to be dead ends, resulting in security personnel wasting time chasing irrelevant alerts and overlooking or ignoring genuine threats. There is so much data being generated now there aren’t enough people to investigate all the alerts. Rules-based systems are complicated to maintain and simply don’t work well enough to justify the effort required to keep them current.
So, traditional antivirus systems have difficulty detecting brand new threats and this will only get worse. A newer approach – one that is automated and can discover new risks – is needed.
Some vendors now overlay machine learning onto the rules-based systems. In systems that rely on weighted rules, the weights can now be better optimised by machine learning techniques than by human intuition or pen-and-paper statistics.
Machine learning integrated into security tools from their inception stands a stronger chance of finding and thwarting new attacks before they can cause significant harm. Such systems continually and dynamically learn what’s “normal” in software structure and behaviour, and network traffic patterns and usage. Millions of variables and data points can be analysed at once to identify abnormal behaviour that could indicate an attack.
How Machine Learning Works
The machine learning system is fed a large number of signals extracted from various sources. These range from network information to binary structure to runtime behaviour. It learns to weight these signals and map them to a severity/danger potential. This culminates in a decision to quarantine a system that is likely compromised. The nature and weighting of the used signals can be determined automatically by the system itself.
One advantage of this approach is that the more data that is fed into the system, the better it can distinguish malicious programs from benign ones. Rules that uniquely identify each malware family no longer have to be manually written. Instead, the system identifies specific useful signals generated from program structure and behaviour on the system and the network. Then uses the collected intelligence to separate benign software from malware.
Machine learning alone isn’t a magic bullet. But it fundamentally changes the security equation by dramatically improving the accuracy of malware detection and risk classification.
Of course, these techniques don’t completely obviate the need for human decision making. However, machine learning can perform the bulk of manual labour, scaling the knowledge of skilled human analysts to large data sizes and handling the complexity beyond human capabilities.
Looking to minimise the risk to your organisation? We partner with Juniper Networks to provide automated, scalable and secure network solutions that offer agility, performance and value.
‘Machine Learning is Critical to Cyber Security’ taken from Juniper Networks ‘Outsmarting Malware’ whitepaper.
This paper discusses a modern approach to cybersecurity that uses adaptable machine learning algorithms combined with several anti-malware technologies to find and foil advanced threats.