Network traffic analysis tools take on a crucial new role

Gartner created its first market guide report on the network traffic analysis tools category in February 2019. These tools, the report explains, have features that allow them to detect suspicious traffic that other security tools missing. Gartner’s report includes a list of some vendors whose products have the following features deemed essential for this new type of security tool:

  • They work in a network packet captures or stream data — eg, NetFlow, sFlow, IPFIX — near real-time or better.
  • They use behavioral network techniques like machine learning (as opposed to signature-based detection).
  • They operate in the detection phase – not forensics.
  • They analyze both north-south and east-west traffic and report suspicious events.

Some restrictive exclusion criteria were also the basis of older generations of security tools, such as working from log files and primarily using rules and signatures.

How network traffic analysis tools work

Network traffic analysis security can be achieved in different ways. Machine learning, a simple form of AI, is one of the key components of network traffic analysis tools because of the way machine learning automates the response to a perceived problem. It is essential that network traffic analysis tools operate in real time, or near real time. New forms of malware can spread very quickly through an organization, following both north-south (client-to-server) and east-west (server-to-server) network traffic paths. Automated traffic analysis and action is necessary to prevent the rapid spread of new malware.

Some organizations are still skeptical automated response systems, preferring to use manual methods to react to security events. However, rapid ransomware distribution is faster than manual methods can do. This fact will eventually force all organizations to move to fully automated response and remediation systems. An adaptable artificial intelligence or machine learning mechanism in any network traffic analysis tool is preferable because variations in malware attack and distribution mechanisms make a heuristic-based system quickly becomes obsolete.

Cyber ​​Defense Matrix

Tools that perform network traffic analysis cover a range of functions. One way to understand the capabilities of these tools is to use the Cyber ​​Defense Matrix overlay products and visualize network security systems. The relevant security function is represented on the horizontal axis of the matrix and the relevant infrastructure component is displayed on the vertical axis. The red outlines in the table below indicate general areas that are handled by tools that perform network traffic analysis.

This graphic, created by Sounil Yu, based on the NIST Cybersecurity Framework and the OWASP Cyber ​​Defense Matrix, provides a practical framework for understanding how network traffic analysis tools work.

Selecting a network traffic analysis tool

There are several things to consider as you learn network traffic analysis tools. How does each tool monitor traffic? Does it need full packet capture or can it operate from flow data, such as NetFlow, sFlow or IPFIX? How many collection points do you need? Some networks have a centralized topology where there are a few key traffic aggregation points, while other network topologies will be more distributed and require many traffic collection points. Determine if network packet brokers or other technology will be needed to collect the necessary traffic information and direct it to the network traffic analysis system. A the network packet broker is working to aggregate data streams and then distribute them to relevant tools (e.g. security tools or monitoring tools). This improves visibility and coverage in the data center.

You need to know if you will need to provide feedback to the network traffic analysis system’s machine learning algorithm. In some cases, you may need to confirm that an anomaly was correctly identified or that an anomaly reported was a false positive. Make sure you understand the level of expertise and the amount of effort you will need to put in for a successful network traffic analysis tool deployment. Of course, you will also need to check the price, including the effort required to implement an open source network traffic analysis tool.

The Gartner report includes a long list of commercial products. Open source tools can offer an alternative if you’re willing to invest a lot of time to understand, install, and maintain them. Note that many open source network traffic analysis systems are a collection of several tools that collect, index, and store traffic data and perform analysis on it. A system that appears to meet Gartner’s criteria is meerkat. the Quick response from Google The incident response framework may be another possibility, if you have a very large infrastructure, although it may not exactly fit the criteria of a network traffic analysis system.

Future directions

The increase in computing power and the decrease in its cost have brought advanced technologies such as machine learning within reach of many applications. Network traffic analysis is one such application. There are also a number of open-source machine learning platforms, allowing companies to incorporate some form of machine learning without having to create their own software from scratch. Expect to see better results as the technology matures, including the development of more open source network traffic analysis tools.

Charles J. Kaplan