Network traffic analysis for IR: statistical analysis
Introduction to Statistical Analysis
Statistical analysis is one of the three main categories of analysis that can be performed on network traffic data. It provides much more detailed analysis than a simple connection scan and takes a different approach to identifying potential indicators of compromise than event-based scanning.
Statistical analysis is generally oriented towards the detection of anomalies. Based on the wealth of information available to the analysis algorithm, it can make educated guesses about what should be considered “normal” versus what is “abnormal” or “abnormal”. Any deviation from the norm can be an indicator that something is going on, which makes statistical analysis ideal for helping a responder determine where their investigative efforts can be focused to maximize their likelihood of success.
Carrying out statistical analyzes
In order to respond successfully and quickly to a potential incident, cyber analysts must first know where to look for potential attack indicators. Data science is extremely good at identifying patterns and correlations from large amounts of data.
Statistical analysis uses data science tools and techniques. Data science is a very broad field and most incident responders do not have the background to be a data scientist.
However, even simple statistical analysis techniques can be extremely useful for incident response. Techniques such as clustering and stack analysis can be easily performed by anyone and can be extremely useful in drawing attention to data that may warrant further investigation.
Clustering is an unsupervised machine learning application where the developer does not provide any input to the algorithm to direct it to a certain solution. Instead, the developer provides the desired number of clusters that he thinks should exist in the dataset, and the algorithm generates what he thinks is the best allocation of data points to the clusters.
Several different clustering algorithms (Read more…)