How To Deploy Malcolm Network Traffic Analysis Tool With Ubuntu Server 22.04

Jack Wallen walks you through the steps of deploying a powerful and easy-to-use network analysis tool on Ubuntu Server 22.04.

Image: Gorodenkoff/Adobe Stock

Malcolm is an open source network traffic analysis tool that uses a framework of tools to create a robust analysis tool for network administrators. Malcolm accepts network traffic data in the form of PCAP (full packet capture) files and Zeek logs.

SEE: Recruitment Kit: Network Engineer (TechRepublic Premium)

Malcolm includes two different interfaces:

  • OpenSearch Dashboards: A flexible data visualization plugin with dozens of pre-built dashboards.
  • Arkime: A powerful tool to find and identify network sessions consisting of suspected security incidents.

Malcolm is easy to use, containerized, secure and in very active development. I want to walk you through the process of deploying this tool on Ubuntu Server 22.04.

What you will need for Malcolm

To get Malcolm up and running, you will need an Ubuntu Server 22.10 instance and a user with sudo privileges. That’s it: let’s get to work.

How to create a new user

The first thing we are going to do is create a new user. SSH or login to your Ubuntu Server instance and run the command:

sudo useradd -m -d /opt/malcolm -s /bin/bash -G sudo malcolm

Change the new user’s password with:

sudo passwd malcolm

Log in as this user with:

su - malcolm

How to Clone Malcolm and Run the Installer

Using git, clone the latest version of Malcolm with:

git clone https://github.com/idaholab/Malcolm

Switch to the newly created directory with:

cd Malcolm

Run the installer with:

sudo ./scripts/install.py

During this first stage of the installation, you will be asked a few questions. For each Y/N ​​question, answer with Y. The only non-Y/N question is:

Enter user account:

To this, answer with:

malcolm

How to configure Malcom

Once you have answered the installer questions, you need to configure Malcolm. Open the configuration file with:

sudo ./scripts/install.py –configure

Again, you will be asked several questions. Here are the questions and the answers you need to give:

  • The Malcolm processes will run as UID 1000 and GID 1000. Is this OK? (We) : Y
  • Setting 10g for OpenSearch and 3g for Logstash. Is it correct? yes
  • Definition of 3 workers for Logstash pipelines. Is it correct? (We) : yes
  • Restart Malcolm when the system or Docker daemon restarts: Yes – be sure to choose the default option, unless stopped.
  • Choose to configure Malcolm with HTTPS: Yes
  • Choose whether Malcolm will run behind any proxy: No
  • Choose networking: press Enter
  • Choose LDAP: No
  • Store OpenSearch index snapshosts locally in /opt/malcolm/Malcom/opensearch-backup? Yes
  • Choose to compress OpenSearch index snapshots: Yes
  • Automatically scan all PCAP files with Suricata: Yes
  • Periodically download updated Suricata signatures: Yes
  • Automatically scan all PCAP files with Zeek: Yes
  • If you want to delete the oldest indexes when the database exceeds a certain size: No
  • Local reverse DNS lookup for source and destination IP addresses in the logs: No
  • Hardware vendor YES searches for MAC addresses: Yes
  • Perform random string notation on some fields: yes
  • Expose the OpenSearch port to external hosts: no
  • Expose the Logstash port to external hosts: no
  • Forward Logstash logs to an external OpenSearch instance: no
  • Expose the Filebeat TCP port to external hosts: no
  • Expose the SFTP server (for PCAP download) to external hosts: No
  • Enable file extraction with Zeek: yes
  • Picking out interesting that the fetch behavior (Figure A).
  • Choose the file retention method: quarantine
  • Analyze extracted files/PE files with ClamAV: yes
  • Analyze extracted files/PE files with Yara: yes
  • Analyze extracted files/PE files with Capa: yes
  • Look for hashes of files extracted with VirusTotal: no
  • Periodically download updated scanner signatures: yes
  • Should Malcolm capture network traffic to PCAP files for analysis with Arkime: yes
  • Specify the capture interface(s) (separated by commas) that Malcolm will use for network traffic: eth0
  • Capture packets using netsniff-ng (Y/n): yes
  • Capture packets using tcpdump (y/N): no
  • Should Malcolm analyze traffic with Suricata: No
  • Capture filter (tcpdump-like filter expression; leave blank to capture all traffic) 8005
  • Disable capture interface hardware offload and adjust ring buffer sizes: (y/n): n

Figure A

Malcolm’s setup process is an ncurses-based user interface that should be familiar to most Linux administrators.

Once you’ve done that, reboot the system with:

sudo reboot

How to create an administrator account for Malcolm

Once the system has restarted, log back in and access the Malcolm account with:

su – malcolm

Change into the Malcolm directory for the user:

cd ~/Malcolm

Run the administrator account configuration script with:

./scripts/auth_setup

Answer all required questions as such:

  • Store admin username/password for Malcolm local access? yes
  • Create a new admin user and assign a password.
  • (Re)generate self-signed SSL certificates for HTTPS web traffic: yes
  • (Re)generate self-signed certificates for a remote log forwarder: yes
  • Save the username/password to forward Logstash events to a secondary external OpenSearch instance: no
  • Save username/password for email alert sender account: no

How to extract the required Docker image

Malcolm is deployed with Docker, so first we need to pull the official image with:

docker-compose pull

The pull will take a while, so sit back and enjoy the ride or go do something else. Give between two to 10 minutes for this to complete.

How to Start and Access Malcolm

To start the Malcolm service, run the command:

./scripts/start

The above command will deploy the Docker container. Give the containers enough time to deploy and you’re good to go. Malcolm has a few different URLs for different tasks. For each component, be sure to log in with the administrator account you created during the setup step.

  • For the OpenSearch dashboard, the address is https://SERVER/dashbaords, where SERVER is the IP address of the hosting server.
  • For the Malcolm Capture File and Log Archive Upload screen, the address is https://SERVER/upload, where SERVER is the IP address of the hosting server.
  • For the Host and Subnet Map Editor, the address is https://SERVER/name-map-ui, where SERVER is the IP address of the hosting server.
  • For the account management screen, the address is https://SERVER:488, where SERVER is the IP address of the hosting server.

And that’s all there is to deploying Malcolm Network Traffic Analyzer. I hope you will take full advantage of this powerful tool.

Subscribe to TechRepublic How to make the technology work on YouTube for all the latest tech tips for professionals from Jack Wallen.

Charles J. Kaplan