arrow-left

All pages
gitbookPowered by GitBook
1 of 7

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Installing Arkime

Arkime Dashboard
  • Download the Arkime installation scripts from herearrow-up-right

  • Copy the tar to the Arkime server and extract using the below command

  • Move to the extracted folder

  • Add permission to execute ‘start.sh’ and ‘stop.sh’ scripts

  • Execute the ‘start.sh’ script, the script will perform the following actions

    1. Start Local ES storage node on port 9200

    2. Setup directories for PCAP and Log capture

circle-info

Arkime uses Elasticsearch(ES) for indexing and searching, So ES must be installed before starting Arkime.

  • Open URL http://<arkime-server-ip>:8005/ from your favourite browser and enter the below credentials

    • User: admin

    • Password: admin

aviz@npbsrv01:~/OPB_Arkime$ tar -zxvf OPB_Analyzer.tar.gz 
OPB_Analyzer/
OPB_Analyzer/stop.sh
OPB_Analyzer/elasticsearch.tar
OPB_Analyzer/start.sh
OPB_Analyzer/arkime.tar
aviz@npbsrv01:~/OPB_Arkime$ 

Ask the user for Interfaces to capture data on

  • Start the Arkime container

  • Installation Pre-Requisites

    This section describes the ports that need to be opened, software and hardware requirements, along with theoretical calculations for storage scaling requirements.

    Customer Firewall Configuration

    Application Name
    Port to be opened

    ElasticSearch

    9200

    Arkime

    8005

    Kibana

    Note: If users want to modify the above-provided ports to run on non-default ports, please refer here to make use of docker port forwarding:

    Browser Requirements: Chrome, Safari

    Software Requirements

    • Ubuntu 22.04

    • Docker needs to be installed.

    • python3 for running helper scripts.

    Solution Components

    hashtag
    Components

    hashtag
    Open Network Operating System

    Open Packet Broker NOS () set of containerized applications built and runs on top of the open-source SONiC NOS. The NOS provides an affordable solution to aggregate, filter, replicate and load balance network traffic from hardware TAPs.

    • Filter, Replicate and Load Balance: APB provides the basic functionalities of any packet broker on commodity ASICs including filtering based on L2/L3/L4 headers, replicating traffic with unique identification and providing symmetric load balancing.

    • Deep Inspection, Truncate, Tunnel: Advance functions including user-defined offset-based inspection of tunnelled packets (IP-in-IP, VXLAN, MPLS, GRE), forward monitored traffic across data centres using IP underlay. One of the unique value-added features includes the truncation of payload reducing the storage cost at tools farm.

    • 5G Ready: APB supports parsing of GTP-C and GTP-U packets, providing advanced filtering and load balancing based on inner headers.

    hashtag
    Packet Capture and Viewer

    Open-source Arkimer (Moloch) is deployed for capturing packets from the network aggregators and load balancers. The capture module can sniff packets from the NIC and also supports processing packets from pcap files. The captured packets can be visualized using a simplified viewer providing insights into the metadata including packet headers and payloads. This component also writes the metadata to a centralized storage cluster for future analysis.

    hashtag
    Centralized Storage and Visualization

    For centralized storage Elastic Storage cluster is utilized which supports distributed scalable storage along with automatic sharding. The metadata saved across the elastic cluster is visualized using Kibana which provides drilled-down information about sessions, network stats and dashboards

    hashtag
    Benefits

    hashtag
    Cloud Network Infrastructure Monitoring

    Monitoring a Cloud network with dislocated data centres can create blindspots that might lead to security and compliance issues. The solution provides comprehensive visibility across the cloud infrastructure, providing end-to-end visibility into the application, and traffic flow and making it easier to correlate network and security events across data centres.

    hashtag
    Advanced Monitoring

    Aviz APB exposes the ASIC capabilities for advanced monitoring including deeper packet inspection for security applications, tunnelling (VXLAN) data across data centres using existing IP underlay, and 5G (GTP) packet processing for load balancing using inner headers. All these advanced capabilities support wire-speed forwarding of monitored traffic.

    hashtag
    Distributed Solution

    The architecture of the complete solution for packet monitoring and analysis is highly distributed, supporting per-site packet capture, analysis and troubleshooting. The extracted metadata from the monitored traffic is stored in a distributed clustered database for centralized analysis and visualization.

    hashtag
    Cost Effective

    The solution provides end-to-end visibility and wire-speed access to network traffic without the need for investing in new hardware. OPBNOS is built on open-source NOS SONiC which is supported by the majority of the ASIC and switch vendors, customers can easily upgrade to this solution on their existing hardware. Packet capture, analysis and visualization are performed using open-source software running on commodity x86 hardware running Linux.

    Configuring APB

    APB provides a highly scalable, flexible and affordable solution to aggregate, filter and load balance network traffic from hardware or virtual TAPs to connected tools for analysis and visualization.

    It provides line rate traffic forwarding using switching ASIC by configuring flow paths between two or multiple ports, By leveraging the APB solution, enterprises can quickly be scaled up or down to meet the ever-changing demands of network visibility and security tools.

    • Download the latest build of APB from herearrow-up-right

    • APB can be installed by following the

    • Login into the switch with default credentials

      • User: admin

      • Password: admin

    • Configure

    • Configure , these are the ports that connect to network-taps

    • Configure , these are the ports that connect to security tools

    • to filter and forward traffic

    • Verify configured flow

    The same rules can also be configured by using the :

    Network Visibility and Analysis with APB, Arkime, Elasticsearch & Kibana

    The cost-efficient Open source Packet Broker Visibility solution harnesses truly open source tools for comprehensive visibility and analysis that benefits up to 80% TCO(Total Cost Ownership).

    The packet stream to Arkime will be fed by APB, providing the capability to filter, load-balance and aggregate traffic from network taps.

    Arkime is a large-scale, open-source, indexed packet capture and search system that augments your current security infrastructure to store and index network traffic in standard PCAP format, providing fast, indexed access.

    Arkime uses Elasticsearch (ES) as its backend data store for storing and indexing network traffic data. Elasticsearch is designed for fast indexing and searching of large volumes of data. It also provides a flexible query language that allows users to easily search and filter data based on various criteria, such as IP addresses, ports, protocols, and time ranges while also being highly scalable and can handle large volumes of data across multiple nodes in a cluster.

    Kibana is a free and open frontend application that sits on top of Elasticsearch(ES), providing search and data visualization capabilities for data indexed in Elasticsearch. Kibana is a data visualization and management tool for Elasticsearch that provides real-time histograms, line graphs, pie charts, and maps. Kibana also includes advanced applications such as Canvas, which allows users to create custom dynamic infographics based on their data, and Elastic Maps for visualizing geospatial data.

    aviz@npbsrv01:~/OPB_Arkime$ cd OPB_Analyzer/
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ ls -l
    total 1208660
    -rw------- 1 aviz aviz 620387328 Dec 23 04:15 arkime.tar
    -rw------- 1 aviz aviz 617262080 Dec 23 04:16 elasticsearch.tar
    -rwxrwxr-x 1 aviz aviz      1403 Dec 23 05:21 start.sh
    -rwxrwxr-x 1 aviz aviz       160 Dec 23 04:43 stop.sh
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ 
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ sudo chmod +x start.sh
    [sudo] password for aviz:
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ sudo chmod +x stop.sh
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ sudo ./start.sh 
    Loading docker images...
    Loaded image: avizdock/docker-arkime:latest
    Loaded image: docker.elastic.co/elasticsearch/elasticsearch:7.17.3
    Enter Semicolon ";" separated list of interfaces to listen for live traffic: enp130s0f1
    vm.max_map_count = 262144
    f07938f9e08183c763791733cda1bd19664675b44269fa7bc7e8d800001156b8
    18f93340d929695b81915bebf8ed4a275439a25c4542a33944f093115facc17c
    Access Arkime Viewer at:
    http://<host>:8005
    username: admin
    password: admin
    
    waiting for initialization...
    
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$
    The following installation scripts assume static IP addresses have been assigned

    to the hosts that run the docker containers, so having the IP address list of the

    hosts would be helpful.

  • Make sure the capture interface for Arkime is in UP state

  • 5601

    HTTP

    80

    SSH/ SCP

    22

    TCP connection for local to remote ES cluster

    9300

    https://docs.docker.com/network/arrow-up-right
  • Unified Management: Aviz FlowVision provides a single pane of glass to manage and visualize the solution. APB also supports industry-standard CLI and RESTful API for integration into customer orchestration systems.

  • APBarrow-up-right

    APB Visibility solution is truly based on open networking principles using open networking switch hardware running software-defined APB built on OpenSource NOS SONiC providing aggregation, filtering and load balancing of monitored traffic. The monitoring solution is completely designed using open-source analytics software for packet processing, storage and visibility on commodity x86 nodes.

    hashtag
    Pre-requisite

    • The following OSes should work out of the box:

      • Arch

      • CentOS/RHEL 7, 8, 9

      • Amazon Linux 2

      • Ubuntu 18.04, 20.04, 22.04

    • An installation of Docker Container Engine.

    Solution Architecture
    Solution Componentschevron-right
    Configuring APBchevron-right
    Installing Arkimechevron-right
    Installing Kibanachevron-right
    Troubleshootingchevron-right
    installation guide
    Management IP
    network ports
    tool ports
    Configure flow
    Aviz FlowVision
    Aviz FlowVision
    Aviz FlowVision
    configure terminal
    interface mgmt
    ip address 10.4.4.96/23 gateway 10.4.4.1
    end
    configure terminal
    interface ethernet Ethernet63/1
    forward-error-correction rs
    type network
    end
    configure terminal
    interface ethernet Ethernet64/1
    forward-error-correction rs
    type tool
    end
    // Traffic Truncation (IPv4 and IPv6)
    
    configure terminal
    interface ethernet Ethernet63/1 
    truncate 64
    
    // Traffic Redirection (L2 and L3 Traffic)
    
    configure terminal
    flow flow01
    network-ports Ethernet63/1
    tool-ports Ethernet64/1
    rule 1 permit vlan 100 counters enable
    rule 2 permit vlan 512 counters enable
    rule 3 permit src-ip 10.10.10.1/32 dest-ip 20.20.20.1/32 counters enable
    rule 4 permit src-ip 213.22.1.2/32 dest-ip 112.23.42.2/32 protocol udp counters enable
    enable
    end
    // LoadBalance Traffic on Multiple Tool Ports
    
    // Traffic Distribution (IPv4 and IPv6) and Symmetric Hashing
    configure terminal
    port-group 10 ports Ethernet6/1, Ethernet7/1
    
    flow flow2
    network-ports Ethernet63/1
    tool-ports port-channel10
    rule 1 permit src-ip 1.1.1.1/32 dest-ip 2.2.2.2/32 protocol tcp counters enable
    rule 2 permit src-ip 2401::1 src-netmask f::f dest-ip 2401::2 dest-netmask f::f counters enable
    enable
    end
    pbnoscli# show flow summary
    Flow-Name       Rule-Id        Status      Counter-Value
    =========================================================
    flow01          2             Active          52562              
    flow01          1             Active          56289              
    pbnoscli#
    pbnoscli# show running-config          
    configure terminal
    interface ethernet Ethernet248
    forward-error-correction rs
    type network
    !
    interface ethernet Ethernet252
    forward-error-correction rs
    type tool
    !
    interface mgmt
    ip address 10.4.4.96/23 gateway 10.4.4.1
    !
    port-group 10 ports Ethernet6/1, Ethernet7/1
    flow flow01
    enable
    network-ports Ethernet63/1
    tool-ports port-group10
    rule 1 permit src-ip 10.10.10.1/32 dest-ip 20.20.20.1/32 counters enable
    rule 2 permit src-ip 213.22.1.2/32 dest-ip 112.23.42.2/32 protocol udp counters enable
    !
    pbnoscli#

    Troubleshooting

    hashtag
    If the UI is not accessible,

    hashtag
    Check the docker status

    Check that Arkime and ES docker are running and the status is ‘UP’ if any of the dockers is not visible. Try running the ‘start.sh’ with the correct permissions, if the issue is not resolved try

    hashtag
    Check that Elasticsearch is running and the status is ‘green’

    hashtag
    Try restarting the dockers

    when Arkime is unable to connect correctly with Elasticsearch the Arkime UI may not be reachable

    Check that UI is reachable by visiting http://arkime-hostname:8005 from your browser

    hashtag
    If ES keeps restarting after a system reboot

    Add the vm.max_map_count setting to a sysctl configuration file to make the change persistent across reboots:

    1. Open the sysctl configuration file /etc/sysctl.conf using a text editor with root privileges: bash

    1. Add the following line to the end of the file:

    Save and close the file

    1. To apply the changes, either reboot your system or run the following command to reload the sysctl settings:

    hashtag
    Changing ES port

    If port 9002 is used by another service running on the server, it can be changed in the ‘start.sh’ script before execution

    Port format: “Global port:Local port”

    Using your favourite text editor, change the ‘Global ports’ to any available and accessible port, also update the same port in the ‘ES_PORT’ attribute.

    hashtag

    hashtag
    Changing Arkime Password

    Log in to the Web GUI and navigate to User> admin_user ⚙️>Password, enter ‘admin’ as the current password and set a new password for the admin user.

    hashtag
    Changing Kibana Port

    If port 5601 is used by another service running on the server, it can be changed in the ‘start.sh’ script before execution

    also if the ES port was changed, the same can be edited here with the ES node IP.

    hashtag
    Stopping Arkime & EC

    Execute the ‘stop.sh’ script to stop & delete Arkime and EC docker containers, the script will not delete the data & es_data folder and the stored PCAPs.

    hashtag
    Stopping Kibana

    Execute the ‘stop.sh’ script to stop & delete Kibana docker containers

    hashtag
    External Links

    • OPBNOS download link -

    • OPBNOS configuration guide -

    • Arkime installation script -

    Aviz support portal -

  • Arkime official website -

  • contacting supportarrow-up-right
    https://aviznetworks.egnyte.com/fl/r4izmRT790arrow-up-right
    https://aviznetworks.gitbook.io/opbnos/arrow-up-right
    https://aviznetworks.egnyte.com/dl/HfSCF1Xr7parrow-up-right
    output of 'sudo dokcer ps'
    Changing ES ports
    Changing password of Arkime GUI
    Kibana Port
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ sudo docker ps
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ curl http://localhost:9200/_cat/health
    1671295994 16:53:14 es-docker-cluster green 1 1 21 21 0 0 0 0 - 100.0%
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ sudo docker restart arkime
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ sudo docker restart elasticsearch
    sysctl -w vm.max_map_count=262144
    sudo nano /etc/sysctl.conf
    vm.max_map_count=262144
    sudo sysctl -p
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ sudo ./stop.sh 
    [sudo] password for aviz: 
    arkime
    elasticsearch
    aviz@npbsrv01:~/OPB_Arkime/OPB_Analyzer$ 
    aviz@npbsrv01:~/OPB_Arkime/OPB_Kibana$ sudo ./stop.sh 
    [sudo] password for aviz: 
    Stopping Kibana...
    kibana
    kibana
    aviz@npbsrv01:~/OPB_Arkime/OPB_Kibana$
    https://support.aviznetworks.com/hc/en-usarrow-up-right
    https://arkime.com/arrow-up-right

    Installing Kibana

    Kibana Dashboard
    • Download the Kibana installation scripts from herearrow-up-right

    • Copy the tar to the Kibana server and extract using the below command

    aviz@npbsrv01:~/OPB_Arkime$ tar -zxvf OPB_Kibana.tar.gz 
    OPB_Kibana/
    OPB_Kibana/stop.sh
    OPB_Kibana/kibana.tar.gz
    OPB_Kibana/start.sh
    aviz@npbsrv01:~/OPB_Arkime$  
    • Move to the extracted folder

    • Execute the ‘start.sh’ script, the script will perform the following actions

      1. Start the Kibana node on port 5601

      2. Connect to the ES Storage node on port 9200

    • Open URL http://<Kibana-server-ip>:5601/ from your favourite browser

    • Create Kibana Dashboards

    Start the Kibana container

    Kibana
    aviz@npbsrv01:~/OPB_Arkime$ cd OPB_Kibana/
    aviz@npbsrv01:~/OPB_Arkime/OPB_Kibana/$ ls -l
    total 312496
    -rw-rw-r-- 1 aviz aviz 319979929 Apr 27 12:13 kibana.tar.gz
    -rwxrwxr-x 1 aviz aviz       747 Apr 28 11:31 start.sh
    -rwxrwxr-x 1 aviz aviz       155 Apr 28 11:33 stop.sh
    aviz@npbsrv01:~/OPB_Arkime/OPB_Kibana$ 
    aviz@npbsrv01:~/OPB_Arkime/OPB_Kibana$ sudo ./start.sh 
    Loading docker images...
    Loaded image: docker.elastic.co/kibana/kibana:7.17.3
    vm.max_map_count = 262144
    WARNING: Published ports are discarded when using host network mode
    5dd385bed21e56393906da9a701e61f69f0e433a4c5a3e11c901860882edd189
    Access Kibana Dashboard at:
    http://<host>:5601
    
    waiting for initialization...
    aviz@npbsrv01:~/OPB_Arkime/OPB_Kibana$