Update the above details in the testbed file with your DUTs - DUT connections, Management IP, Login credentials, Link Speed etc.
The "name" parameter is very important. Provide a string to identify the respective for this parameter. This name is displayed in logs for easy identification of devices.
Refer to the topology diagram to find the link variables and update their values
All testbed files can be found in the folder "~/testbeds/"
Testbed file variables
There are two variables to control the cleanup after each test run in the testbed file.
They are CLEANUP_BY_REBOOT and CLEANUP_BY_CFG_RELOAD:
CLEANUP_BY_REBOOT = True, The script will restore the switch's configuration from the /etc/sonic/clean_config.json file and then reboot the switch. This process consumes additional execution time, but it ensures that the DUTs (Devices Under Test) are consistently configured in a clean and proper manner for subsequent test scripts.
CLEANUP_BY_CFG_RELOAD = True, The script will restore the switch's configuration from the /etc/sonic/clean_config.json file and then issue the sudo config load command to load the clean config file. This method takes less time to have a clean configuration on switches but may not work correctly sometimes.
If both CLEANUP_BY_REBOOT and CLEANUP_BY_CFG_RELOAD are set to False, The scripts use the SONiC CLI procedure to un-configure whatever was configured on the switches by the scripts.
CFG_RELOAD_BY_REBOOT = False, The scripts will save the running configuration and reboot the device wherever config reload is required.
Ideally, the below three parameters should be set to False
CLEANUP_BY_REBOOT = False
CLEANUP_BY_CFG_RELOAD = False
CFG_RELOAD_BY_REBOOT = False
CLEANUP_BEFORE_TEST_RUN = False. It can be used to do the forceful clean-up of all the devices before the test run
Apart from these, the following variables are specified in the testbed file:
CFG_RELOAD_BY_REBOOT = True, The scripts initiate device reboots in instances where the config reload command fails on the DUT due to certain reasons. This measure is taken as a workaround for such situations.
REBOOT_WAIT_TIME = 0, Maximum wait time for the device to reboot
NTP_SERVER = <FTAS VM IP>, FTAS VM serves as NTP server.
SYSLOG_SRVS = {"Servers": ["<FTAS VM IP>", "10.4.5.6"], list of syslog servers to use. First list member should be set to the FTAS VM IP. The second IP can be a dummy IP.
Log_Folder: "/var/log/sonic_logs" : The log folder on FTAS VM to be used for syslog testing
MAX_V4_ACL , Maximum IPv4 ACL rules supported on the platform
MAX_V6_ACL , Maximum IPv6 ACL rules supported on the platform
MAX_SECONDARY_SUBNET, Maximum secondary subnets supported for SVI interface
MAX_IPV4_HOST_ROUTES, Maximum IPv4 host routes supported on the platform
MAX_IPV6_HOST_ROUTES, Maximum IPv6 host routes supported on the platform
MAX_IPV4_PREFIX_ROUTES, Maximum IPv4 prefix routes supported on the platform
MAX_IPV6_PREFIX_ROUTES, Maximum IPv6 prefix routes supported on the platform
TECHSUPPORT = True, Takes techsupport dump (If True) for the DUTs in case of failures
TECHSUPPORT_SINCE = "hour ago", Specifies the argument to the show techsupport command while collecting the techsupport dump in case of failures. This is valid only when TECHSUPPORT = True . Valid values are: hour ago (default), yesterday , or any other values supported by SONiC show techsupport --since option.
TECHSUPPORT_TIMEOUT, specifies the worst case timeout value for techsupport dump generation, in case TECHSUPPORT = True
Stress testcases
STRESS_AVAIL_CORES = 2 , Number of CPU cores reserved for system use; rest other CPU cores will undergo stress testing. Example: if total cores are 16. Then STRESS_AVAIL_CORE=2 would mean that 14 cores will be stressed, and only two cores will be available for the test case.
STRESS_MEM_UTIL = 85 , the percentage memory to be stressed during stress testing. In this case 85% of memory would be stressed, leaving only 15% memory available.
SERVER_IP = <FTAS_VM IP> , IP address of the server hosting the stress-ng docker image . The docker is part of FTAS VM.
SERVER_USER_ID = "oper" , User ID for SCP access to the server hosting the stress-ng Docker image
SERVER_PASSWORD = "oper@123" , Password for SCP access to securely transfer the stress-ng Docker image
Parameters specific to resilience testcases
TRAFFIC_THRESHOLD =1, acceptable packet/frame loss percentage in traffic testcases
CPU_THRESHOLD = 5 , acceptable change in CPU utilization percentage
MEM_THRESHOLD = 5 , acceptable change in memory utilization percentage
Variables with the default values
Variable with Default values
Description
INTF_UP_WAIT_TIME = 30
Timeout for the interface to be 'Operationally UP'
CLI_TIMEOUT = 0
Assign an integer value to this variable if the command execution on the device experiences slowness or takes longer than 30 seconds to respond.
If the value is not explicitly set, the default will be 0, implying a 30-second wait time for the command execution output.
MAX_V4_ACL = 64
The maximum number of supported IPV4 ACL rules.
The variable used in the testcase test_v4_acl_scale_max_supported in test script scalability/taas_qual_scale.py
MAX_V6_ACL = 64
The maximum number of supported IPV6 ACL rules.
The variable used in the testcase test_qual_v6_acl_scale_max_supported in test script scalability/taas_qual_scale.py
MAX_SECONDARY_SUBNET = 25
The maximum number of supported secondary subnets under a vlan.
The variable used in the testcase test_max_secondary_subnet_under_vlan in test script scalability/taas_qual_scale.py
MAX_IPV4_HOST_ROUTES = 1000
The maximum number of IPv4 host routes supported.
The variable used in the testcase test_v4_host_routes_scale_max_supported in test script scalability/taas_qual_scale.py
MAX_IPV6_HOST_ROUTES = 1000
The maximum number of IPV6 host routes supported.
The variable used in the testcase test_v6_host_routes_scale_max_supported in test script scalability/taas_qual_scale.py
MAX_IPV4_PREFIX_ROUTES = 1000
The maximum number of IPV4 prefix routes supported.
The variable used in the testcase test_v4_prefix_routes_scale_max_supported in test script scalability/taas_qual_scale.py
MAX_IPV4_NEXTHOPS = 256
The maximum number of IPV4 next-hop supported.
The variable used in the testcase test_v4_nexthops_scale_max_supported in test script scalability/taas_qual_scale.py
STRESS_AVAIL_CORES = 2
Number of CPU cores reserved for system use. Rest other CPU cores will undergo stress testing
STRESS_MEM_UTIL = 85
Targeted percentage of total system memory to allocate for stress testing
Specify the port connected to the DUT like this: "s1_p1": "1;8" , where the port number 8 on Ixia card 1 is connected to Spine01_Port01 for Ixia traffic.
For example, Ixia port 1;8 can be configured like this: "<chassis_ip>;<card_no>;<Port_no>", where <chassis_ip> refers to the Ixia chassis IP and 1;8 is the port with the card number.
The FTAS VM has been extensively tested with UHD version 1.3.3003.118, It is recommended to use UHD with the same version for smooth operations.
Ixia port format varies and depends on the Ixia Chassis type
Control Ixia traffic rate globally
For the testbeds where links between DUTs have less bandwidth than Ixia ports traffic tests may fail due to traffic drops on the least bandwidth links (i.e. DUTs may be connected with 1G ports and the Ixia link maybe 100G).
To avoid this situation a global parameter for the ixia traffic rate "global_traffic_rate" in percentage could be used to enforce traffic rate in all tests.
When the global_traffic_rate parameter is defined as a sub-parameter in the "Ixia_Ports" all traffic streams in all test scripts would override their own rate value with global_traffic_rate value configured.
When global_traffic_rateis not defined all traffic streams in all test scripts would use their own static values for the traffic rate.
Here the "global_traffic_rate": 25 referrers to 25% speed for all Ixia ports
CLI_TIMEOUT, set the integer value for this variable when the command execution is slow or taking more than 30 sec to respond. Set the default value to 0 in the testbed file.
If you don't have 4 DUTs topology and want to run scripts that require only 2 DUTs, You can use 2dut_topo.py testbed parameter file
There are two physical DUTs in the topology but the script might pick the name Spine and Leaf interchangeably. So in the testbed file, we should define parameters for both but they point to the same physical Spine and Leaf DUTs.
ftas_chaos_topo.py (sample topology file) the testbed parameter file is used for Chaos test scripts only.
A new parameter PCH_CONFIGURATION = False, has been added. This parameter determines if the tests are to be run on routed port (if False) or routed PortChannel (if True) configuration for the interfaces connecting DUTs.
