Test Suite Configuration

Test suite configuration is a Yaml file where you can list all test functions (each test function is a test case) for a batch run. Test suite files are stored in the "~/testsuites" folder

oper@ftasvm:~$ cd testsuites/
oper@ftasvm:~/testsuites$ ls -lrth
total 36K
-rw-rw-r-- 1 oper oper  405 Jan 27 10:19 verify_yaml.py
-rw-rw-r-- 1 oper oper 2.3K Jan 27 10:19 ftas_4dut_suite.yaml
-rw-rw-r-- 1 oper oper 8.9K Jan 31 07:18 ftas_full_suite.yaml
-rw-rw-r-- 1 oper oper  815 Jan 31 07:18 ftas_3dut_suite.yaml
-rw-rw-r-- 1 oper oper 2.1K Jan 31 07:18 ftas_1dut_suite.yaml
-rw-rw-r-- 1 oper oper 5.4K Feb 18 12:54 ftas_2dut_suite.yaml
oper@ftasvm:~/testsuites$ 

ftas_2dut_suite.yaml

---
 "TESTBED_ROOT_FOLDER": "/home/oper/testbeds"
 "TESTSCRIPT_ROOT_FOLDER": "/home/oper"
 "TESTSUITE_ROOT_FOLDER": ""
 "TEST_REPORT_ROOT_FOLDER": "/home/oper/reports"
 "TEST_CONTACT": "[e-mail]"
 "TESTSUITES": {
   # 2 DUT test suites/scripts
    "./essential/taas_acl_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_acl_001": ""}, {"test_acl_002": ""}, {"test_acl_003": ""},{"test_acl_004": ""},{"test_acl_005": ""},{"test_acl_006": ""},
                   {"test_acl_007": ""},{"test_acl_008": ""},{"test_acl_009": ""},{"test_acl_010": ""},{"test_acl_011": ""},{"test_acl_012": ""}]
   },
    "./essential/taas_mtu_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_mtu_001": ""}]
   },
    "./essential/taas_mgmt_Syslog_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_syslog_004": ""},{"test_syslog_002": ""}]
   },
    "./essential/taas_autoneg_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_autoneg_001": ""}]
   },
    "./essential/taas_layer2_Vlan_ixia_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_vlan_016": ""},{"test_vlan_014": ""}]
   },
    "./essential/taas_dropcounters_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_dropcounters_001": ""}, {"test_dropcounters_002": ""}, {"test_dropcounters_003": ""},{"test_dropcounters_004": ""},
     {"test_dropcounters_005": ""}]
   },

    "./essential/taas_layer2_LACP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_lacp_003": ""}, {"test_lacp_005": ""}, {"test_lacp_011": ""},{"test_lacp_012": ""}]
   },
    "./essential/taas_layer2_LLDP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_lldp_001": ""}, {"test_lldp_002": ""}, {"test_lldp_013": ""}]
   },
     "./essential/taas_layer3_ARP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_arp_003": ""}, {"test_arp_007": ""}, {"test_arp_011": ""},{"test_arp_012": ""}]
   },
     "./essential/taas_layer3_IP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_IP_001": ""}, {"test_IP_002": ""}, {"test_IP_005": ""},{"test_IP_006": ""},{"test_IP_011": ""},{"test_IP_014": ""}]
   },
     "./essential/taas_mgmt_Ping_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_ping_001": ""}, {"test_ping_009": ""}, {"test_ping_011": ""}, {"test_ping_013": ""}]
   },
      "./essential/taas_platform_Interface_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_ports_002": ""}, {"test_ports_005": ""}, {"test_ports_006": ""},{"test_ports_008": ""},{"test_ports_009_14": ""}]
   },
    "./qual/taas_qual_Scale.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_qual_scale_001": ""},{"test_qual_scale_002": ""},{"test_qual_scale_003": ""},{"test_qual_scale_004": ""},
                   {"test_qual_scale_005": ""},{"test_qual_scale_006": ""},{"test_qual_scale_007": ""},{"test_qual_scale_008": ""},
                   {"test_qual_scale_009": ""},{"test_qual_scale_010": ""},{"test_qual_scale_011": ""}]
   },
    "./essential/taas_mgmt_SSH_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_ssh_001": ""},{"test_ssh_002": ""},{"test_ssh_003": ""}]
   },
    "./essential/taas_layer3_BGP_ixia_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_bgp_004": ""},{"test_bgp_005": ""},{"test_bgp_006": ""}]
   },
   "./qual/taas_qual_PortCfg_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_ports_fec_001": ""},{"test_ports_mtu_002": ""},{"test_ports_counters": ""}]
   },
   "./qual/taas_qual_L3_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_qual_bgp_004": ""},{"test_qual_bgp_007": ""},{"test_qual_bgp_008": ""},
                   {"test_qual_bgp_009": ""},{"test_qual_bgp_010": ""},{"test_qual_bgp_013": ""},{"test_qual_bgp_014": ""},
                   {"test_qual_bgp_015": ""},{"test_qual_bgp_016": ""},{"test_qual_bgp_017": ""},{"test_qual_bgp_018": ""},
                   {"test_qual_bgp_019": ""},{"test_qual_vlan1": ""},{"test_qual_vrf": ""}]
   },
    "./qual/taas_qual_Mgmt_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_port_span_001": ""},{"test_port_span_002": ""},{"test_port_span_003": ""},
                   {"test_front_panel_ports_ipv6": ""}]
   },

     "./essential/taas_layer2_Vlan_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_vlan_004": ""},{"test_vlan_005": ""},{"test_vlan_007": ""},{"test_vlan_008": ""},{"test_vlan_011": ""}]
   }, 

    "./qual/taas_qual_Security_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_qual_ip6_acl_001": ""},{"test_qual_ip6_acl_002": ""},{"test_qual_ip6_acl_003": ""},
                   {"test_qual_ip6_acl_004": ""},{"test_qual_ip6_acl_005": ""},{"test_qual_ip6_acl_006": ""}]
   },
}

• TEST***_FOLDER - Path to the test artefacts including testbeds, testsuites and reports

• TEST_CONTACT - Email address of the test owner. This information is included in the test report

• TESTSUITES - Define the sub-test suites in a key-value pair for execution

  • "./essential/taas_platform_Interface_test.py" - Location of the test script files.

  • SKIP - Defines whether a particular test script will be included or excluded from the execution. If set to "SKIP": true, then the tests will be excluded from execution.

  • COMMON_TESTBED - The testbed file all test cases use.

  • TESTCASES - A list of test functions (test cases) with the structure of "TESTCASES": [{"test_syslog_004": ""},{"test_syslog_002": ""}].

Custome suites

FTAS comes with following predefined suites which can be run directly on applicable platform and release combinations. These suitefiles are present in ~/testsuites directory

Suitefile	Description	Number of testcases
PI.suite	All platform independent testcases	90
PD.suite	All platform dependent testcases	86
data_1dut.suite	All one DUT testcases applicable on data switches	19
data_2dut.suite	All two DUT testcases applicable on data switches	90
data_3dut.suite	All three DUT testcases applicable on data switches	8
data_4dut.suite	All four DUT testcases applicable on data switches	58
data_complete.suite	All testcases applicable on data switches	176
mgmt_1dut.suite	All one DUT testcases applicable on management switches	18
mgmt_2dut.suite	All two DUT testcases applicable on management switches	77
mgmt_3dut.suite	All three DUT testcases applicable on management switches	8
mgmt_complete.suite	All testcases applicable on management switches	104
edgecore_4630_202111.suite	All testcases applicable on Edgecore 4630 platform for 202111 release	114
edgecore_9716_202111.suite	All testcases applicable on Edgecore 9716 platform for 202111 release	171
nvidia_202205.suite	All testcases applicable on NVIDIA platforms for 202205 release	180
wistron_3200_ecs2.0.0.suite	All testcases applicable on Wistron 3200 platforms for ECS 2.0.0 release	116
wistron_6512_ecs2.0.0.suite	All testcases applicable on Wistron 6512 platforms for ECS 2.0.0 release	116