arrow-left

All pages
gitbookPowered by GitBook
1 of 1

Loading...

Test Suite Configuration

Test suite configuration is a text file where you can list all test functions (each test function is a test case) for a batch run. Test suite files are stored in the "~/testsuites" folder

Below is an example of how a test suite file is structured:

  • TEST***_FOLDER - Path to the test artefacts including testbeds, testsuites and reports

  • TEST_CONTACT - Email address of the test owner. This information is included in the test report

  • TESTSUITES - Define the sub-test suites in a key-value pair for execution

    • "./essential/taas_platform_Interface_test.py" - Location of the test script files.

hashtag
Customer suites

FTAS comes with the following predefined suites which can be run directly on the applicable platform and release combinations. These suite files are present in ~/testsuites directory

Suitefile
Description
Number of testcases

oper@ftasvm:~$ cd testsuites/
oper@ftasvm:~/testsuites$ ls -lrth
total 156K
-rw-rw-r-- 1 oper oper  405 Jul  4 07:57 verify_yaml.py
-rw-rw-r-- 1 oper oper  539 Jul  4 07:57 verify_testbed.suite
-rw-rw-r-- 1 oper oper  808 Jul  4 07:57 mgmt_3dut.suite
-rw-rw-r-- 1 oper oper 4.7K Jul  4 07:57 mgmt_2dut.suite
-rw-rw-r-- 1 oper oper 1.8K Jul  4 07:57 mgmt_1dut.suite
-rw-rw-r-- 1 oper oper  11K Jul  4 07:57 edgecore_9716_202111.suite
-rw-rw-r-- 1 oper oper  839 Jul  4 07:57 data_3dut.suite
-rw-rw-r-- 1 oper oper 1.8K Jul  4 07:57 data_1dut.suite
-rw-rw-r-- 1 oper oper 6.9K Jul  4 07:57 PD.suite
-rw-r--r-- 1 oper oper  12K Jul  6 16:13 wistron_6512_ecs2.0.0.suite
-rw-r--r-- 1 oper oper  12K Jul  6 16:13 wistron_3200_ecs2.0.0.suite
-rw-r--r-- 1 oper oper  12K Jul  6 16:13 nvidia_202205.suite
-rw-r--r-- 1 oper oper 6.5K Jul  6 16:13 mgmt_complete.suite
-rw-r--r-- 1 oper oper 7.7K Jul  6 16:13 edgecore_4630_202111.suite
-rw-r--r-- 1 oper oper 6.4K Jul  6 16:13 data_2dut.suite
drwxrwxr-x 2 oper oper 4.0K Jul  7 07:27 ebay
-rw-r--r-- 1 oper oper 5.9K Jul  7 07:27 PI.suite
-rw-r--r-- 1 oper oper  12K Jul 14 11:02 data_complete.suite
-rw-r--r-- 1 oper oper 4.5K Jul 16 10:37 data_4dut.suite
-rw-r--r-- 1 oper oper 8.2K Jul 17 05:58 copy_data_complete.suite
oper@ftasvm:~/testsuites$ 
PI.suite
---
 "TESTBED_ROOT_FOLDER": "/home/oper/testbeds"
 "TESTSCRIPT_ROOT_FOLDER": "/home/oper"
 "TESTSUITE_ROOT_FOLDER": ""
 "TEST_REPORT_ROOT_FOLDER": "/home/oper/reports"
 "CHECK_COMPATIBILITY": true
 "TEST_CONTACT": "phili@aviznetworks.com"
 "TESTSUITES": {
   # 2 DUT test suites/scripts
    "./feature/taas_acl_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_acl_001": ""}, {"test_acl_002": ""}, {"test_acl_003": ""},{"test_acl_004": ""},{"test_acl_005": ""},{"test_acl_006": ""},
                   {"test_acl_007": ""},{"test_acl_008": ""},{"test_acl_009": ""},{"test_acl_010": ""},{"test_acl_011": ""},{"test_acl_012": ""}]
   },
    "./feature/taas_mtu_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_mtu_001": ""}]
   },
    "./feature/taas_mgmt_Syslog_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_syslog_002": ""},{"test_syslog_004": ""}]
   },
    "./feature/taas_autoneg_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_autoneg_001": ""}]
   },
    "./feature/taas_layer2_Vlan_ixia_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_vlan_014": ""},{"test_vlan_016": ""}]
   },
    "./feature/taas_layer2_LACP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_lacp_003": ""}, {"test_lacp_005": ""}, {"test_lacp_011": ""},{"test_lacp_012": ""}]
   },
    "./feature/taas_layer2_LLDP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_lldp_001": ""}, {"test_lldp_002": ""}, {"test_lldp_013": ""}]
   },
     "./feature/taas_layer3_ARP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_arp_003": ""}, {"test_arp_007": ""}, {"test_arp_011": ""},{"test_arp_012": ""}]
   },
     "./feature/taas_layer3_IP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_IP_001": ""}, {"test_IP_002": ""}, {"test_IP_005": ""},{"test_IP_006": ""},{"test_IP_011": ""},{"test_IP_014": ""}]
   },
     "./feature/taas_mgmt_Ping_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_ping_001": ""}, {"test_ping_009": ""}, {"test_ping_011": ""}, {"test_ping_013": ""}]
   },
      "./feature/taas_platform_Interface_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_ports_002": ""}, {"test_ports_005": ""}, {"test_ports_006": ""},{"test_ports_008": ""},{"test_ports_009_14": ""}]
   },
    "./scalability/taas_qual_Scale.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_10_secondary_subnet_under_vlan": ""}, {"test_20_secondary_subnet_under_vlan": ""}, {"test_max_secondary_subnet_under_vlan": ""}, {"test_v4_host_routes_scale_2k": ""},
                   {"test_v4_host_routes_scale_4k": ""}, {"test_v4_host_routes_scale_max_supported": ""}, {"test_v6_host_routes_scale_2k": ""}, {"test_v6_host_routes_scale_4k": ""},
                   {"test_v6_host_routes_scale_max_supported": ""}, {"test_v4_prefix_routes_scale_2k": ""}, {"test_v4_prefix_routes_scale_4k": ""}, {"test_v4_prefix_routes_scale_max_supported": ""},
                   {"test_v4_nexthops_scale_512": ""}, {"test_v4_nexthops_scale_1024": ""}, {"test_v4_nexthops_scale_max_supported": ""}, {"test_qual_scale_007": ""}, {"test_qual_scale_011": ""},
                   {"test_v4_acl_scale_128": ""}, {"test_v4_acl_scale_256": ""}, {"test_v4_acl_scale_512": ""}, {"test_v4_acl_scale_max_supported": ""},
                   {"test_qual_v6_acl_scale_128": ""}, {"test_qual_v6_acl_scale_256": ""}, {"test_qual_v6_acl_scale_512": ""},
                   {"test_qual_v6_acl_scale_max_supported": ""}]
   },
    "./feature/taas_mgmt_SSH_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_ssh_001": ""},{"test_ssh_002": ""},{"test_ssh_003": ""}]
   },
   "./feature/taas_layer3_bgp_netops_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_2duts_topo.py",
     "TESTCASES": [{"test_bgp_netops_002": ""}, {"test_bgp_netops_005_006": ""}, {"test_bgp_netops_008": ""},
                   {"test_bgp_netops_010": ""}]
   },
    "./feature/taas_layer3_BGP_ixia_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_bgp_004": ""},{"test_bgp_005": ""}]
   },
   "./feature/taas_qual_PortCfg_test.py": {
     "SKIP":false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_ports_fec_001": ""},{"test_ports_mtu_002": ""},{"test_ports_counters": ""}]
   },
   "./feature/taas_qual_L3_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_qual_bgp_004": ""},{"test_qual_bgp_007": ""},{"test_qual_bgp_008": ""},
                   {"test_qual_bgp_009": ""},{"test_qual_bgp_010": ""},{"test_qual_bgp_013": ""},{"test_qual_bgp_014": ""},
                   {"test_qual_bgp_015": ""},{"test_qual_bgp_016": ""},{"test_qual_bgp_017": ""},
                   {"test_qual_bgp_019": ""},{"test_qual_vlan1": ""},{"test_qual_vrf": ""}]
   },
    "./feature/taas_qual_Mgmt_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_port_span_001": ""},{"test_port_span_002": ""},{"test_port_span_003": ""},
                   {"test_front_panel_ports_ipv6": ""}]
   },

     "./feature/taas_layer2_Vlan_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{ "test_vlan_004": "" },{ "test_vlan_005": "" },{ "test_vlan_007": "" }, {"test_vlan_008": ""},{"test_vlan_011": ""}]
   }, 

    "./feature/taas_qual_Security_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "full_mesh_topo.py",
     "TESTCASES": [{"test_qual_ip6_acl_001": ""},{"test_qual_ip6_acl_002": ""},{"test_qual_ip6_acl_003": ""},
                   {"test_qual_ip6_acl_004": ""},{"test_qual_ip6_acl_005": ""},{"test_qual_ip6_acl_006": ""}]
   },
    "./feature/taas_qual_SNMP_test.py": {
     "SKIP": false,
     "COMMON_TESTBED": "ftas_mesh_topo.py",
     "TESTCASES": [{"test_snmp_walk_inf_admin_oper": ""}, {"test_snmp_walk_ip_inf_index": ""}, {"test_snmp_walk_ip_to_mac": ""}]
   }
}
SKIP - Defines whether a particular test script will be included or excluded from the execution. If set to "SKIP": true, then the tests will be excluded from execution.
  • COMMON_TESTBED - The testbed file all test cases use.

  • TESTCASES - A list of test functions (test cases) with the structure of "TESTCASES": [{"test_syslog_004": ""},{"test_syslog_002": ""}].

  • All two DUT testcases applicable on data switches

    162

    data_3dut.suite

    All three DUT testcases applicable on data switches

    18

    data_4dut.suite

    All four DUT testcases applicable on data switches

    115

    data_complete.suite

    All testcases applicable on data switches

    400

    mgmt_1dut.suite

    All one DUT testcases applicable on management switches

    22

    mgmt_2dut.suite

    All two DUT testcases applicable on management switches

    88

    mgmt_3dut.suite

    All three DUT testcases applicable on management switches

    8

    mgmt_complete.suite

    All testcases applicable on management switches

    126

    edgecore_4630_202111.suite

    All testcases applicable on Edgecore 4630 platform for 202111 release

    269

    edgecore_9716_202111.suite

    All testcases applicable on Edgecore 9716 platform for 202111 release

    344

    nvidia_202205.suite

    All testcases applicable on NVIDIA platforms for 202205 release

    235

    wistron_3200_ecs2.2.2.suite

    All testcases applicable on Wistron 3200 platforms for ECS 2.2.2 release

    257

    wistron_6512_ecs2.0.0.suite

    All testcases applicable on Wistron 6512 platforms for ECS 2.2.2 release

    284

    PI.suite

    All platform independent testcases

    148

    PD.suite

    All platform dependent testcases

    209

    data_1dut.suite

    All one DUT testcases applicable on data switches

    82

    data_2dut.suite