# ONES Orchestration

## Configuring Devices

Most fabric orchestration solutions available today are complex and often difficult to understand. ONES provides simple and effective tools, such as predefined templates (YAML file), to configure data centers at scale. ONES allows a customized way of configuring devices that includes enhancements to the standard configuration. 

{% hint style="warning" %}
{% code overflow="wrap" %}

```
The device should not have any IP, VLAN, Portchannel, BGP, SAG, or MCLAG config on any of the Interfaces to avoid overlapping during the orchestration and run into cleanup issues.
Post that, the user needs to save the config to a file, in case of any orchestration failure, or misconfiguration user can rollback to the saved config.

```

{% endcode %}

#### Configuration Commands:

* Save Config:

  <pre><code><strong>sudo config save -y
  </strong></code></pre>
* Copy to File:

  <pre><code><strong>sudo cp /etc/sonic/config_db.json /host/backup_config_db.json
  </strong></code></pre>
* Restore Config: //If needed only

  <pre><code>sudo config reload /host/backup_config_db_json -y
  <strong>
  </strong></code></pre>

{% endhint %}

{% hint style="warning" %}
Speed config for the Host Facing port should be updated before Orchestration if there is any change in the default speed config.

Ex: Interface speed is 25G, but you can use it with 10G Transceiver. In such cases, user needs to update speed.

#### Configuration Commands:

Update Speed:

<pre><code><strong>sudo config interface speed Ethernet0 10000
</strong></code></pre>

{% endhint %}

#### Configuration Commands:

* Save Config:

  <pre><code><strong>sudo config save -y
  </strong></code></pre>
* Copy to File:

  <pre><code><strong>sudo cp /etc/sonic/config_db.json /host/backup_config_db.json
  </strong></code></pre>
* Restore Config: //If needed only

  <pre><code><strong>sudo config reload /host/backup_config_db_json -y
  </strong></code></pre>

**You need to provide the following inputs to configure the devices:**

<details>

<summary>Device Details</summary>

Username - Device User-Name

Management IP - Device IP for management

Device Credentials - Login admin user-name & password with write permission

</details>

<details>

<summary>BGP ASN (Autonomous System Number)</summary>

Local AS number - Local BGP AS number a user wants to use

</details>

<details>

<summary>Subnet Details</summary>

IPv4 Subnet - IPv4 address range a user wants to use in the domain

IPv4 Loopback - IPv4 address range to use on the devices only for the loopback address &#x20;

IPv6 Subnet - IPv6 address range a user wants to use in the domain

</details>

<details>

<summary>Connectivity</summary>

Link connectivity between (SuperSpine, Spine, Leaf & TOR)

Link Type (Access Trunk)

Layer-2 / Layer3 - interface type&#x20;

</details>

<details>

<summary>MCLAG Details</summary>

VLAN - VLAN to b used for interfaces&#x20;

PO Group - PortChannel number to be used to bundle the interface

Keepalive VLAN - VLAN a user wants to use to send keepalive messages

VRF number - VRF number to be used for MC-LAG

</details>

<details>

<summary>Host Interface</summary>

L2 Access & VLAN - Host facing interface with Access port & VLAN allocation

L2 Trunk & VLAN - Host facing interface with trunk port & VLAN allocation

L3 - Host facing interface with Layer 3 properties

</details>

<details>

<summary>Network Service Address</summary>

NTP server - NTP server a user wants to add&#x20;

Syslog - Syslog server IP address&#x20;

SNMP - SNMP Server address a user wants to add

</details>

<details>

<summary>VXLAN</summary>

VLAN Range - VLAN range a user wants to use for VXLAN **Ex. 200-205**&#x20;

VNI Range - VNI range a user wants to use for VXLAN **Ex.20000-20005**

Any-Cast Gateway - Any-cast Gateway Subnet

Host Per Vlan - Allocation of Host per VLAN

IRB VLAN Range - IRB VLAN is a separate input and should not overlap with VXLAN VNI

IRB VNI Range - IRB VNI is separate input and should not overlap with VXLAN VNI

</details>

<details>

<summary>QoS</summary>

dscp\_to\_tc\_map - DSCP to TC mapping a user wants to use **Ex. "26:3 | 48:7"**&#x20;

dot1p\_to\_tc\_map - dot1p to TC mapping inputs

tc\_to\_queue\_map - TC to queue mapping inputs **Ex. "3:3 | 7:7"**&#x20;

tc\_to\_pg\_map - TC to PG mapping inputs **Ex. "3:3 | 7:7"**&#x20;

pfc\_enable - PFC value to be enable **Ex. "3"**&#x20;

pfcwd - user can choose this feature to be enable disable with below inputs

&#x20;       enable: false&#x20;

&#x20;        big\_red\_switch: false&#x20;

&#x20;        action: "alert"&#x20;

&#x20;        restoration\_time: 400&#x20;

&#x20;        detection\_time: 400&#x20;

ecn - user can use Explicit congention notification inputs can be added as per infra, below are the inputs can be used

&#x20;        mode: "ecn"&#x20;

&#x20;        cnp\_queue: 7&#x20;

&#x20;        ecn\_gmin: 500000&#x20;

&#x20;        ecn\_gmax: 1500000&#x20;

&#x20;        ecn\_gmark: 10&#x20;

&#x20;        ecn\_rmin: null&#x20;

&#x20;        ecn\_rmax: null&#x20;

&#x20;        ecn\_rmark: null&#x20;

&#x20;        ecn\_ymin: null&#x20;

&#x20;        ecn\_ymax: null&#x20;

&#x20;        ecn\_ymark: null

scheduler:&#x20;

&#x20;        type: "DWRR"&#x20;

&#x20;        weight: "0:0 | 1:0 | 2:0 | 3:90 | 4:0 | 5:0 | 6:0"&#x20;

</details>

{% hint style="info" %}
ONES requires a minimal set of information from the users to configure the devices. The tool is simple to use and allows to configure a large number of devices simultaneously.
{% endhint %}

## Configuration Overview

This section explains the device configuration procedures, that gets applied seamlessly across the entire fabric

* Navigate to <mark style="color:blue;">**Configurations >> Devices**</mark>&#x20;

<figure><img src="https://content.gitbook.com/content/cd2rizc8cOY7cJKDEOsa/blobs/mSJKtp1DZK9f6wLPN8Mu/Screenshot%202023-11-24%20at%202.55.17%E2%80%AFPM.png" alt=""><figcaption></figcaption></figure>

* Click on the <mark style="color:blue;">**Configure Devices**</mark> button in the top right corner. This opens a new screen with a sample device configuration. You can edit the configurations directly in the UI to make the desired changes. You can also download the sample YAML file by clicking on <mark style="color:blue;">**Download YAML**</mark> button, provided at the bottom of the above screen, make new changes and upload the YAML file using <mark style="color:blue;">**Upload**</mark> <mark style="color:blue;">**YAML**</mark> button.

|                                                        |                                                                      |
| ------------------------------------------------------ | -------------------------------------------------------------------- |
| <mark style="color:blue;">**Configure Devices**</mark> | Allows to make the config changes directly in the in-built UI editor |
| <mark style="color:blue;">**Download YAML**</mark>     | Download the sample YAML file and make the required changes          |
| <mark style="color:blue;">**Upload YAML**</mark>       | Upload the modified YAML file                                        |

## A detailed explanation of all the intend fields

**Inventory**:  Specify how many devices you want to add in any particular role - Super Spine, Spine, Leaf, and ToR.

* ```yaml
  Inventory:
    SSpines: 2      #Define the count of devices in the SuperSpine role
    Spines: 2       #Define the count of devices in the Spine role
    Leafs: 2        #Define the count of devices in the Leaf role
    Tors: 1         #Define the count of devices in the Tor role
  ```

**Connectivity**: Specify the parameters required to establish link connectivity such as:

1. **Device switch ID**: unique ID for every device, required to correctly render the topology
2. **Switch name**: hostname of the device
3. **IP address**: management address of the device
4. **Autonomous System Number (ASN)**: BGP AS number to use
5. **Device Credentials**: Username & Password.&#x20;
6. **MCLAG**: Multichassis Link Aggregation
7. **Links**: links connected with another device

* ```yaml
  Connectivity:
    SSpine: []      #A role that does not have any device, presented as []
    Spine:          #Delcare all details, the device needs to update.
      - switchId: 1              #Every device we define for a given Role should have an ID that is unique to that Role.
        switchName: EC01         #Use the hostname of the switch
        ipAddress: 10.4.4.61     #Management IP address of the device to connect
        ASN: 1001                #BGP Local AS number
        Credentials:             #User needs to enter device credentials
          user: admin
          password: YourPaSsWoRd
        mclag:                    #used to create multichassis link aggregation
          enabled: True            #used to activate mclag property
          peer: L2                 #Neighbor Device ID (Leaf Switch 2), Peer can be Spine or Leaf device
          keepalive_vlan: 10       #Vlan to use for Keepalive messages
          isoverl3: true           #for L2 MCLAG use false, For L3 MCLAG use true
          
        Links:      #The links between all the devices we manage
                    #SS indicates SuperSpine Role
                    #S indicates Spine Role
                    #L indicates Leaf Role
                    #T indicates Tor Role
                    #H indicates Host port
                    #The next number is device's unique ID from the same respective Role
                    #Then interface name and number
          - link: S1_Ethernet96 | L1_Ethernet96  
                    #S1_Ethernet96 : Spine Role, Switch ID 1 with Interface Ethernet 96
                                                is connected to
                    #L1_Ethernet96 : Leaf Role, Switch ID 1 with Interface Ethernet 96
            staticLink: True  #True indicates the status is up
            properties:       #This indicates the use of link properties
              mode: 
                  #L2-Trunk  #Used as Layer-2 Trunk
                    #vlan: 200       #Vlan used for mclag 
                    #mc_po_group: 200  #Po number for mclag
                    #vrf: 1          #VRF to be used for mclag, vrf is mendate in symmetric IRB
                  #L2-Access  #Used as Layer-2 Access
                    #vlan: 200       #Vlan used for mclag 
                  #L3-V4            #Only with Host peer
                    #mc_po_group: 201 #Po number to be used
                  #L3-V6            #Only with Host peer
                    #mc_po_group: 201 #Po number to be used
                  #MC-LAG           #Only when we need MC-LAG between links
                  #MC-LAG-BGP       #BGP Neighbourship between MC-Lag Peers using Peer Link
  ```

**BGP**: Specify if you want to enable regular BGP peering or BGP unnumbered peering.

* ```yaml
  BGP:  
      #BGP_U If set to False, this configures regular BGP peering.
      #BGP_U If set to True, this configures BGP unnumbered peering
      #ND_RA is reserved for future development
    BGP_U: False
    ND_RA: 30    
  ```

**PhysicalIfCfg:** Enable or disable FEC and change the MTU settings on all the links being configured.

* ```yaml
  PhysicalIfCfg:
    FEC: On    #Possible values are On or Off
    MTU: 8000  #MTU value a user wants to change
    AdminStatus: Up #Set the status UP 
  ```

**ASN:** Assign a BGP ASN (Autonomous System Number) from the specified pool. Dynamic assigning ASN will be implemented in ONES release 2.0. For release 1.0, you need to specify the ASN under the device configuration, as shown in the sample YAML file.&#x20;

* ```yaml
  ASN:
    SSpine: 10000-20000
    Spine: 21000-50000
    Leaf: 51000-60000
    Tor: 61000-70000
  ```

**IPv4Pool:** Assign IP pools to different subnets. ONES automatically divides the subnets according to the number of available links.

ONES application uses IPv4 subnets for:

* Interfaces&#x20;
* Loopbacks
* Host interfaces
* To configure BGP neighborship
* ONES automatically advertises these subnets in BGP
  * ```yaml
    IPv4Pool:
      Loopback: 19.168.10.0/24
      LeafSpine: 20.168.10.0/24
      LeafTor: 30.168.10.0/24
      Host: 49.0.0.0/24
    ```

**IPv6Pool**: Assign IPv6 subnet.

ONES application uses:

* IPv6 subnet for interfaces
* Subnets to configure BGP neighborship
* Automatically advertise these subnets in BGP
  * ```yaml
    IPv6Pool:
      LeafSpine:   #Define IPv6 Pool 
      LeafTor:     #Define IPv6 Pool
      Host:        #Define IPv6 Pool
    ```

{% hint style="info" %}
Note\* IPv6 loopback is not supported
{% endhint %}

**NTP**: Provide the NTP server address, to enable NTP. You can choose your desired timezone.

* ```yaml
  NTP:
    server: 128.138.141.172
    timezone: Asia/Kolkata
  ```

<details>

<summary>Supported Time Zone</summary>

Only SONiC supported time zone

</details>

**SYSLOG**: Provide the SYSLOG server address, to enable SYSLOG.

* ```yaml
  SYSLOG:
    server: 10.1.1.11

  ```

**SNMP**: Provide the SNMP server address, to enable SNMP.

* ```yaml
  SNMP:
    trapserver: 10.1.1.11
  ```

**Parameters:**  Provide user the ability to enable VXLAN and related Parameters.

* ```yaml
  Parameters:
    vlan: 200-205
    vni: 20000-20005
    anycast_gateway: 100.10.0.0/23
    anycast_mac: 00:11:22:33:44:55
    hosts_per_vlan: 10
    routing_symmetric: True # True / False to indicate its symmetric routing or asymmetric routing
    irb_vlan: 300-305 # irb vlan is separate input and should not overlap with vxlan vni
    irb_vni: 30000-30005 # irb vni is separate input and should not overlap with vxlan vni  
    sag: False # True for Anycast Gateway otherwise False 
  ```

**QoS:**  Provide user the ability to enable RoCE related configuration.

* ```yaml
  QoS:
    dscp_to_tc_map: "26:3 | 48:7"
    dot1p_to_tc_map: null
    tc_to_queue_map: "3:3 | 7:7"
    tc_to_pg_map: "3:3 | 7:7"
    pfc_enable: "3"
    pfcwd:
      enable: false
      big_red_switch: false
      action: "alert"
      restoration_time: 400
      detection_time: 400
    ecn:
      mode: "ecn"
      cnp_queue: 7
      ecn_gmin: 500000
      ecn_gmax: 1500000
      ecn_gmark: 10
      ecn_rmin: null
      ecn_rmax: null
      ecn_rmark: null
      ecn_ymin: null
      ecn_ymax: null
      ecn_ymark: null
    scheduler:
      type: "DWRR"
      weight: "0:0 | 1:0 | 2:0 | 3:90 | 4:0 | 5:0 | 6:0 
  ```

## Creating Configuration

* Navigate to <mark style="color:blue;">**Configurations >> Configure >> YAML**</mark>

<figure><img src="https://2376813750-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcd2rizc8cOY7cJKDEOsa%2Fuploads%2FCohJmnUf5Z1N40BTsxUJ%2FScreenshot%202025-08-16%20at%202.23.46%E2%80%AFAM.png?alt=media&#x26;token=1d3526d2-5574-4c98-ae0b-ff49281851a1" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2376813750-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcd2rizc8cOY7cJKDEOsa%2Fuploads%2FcRRCkVYy1JgAcoxS5yuF%2Fimage.png?alt=media&#x26;token=37b04ff4-e086-4da2-82a4-8f846c44eca3" alt=""><figcaption></figcaption></figure>

### YAML Config Illustrator

While configuring the topology, users can utilize the "Visualize YAML" feature to preview the structure and layout.

Navigate to <mark style="color:blue;">**Configurations >> Configure >> Configure Devices >> \<Upload Valid YAML> >> Visualize YAML**</mark>

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXfCg515wMo5CCLIqNjNr0Fy2C8IYl0kJOpqTvqOYDKCuF9Tp5OugZCzuxjPzpdD5lSsw-XUZEOFMnqHe2D79R6zPz82Jngi8oeYNxSmCAq9iJb_AWq5Bgr_mbOIAgfxJHryufJKwA?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>

## Applying Configuration

Click on <mark style="color:blue;">**Apply Configs**</mark> button in the bottom right corner of the above screen, to push the configs across the entire fabric.

ONES provides real-time updates when the devices are being configured and validates the configurations automatically to ensure the network is ready to use.

You will see the below screen, after the configurations are successfully verified:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXeMFJpUwi-Zx0wGsGFa6zAtR1PxZCNAS0hvsZf6nkDmMFUHjOUN607bg6R7ETcL0OJQhZDsf_Va6m8TbPQcw2RsGDc_ATk5Xi8T1n0j_Cn_6qFG8PriMliMtICqC1ClrbIomU_tig?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>

### FRR config Issue

{% hint style="warning" %}
With Few versions of FRR, when user gives reboot, existing config is erased and default config is pushed to FRR. This will lead to missing the configuration for users.\
below is the workaround config in this situation
{% endhint %}

* Go inside BGP Container & Open docker\_init.sh

  <pre data-line-numbers><code>root@Leaf-1:/# vi /usr/bin/docker_init.sh
  </code></pre>

* Look for "Split" Keyword & **Comment out** the statemets inside that with **if conditions**

  <pre data-line-numbers><code>elif [ "$CONFIG_TYPE" == "split" ]; then
      #echo "no service integrated-vtysh-config" > /etc/frr/vtysh.conf
      rm -f /etc/frr/frr.conf
  </code></pre>

* Add a new line which writes Service integrated in **vtysh.conf**

  <pre data-line-numbers><code>elif [ "$CONFIG_TYPE" == "split" ]; then
      #echo "no service integrated-vtysh-config" > /etc/frr/vtysh.conf
      echo "service integrated-vtysh-config" > /etc/frr/vtysh.conf
      rm -f /etc/frr/frr.conf
  </code></pre>

* Comment **rm -f /etc/frr/frr.conf** & save the changes

  <pre data-line-numbers><code>elif [ "$CONFIG_TYPE" == "split" ]; then
      #echo "no service integrated-vtysh-config" > /etc/frr/vtysh.conf
      echo "service integrated-vtysh-config" > /etc/frr/vtysh.conf
      #rm -f /etc/frr/frr.conf
  </code></pre>

* Commit the **docker Changes**

  <pre data-line-numbers><code>root@Leaf-1:/home/admin# docker commit bgp
  sha256:c67656393b902cf7bf514094a219eb75f5bf45340023848be51e26632344caa3
  </code></pre>

### Various fields shown in the top right corner of the above slide are explained below:

<figure><img src="https://2376813750-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcd2rizc8cOY7cJKDEOsa%2Fuploads%2FyOENaUINwf8YoiS6dRJH%2Fimage.png?alt=media&#x26;token=0c185b45-0849-49c9-89f0-a711d72f990d" alt=""><figcaption></figcaption></figure>

## Sub-options per device

### Connect

Configuration Page allows a user to connect to the device using console access or using SSH method too.

<mark style="color:blue;">**Navigate >> Configuration >> Configure >> \<Device sub options> >> Connect**</mark>

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXeVDxH5_nLAMeX1G2zPG18AomySqRS2uFnwRmR3YHOmExHnsIWeHyalp-vtK_SLBMJUgzROdv-TJF1-sFHIYys31rX72y_FiglgZD-YEP0oiluiYN3dNjpTVTVXqYjDNjutqJCyjQ?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>

User can choose SSH or console option to access the device.

### Console Logs

Console Logs show the exact configuration loaded on the device and the overview of the config loaded as per YAML & Derived Host IP Range can be viewed from the "Derived\_Config:" section

<mark style="color:blue;">**Navigate >> Configuration >> Configure >> \<Device sub options>  >> Console Logs**</mark>

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXfPB0Z1sSd-U-n0u8975PP83T6BSK4PCH8bu55vgGkdxBVqQperQ_S-5kM75a-hh3S5YKwj-H6rg8ynnBFvuRt8Cvv6_afHQawS-IC2kfKcPmQr9LkMyNVp21Ekfk0hhYatBXn4?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>

* Click on the Console Log of the device&#x20;

### Config

This section describes the configuration feature of ONES using which you can check the applied and running configurations.&#x20;

<mark style="color:blue;">**Navigate >> Configuration >> Configure >> \<Device sub options>  >>  Config**</mark>

* **Click on Config**

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXdlg0J7J3tYJB8kisVXq1iyLJRHHp4vNYJ2AwUmUvc71hgNS0UYglMmUiYu0caQ1_TBoIHKDZEoX7kYvMdYBQygpy4fxQrZOXPxg7evJg2HJ28XDeZS7cz-kh6-HIGfQlIteHADmg?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>

### Compare Configuration

This section describes the compare configuration feature of ONES using which you can compare the applied and running configurations.&#x20;

<mark style="color:blue;">**Navigate >> Configurations >> Configure >> \<Device sub options>   >> Compare Configs**</mark>

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXfU-Lb79IBL1i5ZdHdLR9_p6PWYWS7BuDmPWOj3lsrZikLXaMeCo9ondnkh2FhBc5bCklu0ZNGYZ00hz8kitPwy--8ShI0yNZquaRTrAni7Hmz3X8234LfsHtZK_mVbSrwMZa-Z1A?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>

* **Compare Config**

This page allows the user to compare applied configurations to the running configuration of a selected device.

The user should select only one device from the list and perform 'Compare Config' on this page, it may take several minutes to fetch the running configuration from the device

YAML Editor will appear with two windows comparing applied and running configurations as depicted in the below picture

The difference in configuration is highlighted with colour coding in the respective window of the configuration​​

<figure><img src="https://content.gitbook.com/content/cd2rizc8cOY7cJKDEOsa/blobs/r6E8iFXYkyV3x96PzYAH/Screenshot%202023-11-27%20at%208.52.39%E2%80%AFPM.png" alt=""><figcaption></figcaption></figure>

### Config

<mark style="color:blue;">**Navigate >> Configurations >> Configure >> \<Device sub options>   >> Compare Configs**</mark>

This option allow a user to do Day-N operation or any customized config per device.

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXdnXe8duDSrRxtbpO7BBFWIObmuo-6oWYw3h63q7allW9TT6aw4r9NR84FwTKR570B6s-T0EaFwTBpamLiQ0C_EtQRMUov6AGyoORZn8tRknIDR-NZTDSwkszyuLm5y4IbpqPPd_w?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXcIQgjdfit57rs-WdvsVbszzdsr2hIxYkkQYoATpw5uQla10xpXkiKzmTpi84c42qSem_fEz4-NOnlAmwIo6dBQCSxjIghWQqzEI2z1y8dcbZD5klqV60ZtqYim8dQHO4iYvGCL1g?key=uAIlOqmIVEgi3tN34Pe3qQ" alt=""><figcaption></figcaption></figure>