Split-Mode on Unmanaged Mellanox Switches
Enabling split-mode is really easy (once you know the commands).
To enable splitting on a Mellanox Switch we must do the following:
- Get the LID
- Query the device
- Set the Split-mode
Getting the LID
# Get LID from ibnetdiscover
MLX_SW_LID_ID=$(ibnetdiscover |grep -i switch|grep -i Mellanox| awk -F "# " '{print $NF}'| sed -n 's/.*lid \([0-9]*\).*/\1/p')
MLX_SW_LID="lid-${MLX_SW_LID_ID}"
# Or get it manually by doing the following command
ibnetdiscover
Getting the current status
We can get the device status with the command below. In our case we already have enabled Auto-Split before, but in your case it should appear as disabled or not at all.
# Command
root@pve:~# mlxconfig -d "${MLX_SW_LID}" q
# Output
Device #1:
----------
Device type: Quantum
Name: MQM8790-HS2X_Ax
Description: Mellanox Quantum(TM) HDR InfiniBand Switch; 40 QSFP56 ports; 2 Power Supplies (AC); unmanaged; standard depth; P2C airflow; Rail Kit; RoHS6
Device: "${MLX_SW_LID}" # You'll see your Switch LID here.
Configurations: Next Boot
DISABLE_AUTO_SPLIT ENABLE_AUTO_SPLIT(0)
GB_UPDATE_MODE ALL(0)
GB_VECTOR_LENGTH 0
GB_VECTOR Array[0..7]
PERST_GPIO_CONFIG DISABLE_PERST_GPIO(0)
RO SPLIT_CAP SPLIT_2X(1)
SPLIT_MODE SPLIT_2X(1)
RO SPLIT_NUM_OF_PORTS 40
SPLIT_PORT Array[1..64]
The 'RO' shows parameters which are for read only and cannot be changed
Enabling Auto Split
We can enable auto-splitting with the following command:
mlxconfig -d "${MLX_SW_LID}" set SPLIT_MODE=SPLIT_2X
Restarting the Switch
After we’ve made the SPLIT_MODE changes we need to restart the switch:
flint -d "${MLX_SW_LID}" swreset
Or you can simply physically restart it from the restart button.