The Subnet Manager (SM) in an InfiniBand network is responsible for handling changes in the network, such as adding or removing devices, while ensuring that the network continues to operate efficiently and reliably. Here’s how the Subnet Manager manages these changes:

Adding Devices:
When a new device is added to the InfiniBand network, the Subnet Manager performs the following steps:

  1. Initialization: The new device is powered on and connected to the InfiniBand fabric.
  2. Discovery: The Subnet Manager detects the presence of the new device by monitoring the fabric. The SM identifies the device’s InfiniBand Node GUID (Globally Unique Identifier) and assigns a unique Local Identifier (LID) to it.
  3. Configuration Update: The Subnet Manager updates its internal database to include information about the new device, including its LID, GUID, and other relevant attributes.
  4. Topology Update: The SM updates the network’s topology information to incorporate the new device. This may involve updating routing tables, generating new paths, and ensuring that other devices are aware of the changes.
  5. Routing Table Adjustment: If necessary, the Subnet Manager updates routing tables to account for the new device and ensure that data can be efficiently routed to and from it.
  6. Address Resolution Update: The SM helps facilitate address resolution, ensuring that other devices can map the new device’s InfiniBand address to its physical location.

Removing Devices:
When a device is removed from the InfiniBand network, the Subnet Manager takes the following steps:

  1. Detection: The Subnet Manager monitors the network and detects that a device is no longer active or present.
  2. Topology Update: The SM updates the network’s topology information to remove the device that was removed. This includes adjusting routing tables and paths to account for the absence of the device.
  3. Routing Table Adjustment: The Subnet Manager updates routing tables to ensure that data is no longer routed to the removed device.
  4. Address Resolution Update: The SM updates address resolution information to reflect the fact that the removed device is no longer part of the network.

Handling Network Changes:
In both cases, the Subnet Manager plays a critical role in maintaining the integrity of the network and ensuring that data communication remains efficient. It dynamically adjusts routing information, topology data, and other network parameters to accommodate changes without disrupting ongoing communication or causing congestion.

It’s important to note that the Subnet Manager’s ability to handle network changes efficiently is one of the reasons why InfiniBand is well-suited for dynamic high-performance computing environments, where nodes might be added, removed, or reconfigured based on changing workload demands.