Infiniband configuration in Debian 7

Install the required software

 apt-get -y install infiniband-diags ibutils libmthca ibverbs-utils libibumad libibmad libibverbs1 libmthca1 libmthca1-dbg libmthca-dev libopensm2 opensm 

Load the kernel Modules

Load the proper linux modules. Edit the file /etc/modules and add the following lines

File: /etc/modules


        #tarjeta Mellanox MT26438
        mlx4_ib

        #IP over IB
        ib_ipoib

        # Protocol modules
        # Common modules
        rdma_ucm
        ib_umad
        ib_uverbs

Then, execute the following commands

 modprobe mlx4_ib
        modprobe ib_ipoib
        modprobe rdma_ucm
        modprobe ib_umad
        modprobe ib_uverbs 

Subnetting Daemon Configuration

find the available ports in the card.

 ibstat -p 

The output are the identifiers of the ports. Example:

       0x78e7d10300244e95

Then, edit the file /etc/default/opensm

And change the following line

       PORTS=ALL

Like this

       PORTS="0x78e7d10300244e95"

Start The Subnetting Service

Start the opensm service

 /etc/init.d/opensm start 

Configure the Infiniband Interface

Edit the file /etc/network/interfaces and add the following lines

File: /etc/network/interfaces


        auto ib0
        iface ib0 inet static
        address 192.168.38.71
        netmask 255.255.255.0
        broadcast 192.168.38.255

Then, start the interface

 ifup ib0 

Test the interface using the following commands

 ibstat
        ibhosts 

Debian 7 Infiniband

Contents