Difference between revisions of "Debian 7 Infiniband"

From Supercomputación y Cálculo Científico UIS
Line 27: Line 27:
  
 
{{Command|<nowiki> modprobe mlx4_ib
 
{{Command|<nowiki> modprobe mlx4_ib
        modprobe ib_ipoib
+
modprobe ib_ipoib
        modprobe rdma_ucm
+
modprobe rdma_ucm
        modprobe ib_umad
+
modprobe ib_umad
        modprobe ib_uverbs </nowiki>}}
+
modprobe ib_uverbs </nowiki>}}
  
 
== Subnetting Daemon Configuration ==
 
== Subnetting Daemon Configuration ==
Line 64: Line 64:
  
 
{{File|/etc/network/interfaces|<pre><nowiki>
 
{{File|/etc/network/interfaces|<pre><nowiki>
 
+
auto ib0
        auto ib0
+
iface ib0 inet static
        iface ib0 inet static
+
address 192.168.38.71
        address 192.168.38.71
+
netmask 255.255.255.0
        netmask 255.255.255.0
+
broadcast 192.168.38.255
        broadcast 192.168.38.255
 
 
 
 
</nowiki></pre>}}
 
</nowiki></pre>}}
  
Line 80: Line 78:
  
 
{{Command|<nowiki> ibstat
 
{{Command|<nowiki> ibstat
        ibhosts </nowiki>}}
+
ibhosts </nowiki>}}

Revision as of 16:45, 25 November 2014

Infiniband configuration in Debian 7

Install the required software

apt-get -y install infiniband-diags ibutils libmthca ibverbs-utils libibumad libibmad libibverbs1 libmthca1 libmthca1-dbg libmthca-dev libopensm2 opensm


Load the kernel Modules

Load the proper linux modules. Edit the file /etc/modules and add the following lines

File: /etc/modules
#tarjeta Mellanox MT26438
mlx4_ib

#IP over IB
ib_ipoib

# Protocol modules
# Common modules
rdma_ucm
ib_umad
ib_uverbs

Then, execute the following commands

modprobe mlx4_ib modprobe ib_ipoib modprobe rdma_ucm modprobe ib_umad modprobe ib_uverbs


Subnetting Daemon Configuration

find the available ports in the card.

ibstat -p


The output are the identifiers of the ports. Example:

       0x78e7d10300244e95

Then, edit the file /etc/default/opensm

And change the following line

       PORTS=ALL

Like this

       PORTS="0x78e7d10300244e95"

Start The Subnetting Service

Start the opensm service

/etc/init.d/opensm start


Configure the Infiniband Interface

Edit the file /etc/network/interfaces and add the following lines


File: /etc/network/interfaces
auto ib0
iface ib0 inet static
address 192.168.38.71
netmask 255.255.255.0
broadcast 192.168.38.255

Then, start the interface

ifup ib0


Test the interface using the following commands

ibstat ibhosts