Difference between revisions of "Debian 7 Infiniband"

From Supercomputación y Cálculo Científico UIS
 
(One intermediate revision by the same user not shown)
Line 3: Line 3:
 
<div class="thumbnail img-thumbnail">http://wiki.sc3.uis.edu.co/images/a/a8/Logo_sc33.png</div>  
 
<div class="thumbnail img-thumbnail">http://wiki.sc3.uis.edu.co/images/a/a8/Logo_sc33.png</div>  
 
<p><div class="btn btn-primary"><i class="fa  fa-long-arrow-left"></i> [[Administración del SC3]]</div></p>
 
<p><div class="btn btn-primary"><i class="fa  fa-long-arrow-left"></i> [[Administración del SC3]]</div></p>
 
<div class="column clearfix">
 
    <div class="col-md-14">
 
        <div class="well well-midnight">
 
            <h4></h4>
 
            <p class="bs"></p>
 
        </div>
 
    </div>
 
</div>
 
  
 
<div class="col-md-14">
 
<div class="col-md-14">
Line 39: Line 30:
 
             </nowiki></pre>}}</p>
 
             </nowiki></pre>}}</p>
 
             <p>Then, execute the following commands</p>
 
             <p>Then, execute the following commands</p>
             <p>{{Command|<nowiki>modprobe mlx4_ib
+
             <p>{{Command|<nowiki>
 +
            modprobe mlx4_ib
 
             modprobe ib_ipoib
 
             modprobe ib_ipoib
 
             modprobe rdma_ucm
 
             modprobe rdma_ucm
 
             modprobe ib_umad
 
             modprobe ib_umad
             modprobe ib_uverbs </nowiki>}}</p>
+
             modprobe ib_uverbs
 +
            </nowiki>}}</p>
 
             <p><b>Subnetting Daemon Configuration</b></p>
 
             <p><b>Subnetting Daemon Configuration</b></p>
 
             <p>find the available ports in the card.</p>
 
             <p>find the available ports in the card.</p>
Line 69: Line 62:
 
             <p>{{Command|<nowiki> ifup ib0 </nowiki>}}</p>
 
             <p>{{Command|<nowiki> ifup ib0 </nowiki>}}</p>
 
             <p>Test the interface using the following commands</p>
 
             <p>Test the interface using the following commands</p>
             <p>{{Command|<nowiki> ibstat
+
             <p>{{Command|<nowiki>
             ibhosts </nowiki>}}</p>
+
            ibstat
 +
             ibhosts
 +
            </nowiki>}}</p>
 
         </div>
 
         </div>
 
     </div>
 
     </div>
 
</div>
 
</div>

Latest revision as of 19:13, 9 April 2015


Logo_sc33.png

Infiniband configuration in Debian 7

Install the required software

apt-get -y install infiniband-diags ibutils libmthca ibverbs-utils libibumad libibmad libibverbs1 libmthca1 libmthca1-dbg libmthca-dev libopensm2 opensm

Load the kernel Modules

Load the proper linux modules. Edit the file /etc/modules and add the following lines

File: /etc/modules
            #tarjeta Mellanox MT26438
            mlx4_ib

            #IP over IB
            ib_ipoib

            # Protocol modules
            # Common modules
            rdma_ucm
            ib_umad
            ib_uverbs
            

Then, execute the following commands

modprobe mlx4_ib modprobe ib_ipoib modprobe rdma_ucm modprobe ib_umad modprobe ib_uverbs

Subnetting Daemon Configuration

find the available ports in the card.

ibstat -p

The output are the identifiers of the ports. Example:

0x78e7d10300244e95

Then, edit the file /etc/default/opensm

And change the following line

PORTS=ALL

Like this

PORTS="0x78e7d10300244e95"

Start The Subnetting Service

Start the opensm service

/etc/init.d/opensm start

Configure the Infiniband Interface

Edit the file /etc/network/interfaces and add the following lines

File: /etc/network/interfaces
            auto ib0
            iface ib0 inet static
            address 192.168.38.71
            netmask 255.255.255.0
            broadcast 192.168.38.255
            

Then, start the interface

ifup ib0

Test the interface using the following commands

ibstat ibhosts