Difference between revisions of "Debian 7 Infiniband"
From Supercomputación y Cálculo Científico UIS
| (One intermediate revision by the same user not shown) | |||
| Line 3: | Line 3: | ||
<div class="thumbnail img-thumbnail">http://wiki.sc3.uis.edu.co/images/a/a8/Logo_sc33.png</div> | <div class="thumbnail img-thumbnail">http://wiki.sc3.uis.edu.co/images/a/a8/Logo_sc33.png</div> | ||
<p><div class="btn btn-primary"><i class="fa fa-long-arrow-left"></i> [[Administración del SC3]]</div></p> | <p><div class="btn btn-primary"><i class="fa fa-long-arrow-left"></i> [[Administración del SC3]]</div></p> | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
<div class="col-md-14"> | <div class="col-md-14"> | ||
| Line 39: | Line 30: | ||
</nowiki></pre>}}</p> | </nowiki></pre>}}</p> | ||
<p>Then, execute the following commands</p> | <p>Then, execute the following commands</p> | ||
| − | <p>{{Command|<nowiki>modprobe mlx4_ib | + | <p>{{Command|<nowiki> |
| + | modprobe mlx4_ib | ||
modprobe ib_ipoib | modprobe ib_ipoib | ||
modprobe rdma_ucm | modprobe rdma_ucm | ||
modprobe ib_umad | modprobe ib_umad | ||
| − | modprobe ib_uverbs </nowiki>}}</p> | + | modprobe ib_uverbs |
| + | </nowiki>}}</p> | ||
<p><b>Subnetting Daemon Configuration</b></p> | <p><b>Subnetting Daemon Configuration</b></p> | ||
<p>find the available ports in the card.</p> | <p>find the available ports in the card.</p> | ||
| Line 69: | Line 62: | ||
<p>{{Command|<nowiki> ifup ib0 </nowiki>}}</p> | <p>{{Command|<nowiki> ifup ib0 </nowiki>}}</p> | ||
<p>Test the interface using the following commands</p> | <p>Test the interface using the following commands</p> | ||
| − | <p>{{Command|<nowiki> ibstat | + | <p>{{Command|<nowiki> |
| − | ibhosts </nowiki>}}</p> | + | ibstat |
| + | ibhosts | ||
| + | </nowiki>}}</p> | ||
</div> | </div> | ||
</div> | </div> | ||
</div> | </div> | ||
Latest revision as of 19:13, 9 April 2015
Infiniband configuration in Debian 7
Install the required software
apt-get -y install infiniband-diags ibutils libmthca ibverbs-utils libibumad libibmad libibverbs1
libmthca1 libmthca1-dbg libmthca-dev libopensm2 opensm
Load the kernel Modules
Load the proper linux modules. Edit the file /etc/modules and add the following lines
File: /etc/modules
#tarjeta Mellanox MT26438
mlx4_ib
#IP over IB
ib_ipoib
# Protocol modules
# Common modules
rdma_ucm
ib_umad
ib_uverbs
Then, execute the following commands
modprobe mlx4_ib
modprobe ib_ipoib
modprobe rdma_ucm
modprobe ib_umad
modprobe ib_uverbs
Subnetting Daemon Configuration
find the available ports in the card.
ibstat -p
The output are the identifiers of the ports. Example:
0x78e7d10300244e95
Then, edit the file /etc/default/opensm
And change the following line
PORTS=ALL
Like this
PORTS="0x78e7d10300244e95"
Start The Subnetting Service
Start the opensm service
/etc/init.d/opensm start
Configure the Infiniband Interface
Edit the file /etc/network/interfaces and add the following lines
File: /etc/network/interfaces
auto ib0
iface ib0 inet static
address 192.168.38.71
netmask 255.255.255.0
broadcast 192.168.38.255
Then, start the interface
ifup ib0
Test the interface using the following commands
ibstat
ibhosts
