Felix, qui, quod amat, defendere fortiter audet
Home -> Research -> DFSSSP
Home
  Publications
  Research
    LogGOPSim
    CoMPIler
    NB Collectives
    MPI Topologies
    MPI Datatypes
    Netgauge
    Network Topologies
    Ethernet BTL eth
    ORCS
    DFSSSP
    Older Projects
  Teaching
  BLOG
  Travel
  Miscellaneous
  CV

  Events





  Recent Events





DFSSSP - A Deadlock-Free Fast Routing Algorithm for InfiniBand/OpenSM

DFSSSP Description:

The DFSSSP [1] algorithm is a deadlock-free version of the SSSP algorithm [2], which enables balanced routing along shortest paths. SSSP delivers a significantly higher effective bisection bandwidth [3] than other routing algorithms implemented in OpenSM [1,2].
Download patched OpenSM:
See README file in package!

Installing the patched OpenSM

tar xzf management.tar.gz
cd management
export CONFIG_OPTS="--prefix=${HOME}/ofed"
export CONFIG_OPTS="${CONFIG_OPTS} LDFLAGS=-L${HOME}/ofed/lib"
export CONFIG_OPTS="${CONFIG_OPTS} CPPFLAGS=-I${HOME}/ofed/include"

cd libibumad
./autogen.sh && ./configure ${CONFIG_OPTS}
make
make install
cd ..

cd libibmad
./autogen.sh && ./configure ${CONFIG_OPTS}
make
make install
cd ..

cd opensm
./autogen.sh && ./configure ${CONFIG_OPTS}
make
make install

Running the patched OpenSM with SSSP (needs ROOT rights):

cd $HOME/ofed/sbin
./opensm -R sssp

↪ for additional parameters look at ./opensm --help

Run deadlock-free SSSP routing algorithm (needs ROOT rights):

cd $HOME/ofed/sbin
./opensm -R dfsssp

Run DFSSSP with modified QoS file (to configure SL/VL load):

cd $HOME/ofed/sbin
./opensm -R dfsssp --qos_policy_file ${HOME}/ofed/qos.conf


Possible content of qos.conf file:

qos_max_vls 8
qos_high_limit 0

qos_ca_vlarb_high 0:64,1:64,2:64,3:64,4:64,5:64,6:64,7:64
qos_ca_vlarb_low 0:4,1:4,2:4,3:4,4:4,5:4,6:4,7:4
qos_ca_sl2vl 0,1,2,3,4,5,6,7,0,1,2,3,4,5,6,7

qos_swe_vlarb_high 0:64,1:64,2:64,3:64,4:64,5:64,6:64,7:64
qos_swe_vlarb_low 0:4,1:4,2:4,3:4,4:4,5:4,6:4,7:4
qos_swe_sl2vl 0,1,2,3,4,5,6,7,0,1,2,3,4,5,6,7

↪ for explanation see management/opensm/doc/QoS_management_in_OpenSM.txt

DFSSSP was developed by Jens Domke at the ZIH at the Technische Universität Dresden and the scientific work was advised by Torsten Hoefler.

References

IPDPS'11
[1] J. Domke, T. Hoefler and W. Nagel:
 Deadlock-Free Oblivious Routing for Arbitrary Topologies In Proceedings of the 25th IEEE International Parallel \& Distributed Processing Symposium (IPDPS), presented in Anchorage, AL, USA, pages 613--624, IEEE Computer Society, ISBN: 0-7695-4385-7, May 2011, (acceptance rate: 19.6%, 112/571)
HotI'09
[2] T. Hoefler, T. Schneider and A. Lumsdaine:
 Optimized Routing for Large-Scale InfiniBand Networks In 17th Annual IEEE Symposium on High Performance Interconnects (HOTI 2009), presented in New York, NY, Aug. 2009,
Cluster'08
[3] T. Hoefler, T. Schneider and A. Lumsdaine:
 Multistage Switches are not Crossbars: Effects of Static Routing in High-Performance Networks In Proceedings of the 2008 IEEE International Conference on Cluster Computing, presented in Tsukuba, Japan, IEEE Computer Society, ISSN: 1552-5244, ISBN: 978-1-4244-2640, Oct. 2008, (acceptance rate 30%, 28/92)
serving: 38.107.179.236:58487© Torsten Hoefler