Mellanox Technologies InfiniScale MTS3600Q-1BNC Manual de usuario

www.mellanox.com

InfiniScale® IV 36-Port QSFP 40 Gb/s

InfiniBand Switch User Manual

P/N: MTS3600Q-1BNC, MTS3600Q-2BNC, MTS3600Q-1UNC, MTS3600Q-2UNC,

MTS3600R-1BNC, MTS3600R-2BNC, MTS3600R-1UNC, MTS3600R-2UNC

Rev 2.7

Mellanox Technologies

350 Oakmead Parkway Suite 100

Sunnyvale, CA 94085

U.S.A.

www.mellanox.com

Tel: (408) 970-3400

Fax: (408) 970-3403

Mellanox Technologies, Ltd.

PO Box 586 Hermon Building

Yokneam 20692

Israel

Tel: +972-4-909-7200

Fax: +972-4-959-3245

Mellanox, BridgeX, ConnectX, Virtual Protocol Interconnect, InfiniBlast, InfiniBridge, InfiniHost, InfiniRISC, InfiniScale, and InfiniPCI

are registered trademarks of Mellanox Technologies, Ltd. CORE-Direct, FabricIT, and PhyX are trademarks of Mellanox

Technologies, Ltd. All other trademarks are property of their respective owners.

All other marks and names mentioned herein may be trademarks of their respective companies.

QDR InfiniBand Switch Platform User Manual

Document Number: 2983

Rev 2.7

Mellanox Technologies

NOTE:

THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE

PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE

OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE

CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX

TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX

TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE

HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE

DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT,

INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO,

PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS

INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,

OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND

RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

QDR InfiniBand Switch Platform User Manual Rev 2.7

Mellanox Technologies 3

Table of Contents

Table of Contents 3

List of Figures 5

List of Tables 6

Revision History 7

About this Manual 8

Intended Audience 8

Related Documentation 8

Conventions 9

Mellanox Part Numbering Legend 9

Chapter 1 Overview 10

1.1 Serial Number and Product Version Information 10

Chapter 2 Internally Managed vs. Unmanaged (Externally Managed) 12

Chapter 3 Basic Operation and Installation 13

3.1 Switch Platform Hardware Overview 13

3.1.1 Status LEDs 13

3.2 QSFP Cable Power Budget Classification 17

3.3 Interfaces 17

3.3.1 Port Connector Interfaces 17

3.3.2 Management and FW updating Interfaces 18

3.4 Package Contents 19

3.5 Switch Platform Installation and Operation 20

3.5.1 Installation Safety Warnings 20

3.5.2 Mechanical Installation 22

3.5.3 Grounding the Switch 27

3.5.4 Power Connections and Initial Power On 27

3.5.5 Extracting and Inserting the Power Supply Unit 28

3.5.6 InfiniBand Cable Installation 30

3.5.7 Extracting and Inserting the Fan Unit 31

3.6 Disassembly of the Switch from the Rack 32

3.7 Disposal 32

Chapter 4 Management and Tools Overview 33

4.1 Network Management and Clustering Software 33

4.2 Internally Managed Switch System 33

4.2.1 Configuring the Switch for the First Time 34

4.2.2 Accessing the Switch Locally 34

4.2.3 Starting a Remote Connection to the Switch 35

4.2.4 Downloading Firmware 35

4.3 Unmanaged (Externally Managed) Switch 35

4.3.1 I2C Connector 35

4.3.2 Updating Firmware 36

4.3.3 How to Get Mellanox Firmware Tools (MFT) 39

4.3.4 Open SM 39

Chapter 5 Switch Management Tools 40

5.1 Chassis Management 40

Chapter 6 FabricIT Management 41

6.1 Downloading FabricIT Software and Documents 41

Rev 2.7

Mellanox Technologies

6.2 Installing Licenses 41

6.2.1 Getting the License Key 41

6.2.2 FabricIT Management and Inspection License 42

Chapter 7 Troubleshooting 43

Appendix A Specification 46

Appendix B Switch Hardware Components 51

Appendix C QSFP Interface 52

Appendix D Replacement Parts Ordering Numbers 54

Appendix E Avertissements de sécurité d’installation (French) 55

Appendix F Installation - Sicherheitshinweise (German) 57

Appendix G Advertencias de seguridad para la instalación (Spanish) 59

Appendix H Special Regulations Regarding Finland, Sweden, Denmark, and Norway 62

QDR InfiniBand Switch Platform User Manual Rev 2.7

Mellanox Technologies 5

List of Figures

Figure 1: Generic Product label 10

Figure 2: Management Module Label 11

Figure 3: QSFP Switch System Power and Connector Side Panels 13

Figure 4: Power, Fan, and System LEDs 15

Figure 5: MTS3600 QSFP Power Side Panel 16

Figure 6: PSU Status LEDs 16

Figure 7: Port Numbering 18

Figure 8: Top and Bottom Ports 18

Figure 9: Management Interfaces 18

Figure 10: Rack Installation Kit Parts 23

Figure 11: Which Side of the Rack Do You Want the Connectors? 24

Figure 12: Screwing the Rail Slide onto the switch 24

Figure 13: Caged Nut Spacing 25

Figure 14: Screwing in the Rails 25

Figure 15: Which Side of the Rack Do You Want the Connectors? 26

Figure 16: Two Power Inlets - Electric Caution Notifications 28

Figure 17: Power Supply Unit Extraction 29

Figure 18: Top and Bottom Ports 30

Figure 19: Cable connections to the Switch 31

Figure 20: Host Connection 34

Figure 21: License Key Entitlement Number 42

Figure 22: License Key Generation Form 42

Rev 2.7

Mellanox Technologies

List of Tables

Table 1: Revision History Table 7

Table 2: Reference Documents 8

Table 3: Part Numbering Legend 9

Table 4: Switch Management 12

Table 5: System Status LED Configurations 15

Table 6: PSU Status LED Configurations 16

Table 7: Fan Status LED Configurations 17

Table 8: MTS3600Q Specification Data 46

Table 9: MTS3600R Specification Data 47

Table 10: Switch Certification Status 48

Table 11: InfiniBand QSFP Connector Pinout 52

Table 12: Replacement Parts Ordering Numbers 54

QDR InfiniBand Switch Platform User Manual Rev 2.7

Mellanox Technologies 7

Revision History

Table 1 - Revision History Table

Date Revision Description

February 2011 Rev 2.7 Updated power numbers.

Changed Grounding section

July 2010 Rev 2.6.1 Added “QSFP Cable Power Budget Classification”

Added active power cable power level in Appendix A Specs

Added “Getting the License Key”

May 2010 Rev 2.6 Updated Section 3.2 to explain the I2C connector use in managed switches.

Added safety warnings in Spanish.

Added Note: ETH Ports Must be 10/100M

November 2009 Rev 2.5 removed reference to dual air flow for MTS3600

August 2009 Rev 2.4 Added that CONSOLE, Ethernet, and USB interfaces do not work in Unmanaged

switches

August 2009 Rev 2.3 Updated Figures 4 and 5. Updated interfaces to reflect CONSOLE interface.

August 2009 Rev 2.2 Updated references to initial start up connecting to a host PC.

August 2009 Rev 2.1 Removed procedure to replace the battery.

June 2009 Rev 2.0 Added Management module label and RS232 DB9 to DB9 harness

Added part numbers for the two management modules.

June 2009 Rev 1.9 Made fixes to management section

added Disassembly instructions

June 2009 Rev 1.8 Added Fabric IT section

April 2009 Rev 1.7 inserted procedure to replace the battery.

April 2009 Rev 1.6 Changes to installation kit and procedure

January 2009 Rev 1.5 added installation instructions

added ECM

Removed Watermark

November 2008 Rev 1.4 fixed weight in specs

October 2008 Rev 1.3 Minor Formatting changes.

September 2008 Rev 1.2 Added Reprogramming the MT3600 Through The I2C Port section.

Added power cord OPNs to table 11 in Appendix D.

September 2008 Rev 1.1 Fixed link to Software package page.

September 2008 Rev 1.0 Initial release

Rev 2.7

Mellanox Technologies

About this Manual

This manual describes the installation and basic use of the Mellanox MTS3600 switch, which is

based on the InfiniScale IV InfiniBand switch device.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting

up the switch platforms listed above.

The manual assumes familiarity with the InfiniBand®Architecture Specification.

Related Documentation

Additional Documentation available from Mellanox:

Table 2 - Reference Documents

Switch Firmware and

Firmware Update

Tools

See

http://www.mellanox.com > Support > Download Firmware Tools

Note that the Switch System described in this manual is based on Mellanox

Technologies’ InfiniScale®IV switch device.

Mellanox OFED

Stack for Linux

User’s Manual

See

http://www.mellanox.com > Support > InfiniBand Software and Drivers

Click “Mellanox OpenFabrics Enterprise Distribution for Linux

(MLNX_OFED)”

Select the Linux User’s Manual

The embedded OS and tools on the CPU in the management module is a

subset of the Mellanox OFED stack.

FabricIT Enterprise

Fabric Management

Software CLI User’s

Manual

Talk to your Mellanox representative for information regarding licensing

and implementation of the FabricIT Enterprise Fabric Management Soft-

ware System.

Mellanox Firmware

Tools (MFT)

User’s Manual

Document # 2329

The MFT (Mellanox Firmware Tools) package is a set of firmware tools.

The manual supplied with this package provides an overview of the firm-

ware its installation and replacement. The MFT can be downloaded with its

documentation at:

http://www.mellanox.com > Support > Download Firmware Tools

QDR InfiniBand Switch Platform User Manual Rev 2.7

Mellanox Technologies 9

Conventions

Throughout this manual, the name MTS3600 and the term switch are used to describe both the 36-

port QSFP 40Gb/s InfiniBand Switch and the 36-port QSFP 20Gb/s InfiniBand Switch, unless

explicitly indicated otherwise.

Mellanox Part Numbering Legend

Table 3 - Part Numbering Legend

Place Field Decoder

MT Mellanox Technologies

SSystem Type S = Switch

MM Model 36 = InfiniScale IV

FF Form factor 00 = Rack mounted switch, 10 = Chassis

CInfiniBand Port Config. G = CX4 DDR, J = CX4 QDR, Q = QSFP QDR, R = QSFP DDR,

-Separator

P# Power Supplies 0 = 0, 1 = 1, 2 = 2

MManagement U = Unmanaged, B = Managed

YModule Config. N = IB modules

RRoHS C = RoHS w/ Exemption, R = RoHS Lead-Free

Overview

Rev 2.7

Mellanox Technologies

1 Overview

Mellanox MTS3600 switch systems provide the highest performing fabric solution by delivering

high bandwidth and low latency to Enterprise Data Centers, High-Performance Computing and

Embedded environments. Networks built with MTS3600 systems can carry converged traffic with

the combination of assured bandwidth and granular quality of service. Built with Mellanox’s 4th

generation InfiniScale® IV InfiniBand switch device, MTS3600 systems provide up to 40Gb/s full

bidirectional bandwidth per port. With 36 ports, these systems are among the densest switching

systems available. These stand-alone switches are an ideal choice for top-of-rack leaf connectivity

or for building small to medium size clusters.

The switch comes pre-installed with all necessary firmware and is configured for standard opera-

tion within an InfiniBand fabric. This switch requires an InfiniBand compliant Subnet Manager

running from one of the hosts or Fabric Management software running on the switch. All that is

required for normal operation is to follow the usual precautions for installation and to connect the

switch to the HCAs. Once connected, the Subnet Management software automatically configures

and begins utilizing the switch.

It is recommended that the Mellanox OpenFabrics software package be installed on all nodes con-

nected to the MTS3600. The software package provides a subnet manager and network manage-

ment tools as well as connectivity software for servers and storage, and is available on the

Mellanox web site. See Chapter 3 for more information.

Basic installation, hot-swapping components and hardware maintenance is covered in “Basic

Operation and Installation” on page 13.

1.1 Serial Number and Product Version Information

The serial number and product version information are found on the label seen in the figure below.

Figure 1: Generic Product label

Also on this label is the GUID identifier for the switch. There is also a label with the MAC for the

management module.

Otros manuales para InfiniScale MTS3600Q-1BNC

Este manual sirve para los siguientes modelos

Tabla de contenidos

Otros manuales de Cambiar de Mellanox Technologies

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Manual de funcionamiento

Mellanox Technologies

Mellanox Technologies SX6518-NR Manual

Mellanox Technologies

Mellanox Technologies Spectrum SN2700 Manual de usuario

Mellanox Technologies

Mellanox Technologies SX10XX Instrucciones de uso

Mellanox Technologies

Mellanox Technologies 1G Switch Air Duct Rail Kit Manual de usuario

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Manual de funcionamiento

Mellanox Technologies

Mellanox Technologies SX1710 Manual de usuario

Mellanox Technologies

Mellanox Technologies InfiniScale MTS3600Q-1UNC Manual de usuario

Mellanox Technologies

Mellanox Technologies MIS5025 Manual de usuario

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Manual de usuario

Mellanox Technologies

Mellanox Technologies Grid Director 4036 Manual de usuario

Mellanox Technologies

Mellanox Technologies MIS5022Q-1BFR Manual de usuario

Mellanox Technologies

Mellanox Technologies SB7700 Manual de usuario

Mellanox Technologies

Mellanox Technologies Spectrum SN2700 Instrucciones de uso

Mellanox Technologies

Mellanox Technologies Vantage 8500 Manual de usuario

Mellanox Technologies

Mellanox Technologies IS5025 Manual de usuario

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX1012B-1BFS Manual de usuario

Mellanox Technologies

Mellanox Technologies InfiniScale MTS3600Q-1BNC Manual de usuario

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX6018F-1SFS Instrucciones de uso

Mellanox Technologies

Mellanox Technologies IS5025D-1BRC Manual de usuario

Mellanox Technologies

Mellanox Technologies SX6710 Instrucciones de uso

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX6005F-1BFS Instrucciones de uso

Mellanox Technologies

Mellanox Technologies MTS3600 Manual de usuario

Mellanox Technologies

Mellanox Technologies SX1012 Manual de usuario

Manuales populares de Cambiar de otras marcas

URC

URC Total Control MFS-8 Manual de usuario

Azbil

Azbil HPQ-D11 Manual de usuario

Sony

Sony DFS-900M Manual de usuario

SICK

SICK i17S Manual de usuario

F&F

F&F SZR-277 Manual de usuario

Shure

Shure UAMS/BK Manual de usuario

Dräger

Dräger X-zone Switch Off Manual de usuario

Asco

Asco Joucomatic 349 Series Manual de usuario

BT Mini Hub Manual de usuario

D-Link

D-Link DES-1226G Manual de usuario

Johnson Controls

Johnson Controls FS80-C Manual de usuario

Alcatel-Lucent

Alcatel-Lucent 7450 ESS-12 Manual de usuario

MicroNet

MicroNet SP1200A Manual de usuario

Ruijie

Ruijie RG-PF2920 Series Manual de instalación y funcionamiento

Altinex

Altinex DA1914SX Manual de usuario

MiLAN

MiLAN MIL-SM2401M Manual de usuario

RJM

RJM MASTERMIND PBC/6X Manual de usuario

Atlas

Atlas MMK-KVM8 Manual de usuario