C H A P T E R  4

Connecting Your Fibre Channel Array

This chapter describes procedures for cabling the Sun StorEdge 3510 FC Array and for connecting the array to power and to network devices.

The following topics are covered in this chapter:

Before you connect the Sun StorEdge 3510 FC Array to the network, position the Sun StorEdge 3510 FC Array in the rack, cabinet, or other location where it will be used.



caution icon

Caution - When positioning the array, do not block the air vents at the front or back of the unit. Follow all safety precautions specified in the Sun StorEdge 3000 Family Safety, Regulatory, and Compliance Manual.





caution icon

Caution - When you power off an array, wait five seconds before you power it back on. If you power the array off and on too quickly, unexpected results can occur.




4.1 Converting Your Front Bezel Locks So the Keys Cannot Be Removed

The bezel on your array has two locks whose keys can be removed when the locks are in either the locked or open position. It is possible to reconfigure the locks so that the keys cannot be removed.

 FIGURE 4-1 Front Bezel and Front Bezel Locks of an Array

This figure displays the front bezel and the bezel locks on the right and left sides of the bezel.

To change the locks so the keys cannot be removed, follow these steps:

1. Remove the bezel by gently pivoting the swing arms out of their ear sockets.

See Removing the Front Bezel and Ear Caps for step-by-step instructions on how to remove the bezel.

2. Make sure the key is in the locked position, with the pawl extending horizontally past the edge of the bezel.

 FIGURE 4-2 Sequence of Steps to Change Front Bezel Locks So Keys Cannot Be Removed

Illustration showing the five steps in changing front bezel locks so the keys cannot be removed.

3. Hold the key in place and use a 12 mm or 3/8-inch nut driver to remove the locking nut that holds the pawl in place, as shown in the first panel of FIGURE 4-2.



caution icon

Caution - Be sure to hold the key in place. Otherwise there is a risk of breaking the small tab on the lock that serves as a stop.



4. Lift the pawl off the threaded part of the lock body, as shown in the second panel of FIGURE 4-2.

5. Set the pawl aside, face up, so that you can remember its orientation when you replace it.

6. Use the key to turn the lock 180 degrees, as shown in the third panel of FIGURE 4-2

7. Replace the pawl in the same orientation as before, as shown in the fourth panel of FIGURE 4-2.

8. Hold the key in place and use the nut driver to refasten the locking nut that holds the pawl in place, as shown in the fifth panel of FIGURE 4-2. Be careful not to cross-thread the nut.



caution icon

Caution - Be sure to hold the key in place. Otherwise there is a risk of breaking the small tab on the lock that serves as a stop.



9. Replace the bezel.



Note - To convert your bezel locks so that the keys can be removed, repeat this procedure.




4.2 Fibre Channel Array Connections

FIGURE 4-3 identifies the hardware connections on the back of a dual-controller FC array.

 FIGURE 4-3 Hardware Connections on the Back of a Dual-Controller FC Array

Figure shows the back panel of a dual-controller FC array with the hardware connections identified.

Management is in-band through fibre host connections and out-of-band through the serial port and Ethernet port on the back of each controller.


4.3 Connecting the Chassis to an AC Power Outlet

When you connect the AC power cords, you should install the provided two cord locks at the same time. The provided AC cord locks are used to securely fasten the AC cable connectors.



caution icon

Caution - If the array is connected to AC power sources not within the designated 90-135, 180-265 VAC PFC range, damage might occur to the unit.





Note - To ensure power redundancy, be sure to connect the two power supply modules to two separate circuits (for example, one commercial circuit and one UPS).



To connect the AC power cords, perform the following procedure:

1. Obtain an appropriate AC power cable.

2. Use a screwdriver to remove the screw and cylindrical standoff from one of the two cord locks provided and set them aside for reassembly later.

3. Slide the cord lock over the AC power connector.

4. Hold the cylindrical standoff between the two screw-holes on the flanges of the cord lock.

5. Insert the screw into the first screw-hole, through the standoff, and then into the threaded screw-hole on the other flange.

6. Tighten the screw with a screwdriver until the flanges bottom out on the cylindrical standoff.

7. Push the power cord into the power supply receptacle until it is firmly seated.

8. Push the green ejector handle forward until it is seated against the power supply.

9. Turn the thumbscrew of the green ejector handle clockwise until it is finger-tight to secure the handle and the cord lock.

 FIGURE 4-4 Installed Cord Lock

Figure shows the sequence of steps to install a cord lock.

10. Repeat this procedure for the second cord lock and second power cable.


4.4 Connecting the Chassis to DC Power Outlets

Two DC power cords are packaged with each DC array. To connect the DC power cords, perform the following procedure:

1. Connect a DC power cable to the first power supply and to a power outlet.



Note - Use only the DC power cables provided with the array.



2. Check the DC cable part number and wire labels carefully before connecting the cable to the source.

TABLE 4-1 DC Cable Wiring for Cable 35-00000148

Pin Number

Voltage

Color

A3

Return

Red

A2

GND (Chassis Ground)

Green/Yellow

A1

-48vdc

Black


TABLE 4-2 DC Cable Wiring for Cable 35-00000156

Pin Number

Voltage

Color

A3

L+

Red

A2

GND (Chassis Ground)

Green/Yellow

A1

L-

White




caution icon

Caution - If the Sun StorEdge 3510 FC Array is connected to DC power sources not within the designated -48V DC (-36 VDC to -72 VDC) range, damage might occur to the unit.





Note - To ensure power redundancy, be sure to connect the two power supply modules to two separate circuits (for example, one commercial circuit and one UPS).





Note - To extend the length of the DC power cable as needed, strip the last 1/4" of the cable, insert the stripped end into a provided Panduit tube, and crimp the tube.



3. Tighten the cable-locking screws to attach the cable securely to the power supply power outlet.

4. Connect the second power cable to the second power supply and to a second power outlet. Tighten the cable-locking screws.

If one power supply fails, the second power supply automatically takes the full load.


4.5 Powering Up and Checking LEDs

Perform the initial check of the array according to the following procedure:

1. Connect two AC (or DC) power cables to the power and fan modules on the back of the array.

2. Power on the array by turning on each power switch.

See Power-On Sequence for the power-up sequence to use when operating RAID arrays and expansion units. See Connecting the Sun StorEdge 3510 FC JBOD Array to a Host Computer for the power-up sequence to use when operating standalone JBODs directly attached to hosts.

3. Check for the following LED activity:

All front panel LEDs turn solid green to indicate good operation.

 FIGURE 4-5 Front Panel With LEDs Displayed

Figure shows the front panel of an FC array with the LEDs displayed.

See the chapter Checking LEDs for more information about your array's LEDs.


4.6 Reviewing and Changing SFP Ports (Optional)

Each I/O controller module has six ports that accept SFP transceivers. These ports are labeled FC0 through FC5. The default configurations do not include an SFP connector on every SFP port. You might want to add or rearrange SFPs, depending on the configuration mode (loop or point-to-point), number of planned host connections, the necessary number of redundant connections to hosts, and the number of expansion units needed.

The SFP is a single-port, optical SFP transceiver for multimode (short wave) or single-mode (long wave) use. They are compatible with the Small Form Factor Pluggable Multi-Sourcing Agreement (MSA, Sep. 2000), and 1x and 2x Fiber Channel. The optical connector used is the low-profile LC connector.

SFPs are Sun field-replaceable units (FRUs) and can be ordered from Sun Microsystems. These SFPs have been selected and tested to provide the necessary reliability and performance. SFPs from other vendors are not supported.

To see an example of an array that has SFP connectors in all the SFP ports of the RAID array and the upper expansion unit, refer to the illustration RAID Array Attached to Two Hosts and Two Expansion Units.

To review various configuration options, see the discussions following Configuration Overview. Refer also to the configuration options presented in the Sun StorEdge 3000 Family Best Practices for the Sun StorEdge 3510 FC Array.

4.6.1 Port Connectivity in a Dual-Controller Array

Drive channels connect to the internal drives in the array and can also connect to drives in external expansion units. Configured for redundancy, each controller in a dual-controller RAID array has two adjacent dedicated drive channels on one loop to load-balance I/O operations (see FIGURE 4-6). Each drive channel has two SFP ports that can be connected to expansion units. Drive channels 2 and 3 access all the disk drives and are interconnected to load-balance I/O operations.

 FIGURE 4-6 Dedicated Drive Channels 2 (on the Upper Controller) and 3 (on the Lower Controller) in a Dual-Controller Array

Figure shows the back of an array with each controller configured with two drive channel ports connected on a single loop.

The controller module in slot A (the upper slot) houses drive channel 2, which connects to the 12 internal disk drives through their A ports. The controller module in slot B (the lower slot) houses drive channel 3, which connects to the 12 internal disk drives through their B ports.

Host channels connect to host computers either directly or through storage switches or other devices. In a default dual-controller RAID configuration, each controller has four host channels, channels 0, 1, 4, and 5 (FIGURE 4-7). Port bypass circuits connect each pair of host SFP ports on a host channel; as a result, each host channel accesses both controllers.

 FIGURE 4-7 Host Channels on a Dual-Controller Array

Figure shows the back of an array with each controller in the default configuration with each host channel looped between the two controllers.

4.6.2 Default SFP Placement

In dual-controller arrays, SFPs are initially plugged into one of each pair of host and drive ports. The default port connections are:

This configuration provides connections to all four host channels as well as to both drive channels.

 FIGURE 4-8 Default Dual-Controller SFP Placement

Figure shows the default dual-controller SFP placement.

In the default single-controller array, SFPs are initially plugged into FC0, FC1, FC4, and FC5. No SFPs are plugged into the drive channels. This configuration is appropriate for connecting to up to four hosts or fibre switches, with no connection to expansion units.

 FIGURE 4-9 Default Single-Controller SFP Placement

Figure shows the default single-controller SFP placement.

In a default JBOD expansion unit array, SFPs are initially plugged into the upper left port and lower right port.

 FIGURE 4-10 Default JBOD Expansion Unit SFP Placement

Figure shows the default JBOD Expansion Unit SFP placement.

4.6.3 Changing Your SFP Configuration

Fibre Channel arrays use SFP connectors to attach to hosts and expansion units. These SFP connectors resemble the one shown in FIGURE 4-11, with a single connector at the end that plugs into an SFP port on the array or expansion unit chassis, and a duplex jack into which you insert a cable to make the connection.

 FIGURE 4-11 Typical SFP Connector Used to Connect Cables to Chassis SFP Ports

Figure shows typical SFP connector used to connect cables to array chassis ports.


4.7 Configuring a COM Port to Connect to a RAID Array

The RS-232 COM port on either controller module is used to configure and monitor the RAID array. It can be connected to a VT100 terminal or terminal emulation program, to a terminal server, or to the serial port of a server.

1. Use a null modem serial cable to connect the COM port of the RAID array to the serial port on a host workstation.

A null modem serial cable is included in your package contents.

2. Set the serial port parameters on the workstation as follows:

For platform-specific details, see the appendix that describes the operating system your server uses.


4.8 Setting an IP Address

The controller Ethernet port offers interactive out-of-band management through two interfaces:

To access the array using the Ethernet port, you must set up an IP address for the controller. You can set the IP address by typing in values for the IP address itself, the subnet mask, and the IP address of the gateway manually. If your network is using a Reverse Address Resolution Protocol (RARP) server or a Dynamic Host Configuration Protocol (DHCP) server to automatically configure IP information for devices on the network, you can specify the appropriate protocol instead of typing in the information manually.



caution icon

Caution - If you assign an IP address to an array to manage it out-of-band, for security reasons make sure that the IP address is on a private network rather than a publicly routable network.



There are two main reasons for placing your arrays on a private subnet:

Since it is not possible to predict the side-effects of all current and future security software products, place your Sun StorEdge 3310 SCSI arrays and Sun StorEdge 3510 FC arrays on private subnets.

To set the IP address, subnet mask, and gateway address of the RAID controller, perform the following steps.

1. Access the array through the COM port on the controller module of the array.

2. From the Main Menu, choose "view and edit Configuration parameter" and press Return.

3. Choose "Communication Parameters" and press Return.

4. Choose "Internet Protocol (TCP/IP)" and press Return.

5. Choose the chip hardware address.

6. Choose "Set IP Address" and press Return.

7. Type in the desired IP address, subnet mask, and gateway address, choosing each menu option in turn.

If your network sets IP addresses using a RARP server, simply type RARP rather than an IP address and do not type in a subnet mask or gateway address. If your network sets IP addresses using a DHCP server, simply type DHCP rather than an IP address and do not type in a subnet mask or gateway address.

8. Press Esc to continue.

A confirmation prompt is displayed.

Change/Set IP Address ?

9. Select Yes to continue.



Note - You must reset the controller for the configuration to take effect.



You are prompted to reset the controller.

10. Select Yes to reset the controller.

The controller takes a few minutes to format a small storage sector on each physical drive before logical drives can be successfully initialized.


4.9 Setting Up Out-of-Band Management Over Ethernet

The Ethernet connection enables you to configure and monitor RAID arrays and expansion units remotely by using the telnet command to access the firmware application on the array and by using the Sun StorEdge Configuration Service software.



caution icon

Caution - If you assign an IP address to an array to manage it out-of-band, for security reasons make sure that the IP address is on a private network rather than a publicly routable network.



1. To access the RAID array over an Ethernet connection, first set the IP address of the RAID array by using the RAID array's COM port and the RAID firmware. See Setting an IP Address for more information.

2. Connect the RAID array's Ethernet port on each controller to the network.



Note - In a dual-controller RAID array, be sure to connect both Ethernet ports to the network, which provides failover if one controller fails.



3. To use the firmware application program from the host server, connect to the IP address of the RAID array controller with the following command:

# telnet IP-address

4. Use the Solaris operating system tip command or a terminal emulation program to access the firmware application program. See Configuring a COM Port to Connect to a RAID Array for more information.

5. Press Ctrl-L to refresh the screen and view the Main Menu.



Note - If you reset the controller during a telnet session, you are disconnected from the RAID array. Use the telnet command to log back in to the array.



6. To connect the Sun StorEdge Configuration Service program (on a host server) to a RAID array that has an IP address, refer to the out-of-band management instructions in the Sun StorEdge 3000 Family Configuration Service User's Guide.

The same document's "Email and SNMP" appendix provides information about configuring Sun StorEdge Configuration Service software to use Simple Network Management Protocol (SNMP) traps and Management Information Bases (MIBs) to provide information to other out-of-band enterprise management software. The "Monitoring the Array" chapter explains the use of Sun StorEdge Configuration Service agents to redirect event messages into host system logs.


4.10 Cabling to Expansion Units

FIGURE 4-12 shows an example of a RAID array connected to two expansion units. You can connect up to eight expansion units to a RAID array in certain configurations. These configurations are described in the Sun StorEdge 3000 Family Best Practices Manual.

The connections to the expansion units are designed to keep all A drive ports on the same loop and all B drive ports on the same loop. Several cabling configurations are possible, each with its own advantages and disadvantages. Refer to the Sun StorEdge 3000 Family Best Practices Manual for the Sun StorEdge 3510 FC array for more information about the suitability of various configurations for various application requirements, as well as for information about configurations involving more than two expansion units.

 FIGURE 4-12 RAID Array Attached to Two Hosts and Two Expansion Units

Figure shows cabling for RAID array and two expansion units.

In the RAID array, two of the unused SFP host ports could be used to provide redundant pathing to the two servers, and the other four unused SFP host ports could be connected to two more servers in a redundant configuration.

Similarly, you can connect expansion units to other channels (which are separate from channels 2 and 3) if you configure channels 0, 1, 4, or 5 as drive channels. For details, refer to Configuring FC Channels as Host or Drive (Optional).



caution icon

Caution - To avoid drive assignment conflicts, make sure that each connected array and expansion unit uses a different loop ID, as described in the following section.



4.10.1 Scaling Beyond 36 Disks



Note - High capacity Sun StorEdge 3510 FC array configurations with more than two expansion enclosures are supported, with certain limitations. For example, only the sequential cache optimization mode can be utilized in most situations. Other important limitations may exist.



Sun StorEdge 3510 FC arrays typically allow the connection of up to two expansion units to support a maximum of 36 disks. However, you can create larger configurations that support as many as eight expansion units and up to 108 disks if you use the guidelines in this section.

Carefully consider the limitations of these special high-capacity configurations. Using multiple Sun StorEdge 3510 FC arrays connected to the same SAN normally provides significantly better performance than one high-capacity configuration.

Limitations

Refer to the Sun StorEdge 3000 Family Best Practices Manual for the Sun StorEdge 3510 FC array for cabling diagrams for configurations ranging from three to eight expansion units.



Note - Large configurations may require the use of one or more optional extended-length cables, part number X9732A. Other items may also be required. Refer to the Sun StorEdge 3000 Family FRU Installation Guide for information about supported cables, SFPs, and other user-replaceable items.




4.11 Setting Loop IDs on Expansion Units

When an expansion unit is attached to a RAID array, unique, hard-assigned loop IDs are assigned to each expansion unit drive. A loop ID is the decimal version of an AL_PA. The lowest number loop ID is the lowest priority address on the loop.

On the left front side of an expansion unit, an ID switch is used to set the loop IDs for the disk drives to a different range of values so that the same IDs are not repeated on the same loop.

Setting the loop ID on an array requires dropping the front bezel out of the way and removing the small vertical plastic cap on the left side of the bezel that cover the left rackmount tab. Rackmount tabs are often referred to as "ears."

 FIGURE 4-13 Front Bezel and Front Bezel Locks of an Array

This figure displays the front bezel, bezel locks, and plastic ear cap covering FC loop ID switch.

1. Use the provided key to unlock both bezel locks.

2. Grasp the front bezel cover on both sides and pull it forward and then down.

3. Remove the plastic cap from the left ear of the array.

a. Squeeze both sides of the cap at the top and the bottom.

b. Turn the cap toward the center of the array until it disengages and pull it free.



caution icon

Caution - To avoid damage to the cap, do not pull the cap forward directly or pull from only its top or bottom.



 FIGURE 4-14 ID Switch Located on the Left Front Side of the Array or Expansion Unit

Figure shows ID Switch Located on the Left Front Side of the Array or Expansion Unit.

4. Press the upper or lower switch button to change the ID number so that each connected RAID array and expansion unit uses a different loop ID.



Note - By default, the ID switch on all RAID arrays and expansion units is set to 0, where the range of drive IDs is automatically 0 to 11 for 12 drives (the IDs 12-15 are ignored).





caution icon

Caution - Ensure that the loop IDs of expansion units do not duplicate the loop IDs of other connected expansion units or RAID arrays.



The ID switch offers eight ID ranges. Each set contains 16 IDs (the last four IDs in each range are ignored). These ranges are shown in TABLE 4-3.

TABLE 4-3 ID Switch Settings for Expansion Units

ID Switch Setting

Range of IDs

0

0-15

1

16-31

2

32-47

3

48-63

4

64-79

5

80-95

6

96-111

7

112-125


For an example of properly set loop IDs, consider the configuration shown in FIGURE 4-12. You need to make sure the RAID array and each of the two expansion units have been assigned different loop IDs. If you set each of the loop ID switches so that the RAID array is assigned loop ID 0, expansion unit 1 has loop ID 1, and expansion unit 2 has loop ID 2. The range of IDs assigned to the drives is shown in TABLE 4-4.

TABLE 4-4 Sample Array and Expansion Units With Different Loop IDs and Drive IDs

Fibre Channel Device

Loop ID Switch Setting

Range of Drive IDs

RAID array

0

0-15

Expansion unit 1

1

16-31

Expansion unit 2

2

32-47


5. Prepare the plastic left ear cap for replacement by aligning the inside round notches of the cap with the round cylindrical posts (ball studs) on the ear.

6. Push the top and bottom of the ear cap onto the ear, pressing in on the top side toward the center of the array first.

7. Continue pushing the top and bottom of the ear cap onto the ear, pressing on the side toward the outside of the array.

Do not use force when placing a cap on an ear.

8. Lift the bezel into position and press it onto the front of the chassis until it is flush with the front.

9. Use the key to lock both bezel locks.


4.12 Connecting Ports to Hosts

In a default array configuration, channels 0, 1, 4, and 5 are host channels, so you can directly connect an FC array to four host computers. SFP connectors are plugged into channels 0 and 4 on the upper controller and channels 1 and 5 on the lower controller for this purpose.

If you want to connect an FC array to more than four host computers without changing the default configuration, you can connect these four host channels to ports on storage switches in a storage area network (SAN) configuration.



Note - Connecting the Sun StorEdge 3510 RAID array to Fibre Channel HBAs on the same channel that use different speeds (1 Gbyte & 2 Gbyte) is not supported. You can, however, mix 1-Gbyte and 2-Gbyte Fibre Channel HBAs on different channels. This limitation is due to the design of the Sun StorEdge 3510 RAID array's port bypass circuitry and the inability of Fibre Channel to support auto-negotiation in a multi-drop loop configuration.



Use fiber-optic cables to connect from one to four host channels to Fibre Channel HBAs on your host computers or to other devices such as storage switches:

1. Connect a fiber-optic cable to an HBA or FC port on each host or storage switch you want to connect to the array.

2. Connect the SFP connector at the other end of each of these fiber-optic cables to host channel SFP connectors on the back of the array.

If there is no SFP connector in the port you want to use, first insert an SFP connector into the port as described in Changing Your SFP Configuration.


4.13 Power-On Sequence

Power on the equipment in the following order, so that the host computer discovers all connected arrays:

a. Expansion units

b. FC arrays

c. Host computers

When you power on a controller, the Tip connection window displays a series of messages such as the following:

3510         Disk Array is installed with 1024MBytes SDRAM
Total SCSI channels: 6
SCSI channel: 0 is a host channel, id: 40
SCSI channel: 1 is a host channel, id: 41
SCSI channel: 2 is a drive channel, id: 14, 15
SCSI channel: 3 is a drive channel, id: 14, 15
SCSI channel: 4 is a host channel, id: 70
SCSI channel: 5 is a host channel, id: 71
Scanning SCSI channels. Please wait a few moments!
Preparing to restore saved persistent reservations. Type 'skip' to skip:

At this point if you type skip the channels will not be scanned. If you do not type anything, the channels are scanned and persistent group reservations are restored.