ECS 2.0 – Add a 60-disk upgrade to a U-Series ECS Appliance
Table of Contents
A 60-disk upgrade adds 15 disks each to four DAE/node pairs.
Determine which racks will be upgraded by referencing the sales order form to locate the PSNT number. The target racks will have this number on a tag prominently affixed to the rear of the rack. Before beginning, determine if the target nodes are running ECS 1.2 (or later) software or an older version of ECS software. The procedure is slightly different depending on the software version
Before beginning the upgrade procedure, make sure the following rules will be followed:
- A DAE must have either 15, 30, 45, or 60 disks.
- The disk drives must be installed according to the layouts shown in the figures below.
- The bottom four DAEs must contain the same number of disk drives.
- The bottom four DAEs must have 60 disks before the rack can be upgraded with four more DAE/node pairs.
- The top four DAEs must contain the same number of disk drives.
- All disk drives in a DAE must be the same size and speed.
- If a DAE has bad (failed) disk drives, the upgrade procedure can proceed without replacing the bad disk drives.
The first figure shows the minimum ECS U-Series Appliance configuration.
- Disk drive slots A0 through A11 are always populated.
- Diks drive slots B0 through B2 are always populated.
Determine which drive slots should have their filler removed and disk drive installed using the figures shown below.
To upgrade from 15 to 30 disks per DAE:
- Populate disk drive slots B3 through B11.
- Populate disk drive slots C0 through C5.
To upgrade from 30 to 45 disk drives:
- Populate disk drive slots C6 through C11.
- Populate disk drive slots D0 through D8.
To upgrade from 45 to 60 disk drives:
- Populate disk drive slots D9 through D11.
- Populate disk drive slots E0 through E11.
cs_hal list disks command
In the abbreviated cs_hal list disks output below, notice the different types of rows:
- The first three rows represent the RAID structure of the node's two internal drives.
- The next row shows a GOOD storage drive in the SMART Status column. Device names, slot numbers, and serial numbers can be used in cs_hal commands as long as they are unique. The enclosure (DAE) name can also be used in cs_hal commands.
- The next row represents a FAILED storage drive. The Partition Name indicates the disk on the corresponding drive is assigned to the Object service, the disk is formatted, and the disk health is Bad.
Use the value in the SMART Status column when determining the current health of the disk drive.
- The next row represents a SUSPECT storage drive.
- The last row represents either an empty slot or an undetectable drive.
- The DiskSet column is reserved for future use.
[root@layton-cyan ~]# cs_hal list disks Disks(s): SCSI Device Block Device Enclosure Partition Name Slot Serial Number SMART Status DiskSet ----------- ------------ ----------- ----------------------------------- ---- ------------------- -------------- ------------ n/a /dev/md0 RAID vol n/a n/a not supported n/a /dev/sg4 /dev/sdb internal 0 KLH6DHXJ GOOD /dev/sg5 /dev/sdc internal 1 KLH6DM1J GOOD /dev/sg8 /dev/sdf /dev/sg0 Object:Formatted:Good A08 WCAW32601327 GOOD /dev/sg9 /dev/sdg /dev/sg0 Object:Formatted:Bad A09 WCAW32568324 FAILED: self-test fail; read element; /dev/sg10 /dev/sdh /dev/sg0 Object:Formatted:Suspect A10 WCAW32547329 SUSPECT: Reallocated_Sector_Count(5)=11 ... unavailable /dev/sg0 E05 internal: 2 external: 30 total disks: 32
cs_hal list daes
Use this command to find the enclosure ID of the DAE paired with the node.
# cs_hal list daes Enclosure(s): SCSI Device Ext Disks ----------- --------- /dev/sg2 15 total: 1
cs_hal list node
Use this command to find the name of the node you are working on. This lets you identify the DAE associated with the node by referring to a diagram.
# cs_hal list node Node(s): Name HBAs Enclosures Int Disks Ext Disks ---------- ---- ---------- --------- --------- provo-sage 2 1 1 16Back to Top
Before you begin
Install disk drives on one node/DAE pair at a time.
- To access the ECS rack using the private (192.168.219.xxx) network from a laptop:
- From the rear of the rack, locate the 1 GbE private switch network ports by opening the rear door.
- On the Arista 1 GbE (turtle) switch, attach a network cable from your laptop to port 24 on the switch.
- Set the network interface on the laptop to the static address192.168.219.99, subnet mask 255.255.255.0, with no gateway required.
- Verify that the temporary network between the laptop and rack's private management network is functioning by using the
C:\>ping 192.168.219.1 Pinging 192.168.219.1 with 32 bytes of data: Reply from 192.168.219.1: bytes=32 time<1ms TTL=64 Reply from 192.168.219.1: bytes=32 time<1ms TTL=64 Reply from 192.168.219.1: bytes=32 time<1ms TTL=64 Reply from 192.168.219.1: bytes=32 time<1ms TTL=64 Ping statistics for 192.168.219.1: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0msIf 192.168.219.1 does not answer, try 192.168.218.2. If neither responds, verify the laptop IP/subnet mask, network connection, and switch port connection.
- Start an ssh session with the first node.
- Use the
cs_hal list disks command:
# cs_hal list disksDisk drive slots listed as occupied in the output should match your expectations from the planning phase.
- Locate the correct DAE:
- Use the
cs_hal list node command to verify the name of the node:
# cs_hal list node Node(s): Name HBAs Enclosures Int Disks Ext Disks ---------- ---- ---------- --------- --------- provo-sage 2 1 1 16
- Use the following wiring diagram to physically identify the correct DAE:
- Use the cs_hal list node command to verify the name of the node:
- Open the DAE which you identified with the wiring diagram.
- Remove fillers and populate disk drives in slots as shown in the appropriate disk layout diagram.
- Close the DAE and wait 2 minutes.
- Use the
cs_hal list disks command to verify that the node recognized the correct number of disk drives.
# cs_hal list disks | grep -i external external: <number of disks>
Where <number of disks> equals 30, 45, or 60.
- If the result is not the expected result, use the
cs_hal list disks command to determine which disk drives the node cannot recognize. Use the appropriate disk layout diagram for comparison.
# cs_hal list disks
- If a disk drive is not recognized in the output:
- Open up the DAE and verify that a disk drive was added to that slot.
- If a disk drive is present, reseat it or add a disk drive to the slot.
- Run the cs_hal list disks command again.
- If the disk drive is still not recognized in the output, or the output lists it as FAILED, order a replacement disk drive.
- Repeat steps 1 through 9 on the remaining three node/DAE pairs.
- Wait 10 minutes after completing the upgrade on the final node. Then run the
cs_hwmgr command to verify that all drives are visible to the system.
# viprexec "cs_hwmgr ListDrives | grep -i count" Output from host : ???.???.???.1 Persistent drive count: <number of disks> Output from host : ???.???.???.2 Persistent drive count: <number of disks> Output from host : ???.???.???.3 Persistent drive count: <number of disks> Output from host : ???.???.???.4 Persistent drive count: <number of disks>
The viprexec command runs the command placed in quotes on each node in the current ECS rack. Caution should be taken before using this command with other node-level commands.
- The final number of the displayed IP address (.1 and so on) indicates the node number in the rack.
- <number of disks> equals 30, 45, or 60.
- If this output does not match the expected drive count, wait 10 more minutes and repeat the viprexec command. If the drive count is still not correct, ssh into the host with the incorrect drive count and repeat step 11.