ECS 2.0 – Add a 60-disk upgrade to a U-Series ECS Appliance

Table of Contents

60-disk upgrade planning

Planning considerations for a 60-disk upgrade for an ECS U-Series appliance

A 60-disk upgrade adds 15 disks each to four DAE/node pairs.

Determine which racks will be upgraded by referencing the sales order form to locate the PSNT number. The target racks will have this number on a tag prominently affixed to the rear of the rack. Before beginning, determine if the target nodes are running ECS 1.2 (or later) software or an older version of ECS software. The procedure is slightly different depending on the software version

Before beginning the upgrade procedure, make sure the following rules will be followed:

The first figure shows the minimum ECS U-Series Appliance configuration.

15-disk layout

Determine which drive slots should have their filler removed and disk drive installed using the figures shown below.

To upgrade from 15 to 30 disks per DAE:

30-disk layout

To upgrade from 30 to 45 disk drives:

45-disk layout

To upgrade from 45 to 60 disk drives:

60-disk layout

Back to Top

cs_hal commands

Introduces the cs_hal commands used in this procedure.

cs_hal list disks command

In the abbreviated cs_hal list disks output below, notice the different types of rows:

[root@layton-cyan ~]# cs_hal list disks
Disks(s):
SCSI Device Block Device Enclosure   Partition Name                      Slot Serial Number       SMART Status   DiskSet
----------- ------------ ----------- ----------------------------------- ---- ------------------- -------------- ------------
n/a         /dev/md0     RAID vol    n/a                                 n/a  not supported       n/a
/dev/sg4    /dev/sdb     internal                                        0    KLH6DHXJ            GOOD
/dev/sg5    /dev/sdc     internal                                        1    KLH6DM1J            GOOD
/dev/sg8    /dev/sdf     /dev/sg0    Object:Formatted:Good               A08  WCAW32601327        GOOD
/dev/sg9    /dev/sdg     /dev/sg0    Object:Formatted:Bad                A09  WCAW32568324        FAILED: self-test fail; read element;
/dev/sg10   /dev/sdh     /dev/sg0    Object:Formatted:Suspect            A10  WCAW32547329        SUSPECT: Reallocated_Sector_Count(5)=11
...
unavailable              /dev/sg0                                        E05    

   internal: 2
   external: 30

total disks: 32

cs_hal list daes

Use this command to find the enclosure ID of the DAE paired with the node.

# cs_hal list daes

Enclosure(s):
SCSI Device Ext Disks
----------- ---------
/dev/sg2     15

total: 1

cs_hal list node

Use this command to find the name of the node you are working on. This lets you identify the DAE associated with the node by referring to a diagram.

# cs_hal list node

Node(s):
Name       HBAs Enclosures Int Disks Ext Disks
---------- ---- ---------- --------- ---------
provo-sage 2    1          1         16

Back to Top

Perform a 60-disk upgrade

Add 15 disks to four disk array enclosures (DAEs) to complete a 60-disk upgrade.

Before you begin

Install disk drives on one node/DAE pair at a time.

Procedure

  1. To access the ECS rack using the private (192.168.219.xxx) network from a laptop:
    1. From the rear of the rack, locate the 1 GbE private switch network ports by opening the rear door.
    2. On the Arista 1 GbE (turtle) switch, attach a network cable from your laptop to port 24 on the switch.

      Locate port 24 on the private 1 GbE switch

    3. Set the network interface on the laptop to the static address192.168.219.99, subnet mask 255.255.255.0, with no gateway required.
    4. Verify that the temporary network between the laptop and rack's private management network is functioning by using the ping command.
      C:\>ping 192.168.219.1 
      Pinging 192.168.219.1 with 32 bytes of data:
      Reply from 192.168.219.1: bytes=32 time<1ms TTL=64
      Reply from 192.168.219.1: bytes=32 time<1ms TTL=64
      Reply from 192.168.219.1: bytes=32 time<1ms TTL=64
      Reply from 192.168.219.1: bytes=32 time<1ms TTL=64
      
      Ping statistics for 192.168.219.1:
         Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
      Approximate round trip times in milli-seconds:
         Minimum = 0ms, Maximum = 0ms, Average = 0ms
      
      Note Image
      If 192.168.219.1 does not answer, try 192.168.218.2. If neither responds, verify the laptop IP/subnet mask, network connection, and switch port connection.

    5. Start an ssh session with the first node.
  2. Use the cs_hal list disks command:
    # cs_hal list disks
    Note Image
    Disk drive slots listed as occupied in the output should match your expectations from the planning phase.

  3. Locate the correct DAE:
    1. Use the cs_hal list node command to verify the name of the node:
      # cs_hal list node
      
      Node(s):
      Name       HBAs Enclosures Int Disks Ext Disks
      ---------- ---- ---------- --------- ---------
      provo-sage 2    1          1         16
    2. Use the following wiring diagram to physically identify the correct DAE:

      U-Series ECS Appliance wiring

  4. Open the DAE which you identified with the wiring diagram.
  5. Remove fillers and populate disk drives in slots as shown in the appropriate disk layout diagram.
  6. Close the DAE and wait 2 minutes.
  7. Use the cs_hal list disks command to verify that the node recognized the correct number of disk drives.
    # cs_hal list disks | grep -i external
    
    external: <number of disks> 

    Where <number of disks> equals 30, 45, or 60.

  8. If the result is not the expected result, use the cs_hal list disks command to determine which disk drives the node cannot recognize. Use the appropriate disk layout diagram for comparison.
    # cs_hal list disks
  9. If a disk drive is not recognized in the output:
    1. Open up the DAE and verify that a disk drive was added to that slot.
    2. If a disk drive is present, reseat it or add a disk drive to the slot.
    3. Run the cs_hal list disks command again.
    4. If the disk drive is still not recognized in the output, or the output lists it as FAILED, order a replacement disk drive.
  10. Repeat steps 1 through 9 on the remaining three node/DAE pairs.
  11. Wait 10 minutes after completing the upgrade on the final node. Then run the cs_hwmgr command to verify that all drives are visible to the system.
    # viprexec "cs_hwmgr ListDrives | grep -i count"
    
    Output from host : ???.???.???.1
    Persistent drive count: <number of disks>
     
    Output from host : ???.???.???.2
    Persistent drive count: <number of disks>
     
    Output from host : ???.???.???.3
    Persistent drive count: <number of disks>
     
    Output from host : ???.???.???.4
    Persistent drive count: <number of disks>
    

    Where:

    • The final number of the displayed IP address (.1 and so on) indicates the node number in the rack.
    • <number of disks> equals 30, 45, or 60.
    Note Image
    The viprexec command runs the command placed in quotes on each node in the current ECS rack. Caution should be taken before using this command with other node-level commands.

  12. If this output does not match the expected drive count, wait 10 more minutes and repeat the viprexec command. If the drive count is still not correct, ssh into the host with the incorrect drive count and repeat step 11.

Results

The node and ECS system automatically detect the disk drives and initialize them for use. To validate that the disks are recognized as good by the object sevice, use the cs_hwmgr disk --list-by-service command.

Back to Top