Understanding and Setting Up Metrics-Based Port Selection in ViPR Controller

Table of Contents

Overview of metrics-based port selection

Learn how to define maximum performance-based limits for ports and how those limits are used by ViPR Controller for allocating new ports. Allocating new ports based on performance metrics, computed metrics, and user-defined maximum limits is supported on VMAX, VNX for Block, and Hitachi Data Systems (HDS).

Several performance-based metrics are collected from VMAX, VNX for Block, and HDS and are used to determine:
  • Port percent busy.
  • CPU percent busy.
Two additional metrics are also computed:
  • Number of initiators using a storage port.
  • Number of volumes using a storage port.
These metrics are then used to allocate new ports to avoid:
  • Ports that are overloaded with too many volumes or too high of an I/O load.
  • Ports that reside on CPUs where the CPU percent busy is too high or the CPU is servicing too many volumes.
  • Allocating more storage on arrays that are overloaded.

For information on how ViPR Controller allocates new ports, based on these metrics, see How does ViPR Controller select a port when using performance metrics.

Before ViPR Controller can allocate new ports based on performance metrics, there are configuration requirements you must set up on HDS, and VNX for Block storage systems. For configuration requirements refer to the: ViPR Controller Virtual Data Center Requirements and Information Guide on the ViPR Controller Product Documentation Index.

You must also enable ViPR Controller for the collection of metrics from the storage arrays, as described in Set up metering prerequisites in ViPR Controller.

Back to Top

How does ViPR Controller select a port when using performance metrics

ViPR Controller takes averages of the performance-based metrics collected from the storage arrays and the number of initiators and volumes that it has already allocated to ports, and then compares these metrics to maximum limits (ceilings) that you configure fo determine which ports to select.

Metric calculations and averages

Several performance-based metrics are collected from VMAX, VNX for Block, and HDS and are used to determine:

In addition, two additional metrics are computed byViPR Controller :
  • Number of initiators ViPR Controller has already assigned to a storage port
  • Number of volumes ViPR Controller has already assigned to a storage port
    Note Image
    On VMAX2, the number of volumes is computed across both ports on a director.

These numbers may not reflect all exports done outside of ViPR Controller.

Averaging the metrics values

The metrics collected for CPU Percent Busy and Port Percent Busy are averaged over time so that they reflect a relatively long term view of whether the port is overloaded. The system administrator can control this averaging process. There are three important time periods:
  • The Metering Interval controls how often metering records will be read from the storage arrays. The default time period for this is one hour. This can be reduced to 30 minutes, or increased to multiple hours. To get accurate metrics on heavily loaded ports, it may be necessary to decrease the metering interval to 30 minutes, although this will cause increased ViPR Controller load for systems with many arrays. Increasing the metering interval will reduce the load. It is not recommended to have a metering interval greater than four hours. For information on how to set Metering Interval, see Set up metering prerequisites in ViPR Controller.
  • The Days to Average Utilization, one of the ViPR Controller Port Allocation parameters, controls how long various samples are averaged together using a modified moving average. The default averaging period is 1 day, but you can configure the period from 1 to 30 days. The longer the averaging period, the less an instantaneous change in load is reflected in the average, and the less affect a current sample will have on the average. After the averaging period has been completed, a new average starts and will be computed. For information on how to set Days to Average Utilization, see Change the default port allocation parameters.
  • At the end of each averaging period, the modified moving average is added into a longer term Exponential Moving Average (EMA) that is calculated for each metric. The purpose of the EMA is to retain history about the port's utilization over time. An EMA is used because it weights recent values higher, and past values with exponentially decreasing weights as the sample's age increases. In that way recent port utilization is more important than past utilization. The Weight for Exponential Moving Average controls the weight of the current modified moving average versus past averages. For information on how to set Weight for Exponential Moving Average, see Change the default port allocation parameters.

The default weight of the EMA is set at 0.6, but you can configure the weight from greater than 0 to less than or equal to 1. The higher the EMA weighting factor the more weight that the current modified moving average has on the EMA. A value of 1.0 uses only the current averaging period. For example, if the EMA weight is 0.6, the current modified moving average is multiplied by 0.6 and added to the previous EMA multiplied by 0.4 (1 - 0.4).

User-configurable parameters

There are several maximum limits (ceilings) that you can set, in addition to sampling times and the weight to use for the exponential moving average (EMA).

When a port reaches or exceeds one of the ceiling values, it is no longer available for new allocations, even if that causes provisioning to fail. You can change the settings using both the ViPR Controller UI and the REST API. See Change the port allocation parameters using the UI and Change the default port allocation parameters using the REST API.

You can change the following settings:
  • Maximum number of initiators that can use the port before new allocations are not allowed.
  • Maximum number of volumes that can use the port before new allocations are not allowed.
    Note Image
    Volumes may be added to existing exports, such as masking views, storage groups, and storage views, with allocating new ports. These will put additional port load on the ports in that existing export. Therefore, you should set your ceilings lower than the maximum limit you require.

  • Maximum average port percent busy value (from 0 - 100%) before new allocations are not allowed.
  • Maximum average CPU percent busy value (from 0 - 100%) before new allocations are not allowed.
  • The sample averaging time in days (1 -30 days)
  • The weight for the EMA (the EMA factor).
  • Metrics enabled
    • true = use collected metrics and calculate the port percent busy and the CPU percent busy.
    • false = only use the number of initiators and the number of volumes to allocate ports; ignore the collected metrics and do not calculate port percent busy and CPU percent busy.
Note Image
You should take care in setting ceilings. These are absolute limits. Ports which have one or more metrics over a ceiling will not be used for any allocations until such time as all metrics return to a value under the ceilings (or the ceiling limits are increased).

Allocating a port

The EMA Factor and (1- EMA Factor) values that you configured are used when ViPR Controller allocates a port. ViPR Controller takes the (modified moving average x EMAfactor) and the (EMA x 1 - EMAfactor) and does an instantaneous check of these values against the ceilings that you configured. For example, if you have the EMA factor set at 0.6, then ViPR Controller takes the (modified moving average x 0.6) and the (EMA x 0.4) for the instantaneous check against your configured ceiling values.

The port with the lowest metric, which has not reached or exceeded a ceiling is selected. When you require more than one port allocated, ViPR Controller tries to choose two ports that are on different hardware units. For example, you need two ports on a VMAX and there are 3 ports available:
  • 7E0 has a port metric of 10
  • 7F0 has a port metric of 20
  • 8E0 has a port metric of 30
ViPR Controller chooses 7E0 as the first port since it has the lowest port metric, but 8E0 is chosen as the second port. Port 8E0 has a higher port metric than 7F0, but 8E0 is on a different director and, therefore, on different hardware units. This provides redundancy against hardware failures.
Note Image
If you have already allocated ports to a host or cluster, and you are just adding volumes to the same host, then ViPR Controller does not reallocate ports, it just adds the volumes to the export structure.

Back to Top

Global default port selection

ViPR Controller has a default port selection algorithm that can be used globally across all arrays.

The global default port selection algorithm is used:

Calculated values

ViPR Controller automatically calculates two values from its database:
  • Number of initiators ViPR Controller has already assigned to a storage port.
  • Number of volumes ViPR Controller has already assigned to a storage port.
    Note Image
    On VMAX2, the number of volumes is computed across both ports on a director.

These numbers may not reflect all exports done outside of ViPR Controller.

User-configurable parameters

You can set a maximum limit for the number of initiators and volumes that use the port before new allocations are not allowed.

Volumes may be added to existing exports, such as masking views, storage groups, and storage views, with allocating new ports. These will put additional port load on the ports in that existing export. Therefore, you should set your ceilings lower than the maximum limit you require.

When a port exceeds one of the ceiling values, it is no longer available for new allocations, even if that causes provisioning to fail. You can change the settings using both the ViPR Controller UI and the REST API. See Change the port allocation parameters using the UI and Change the default port allocation parameters using the REST API.

Note Image
You should take care in setting ceilings. These are absolute limits. Ports which have one or more of the number of initiators or volumes over their ceiling will not be used for any allocations until such time as both the number of initiators and the number of volumes return to a value under the ceilings (or the ceiling limits are increased).

Allocating a port

The port is determined as follows:

  1. Ports are checked against the ceilings for the number of initiators and volumes.
  2. The ports below their ceilings are checked for redundancy. When you require more than one port allocated, ViPR Controller tries to choose two ports that are on different hardware units.
  3. From the set of ports with the most redundancy, the ports with the fewest number of volumes are selected.
Note Image
If you have already allocated ports to a host or cluster, and you are just adding volumes to the same host, then ViPR Controller does not reallocate ports, it just adds the volumes to the export structure.

Back to Top

Set up metering prerequisites in ViPR Controller

There are two configuration properties that you must ensure are set in ViPR Controller to enable the collection of metrics from VMAX, VNX for Block, and HDS.

You can set these configuration properties using both the ViPR Controller UI and with the REST API.

Back to Top

Use the ViPR Controller UI

After logging into the ViPR Controller UI as a system administrator, check whether metering is enabled, and to what value the metering interval is set.

Selecting Settings > General Configuration > Controller displays:
  • the value of Enable Metering which should be set to true to collect metrics from the arrays.
  • the value of Metering Interval which sets how often ViPR Controller will collect data from the arrays. The metering interval can be set from 1800 seconds (30 minutes) up to 4 hours. The lower the number of seconds, the more accurate will be the results. However, the higher the metering interval is, the less overhead there is on ViPR Controller and the array.

Back to Top

Using the ViPR Controller REST API

There are two configuration properties that you must ensure are set to enable the collection of metrics from VMAX, VNX for Block, and HDS. These properties are controller_enable_metering and controller_metering_interval.

Before you begin

  • Authenticate with the ViPR REST API as a System Administrator. See the ViPR Controller REST API Virtual Data Center Configuration Guide on the ViPR Controller Product Documentation Index.
  • Configuration properties to verify:
    • controller_enable_metering - should be set to true to collect metrics from the arrays.
    • controller_metering_interval - sets how often ViPR Controller will collect data from the arrays. The metering interval can be set from 1800 seconds (30 minutes) up to 4 hours. The lower the number of seconds, the more accurate will be the results. However, the higher the metering interval is, the less overhead there is on ViPR Controller and the array.
  • The EMC ViPR Controller REST API Reference on the ViPR Controller Product Documentation Index provides a description and complete list of parameters for the REST API methods used in this article.

Procedure

  1. Get the list of configuration properties and their values by sending a GET /config/properties request.
    Request
    GET https://<ViPR_Controller_VIP>:4443/config/properties
    Content-Type: application/xml
    X-SDS-AUTH-TOKEN: <AUTH_TOKEN>
    Response
    Note Image
    For readability, only a small number of the configuration properties are shown in the response.

    HTTP 200 OK
    Content-Type: application/xml
    
    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <property_info>
        <properties>
            <entry>
                <key>backup_external_location_password</key>
                <value/>
            </entry>
            <entry>
                <key>backup_external_location_url</key>
                <value/>
            </entry>
            <entry>
                <key>backup_external_location_username</key>
                <value/>
            </entry>
            <entry>
                <key>backup_scheduler_copies_to_keep</key>
                <value>5</value>
            </entry>
            <entry>
                <key>controller_enable_autoscan</key>
                <value>true</value>
            </entry>
            <entry>
                <key>controller_enable_metering</key>
                <value>true</value>
            </entry>
            <entry>
                <key>controller_enable_monitoring</key>
                <value>true</value>
            </entry>
            <entry>
                <key>controller_metering_interval</key>
                <value>3600</value>
            </entry>
            <entry>
                <key>controller_netapp_firmware_version</key>
                <value>8.1.1</value>
            </entry>
        </properties>
    </property_info>
    
  2. Verify that controller_enable_metering is set equal to true, and that controller_metering_interval is set to the required value.
Back to Top

Prerequisites for VNX and HDS metrics-based port selection

There are configuration settings on the VNX and HDS that are required for metrics-based port selection.

For prerequisite configuration settings on VNX, see the ViPR Controller Virtual Data Center Requirements and Information Guide on the ViPR Controller Product Documentation Index.

For prerequisite configuration settings for HDS, see the ViPR Controller Virtual Data Center Requirements and Information Guide on the ViPR Controller Product Documentation Index.

Back to Top

Change the default port allocation parameters

System administrators can change the default values of the port allocation parameters.

You can use ViPR Controller UI or the REST API to change the port allocation parameters.

Back to Top

Change the port allocation parameters using the UI

You change the default values of the port allocation parameters by adding a new parameter setting. When you add a new parameter setting, ViPR Controller will use your setting value instead of the default value.

Before you begin

The following parameters can be changed:
  • Initiator Ceiling = Maximum number of initiators that can use the port before new allocations are not allowed.
  • Volume Ceiling = Maximum number of volumes that can use the port before new allocations are not allowed.
  • Port Utilization Ceiling = Maximum average port percent busy value (from 0 - 100%) before new allocations are not allowed.
  • CPU Utilization Ceiling = Maximum average CPU percent busy value (from 0 - 100%) before new allocations are not allowed.
  • Days To Average Utilization = The sample averaging time in days (1 -30 days) . Default is one day.
  • Weight For Exponential Moving Average = The EMA weight for the current sample. The EMA weight is greater than zero and less than or equal to 1.0. A value of 1.0 uses only the current averaging period.
  • Metrics Enabled
    Note Image
    CPU percent busy is not calculated for HDS

    • true = use collected metrics and calculate Port percent busy and CPU percent busy.
    • false = only use the number of initiators and the number of volumes to allocate ports; ignore the collected metrics and do not calculate Port percent busy and CPU percent busy.

Procedure

  1. Log into the ViPR Controller UI with System Administrator privileges.
  2. Select Physical Assets > Controller Config
  3. Select Port Allocation.
  4. Select the port allocation parameter that you want to change.
  5. Click Add.
  6. Select the Scope Type.
  7. Select the Scope Value.
  8. Type the value of the parameter.
  9. Click Save.
Back to Top

Change the default port allocation parameters using the REST API

You can change the default values of the port allocation parameters by adding a new parameter setting. When you add a new parameter setting, ViPR Controller will use your setting value instead of the default value.

Before you begin

  • Autheticate with the ViPR Controller REST API as a System Administrator. See the ViPR Controller REST API Virtual Data Center Configuration Guide on the ViPR Controller Product Documentation Index.
  • Verify how often ViPR Controller will collect data from the array.
  • Verify that metrics collection from the arrays is enabled.
  • The following configuration types can be set for port allocation:
    • PortAllocationInitiatorCeiling - Maximum number of initiators that can use the port before new allocations will not be allowed.
    • PortAllocationVolumeCeiling - Maximum number of volumes that can use the port before new allocations will not be allowed.
    • PortAllocationPortUtilizationCeiling - Maximum average port percent busy value (from 0 to 100%) before new allocations are not allowed.
    • PortAllocationCpuUtilizationCeiling - Maximum average CPU percent busy value (from 0 to 100%) before new allocations are not allowed.
    • PortAllocationDaysToAverageUtilization - The sample averaging time in days (1 -30 days) .
    • PortAllocationEmaFactor - The EMA weight for the current sample. The EMA weight is greater than zero and less than or equal to 1.0. A value of 1.0 uses only the current averaging period.
    • PortAllocationMetricsEnabled
      Note Image
      CPU percent busy is not calculated for HDS

      • true = use collected metrics and calculate Port percent busy and CPU percent busy.
      • false = only use the number of initiators and the number of volumes to allocate ports; ignore the collected metrics and do not calculate Port percent busy and CPU percent busy.

The EMC ViPR Controller REST API Reference provides a complete list of parameters of the REST API methods used in this discussion. You can access this document on the ViPR Controller Product Documentation Index.

In this example, the maximum number of volumes that can use the port before new allocations will not be allowed is changed to 2000000000 for VMAX.

Procedure

  1. Retrieve the variables that define the configuration type by sending GET /config/controller/types/<configuration_type>.
    Each configuration type includes one or more scopes that can be defined. The response includes the type of scope and the valid values for the scope. In addition, each variable that is returned includes the name of the variable and sample values.
    Request
    GET https://<ViPR_Controller_VIP>:4443/config/controller/types/PortAllocationVolumeCeiling
    Content-Type: application/xml
    X-SDS-AUTH-TOKEN: <AUTH_TOKEN>
    Response
    HTTP 200 OK
    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <config_type>
        <name>PortAllocationVolumeCeiling</name>
        <rules>
            <rule>MinConstraintOne</rule>
        </rules>
        <scopes>
            <scope>
                <type>global</type>
                <value>default</value>
            </scope>
            <scope>
                <type>systemType</type>
                <value>vnxblock</value>
                <value>vmax</value>
                <value>hds</value>
            </scope>
        </scopes>
        <type>Integer</type>
    </config_type>
  2. Add your value for PortAllocationVolumeCeiling using POST /config/controller. To change the value for VMAX, the type of scope should be systemType and the value of the scope should be vmax. The new configuration for PortAllocationVolumeCeiling is returned.
    Request
    POST https://<ViPR_Controller_VIP>:4443/config/controller/
    Content-Type: application/xml
    X-SDS-AUTH-TOKEN: <AUTH_TOKEN>
    
    <config_create>
         <config_type>PortAllocationVolumeCeiling</config_type>
         <value>2000000000</value>
         <scope>
              <type>systemType</type>
              <value>vmax</value>
         </scope>
    </config_create>
    
    Response
    HTTP 200 OK
    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <config>
        <id>urn:storageos:CustomConfig:45649fa3-274c-4dbd-b50c-9e3509ab1b6f:vdc1</id>
        <link rel="self" href="/config/controller/urn:storageos:CustomConfig:45649fa3-274c-4dbd-b50c-9e3509ab1b6f:vdc1"/>
        <name>systemType.vmax.PortAllocationVolumeCeiling</name>
        <tags/>
        <config_type>
            <name>PortAllocationVolumeCeiling</name>
            <link rel="self" href="/config/controller/types/PortAllocationVolumeCeiling"/>
        </config_type>
        <registered>true</registered>
        <scope>
            <type>systemType</type>
            <value>vmax</value>
        </scope>
        <system_default>false</system_default>
        <value>2000000000</value>
    </config>
    
    The response also includes the URN for your new configuration. If you want to modify the value of your configuration, you can use the URN in the request PUT /config/controller/{ID} to change the value. For example:
    PUT https://<ViPR_Controller_VIP>:4443/config/controller/urn:storageos:CustomConfig:45649fa3-274c-4dbd-b50c-9e3509ab1b6f:vdc1
    Content-Type: application/xml
    X-SDS-AUTH-TOKEN: <AUTH_TOKEN>
    
    <config_update>
        <value>1900000000</value>
    </config_update>
    
    HTTP 200 OK
    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <config>
        <id>urn:storageos:CustomConfig:45649fa3-274c-4dbd-b50c-9e3509ab1b6f:vdc1</id>
        <link rel="self" href="/config/controller/urn:storageos:CustomConfig:45649fa3-274c-4dbd-b50c-9e3509ab1b6f:vdc1"/>
        <name>systemType.vmax.PortAllocationVolumeCeiling</name>
        <tags/>
        <config_type>
            <name>PortAllocationVolumeCeiling</name>
            <link rel="self" href="/config/controller/types/PortAllocationVolumeCeiling"/>
        </config_type>
        <registered>true</registered>
        <scope>
            <type>systemType</type>
            <value>vmax</value>
        </scope>
        <system_default>false</system_default>
        <value>1900000000</value>
    </config>
    
Back to Top

Additional REST API calls to manage your custom port allocation settings

The table shows some additional APIs that are used to manage your custom port allocation settings.

Back to Top

HDS performance metrics

The table describes the metrics that are collected from HDS which ViPR Controller uses to allocate ports.

These metrics are used to calculate:
  • Percent busy for the port (FEPort) which is computed from kbytesTransferred over the time period since the last valid sample.

Back to Top

VMAX performance metrics

The VMAX metrics collection is contingent on having metering turned on and configured.

The table describes the metrics that are collected from VMAX which ViPR Controller uses to allocate ports.

These metrics are used to calculate two values:
  • Percent busy for the port (FEPort) which is computed from kbytesTransferred over the time period since the last valid sample.
  • Percent busy for the CPU (FEAdapt) which is computed from the non IdleTime over the time period since the last valid sample.

Back to Top

VNX for Block performance metrics

The table describes the metrics that are collected onVNX for Block which ViPR Controller uses to allocate ports.

Note Image
VNX for Block metrics collection is contingent on having metering turned on and configured. See Prerequisite configuration settings for VNX for Block for more information..

These metrics are used to calculate two values:
  • Percent busy for the port (FEPort) which is computed from kbytesTransferred over the time period since the last valid sample.
  • Percent busy for the CPU (FEAdapt) which is computed from the non idle time over the time period since the last valid sample.

Back to Top