Add Third-Party Block storage to ViPR using the REST API

Table of Contents

Back to Top

Overview

This article provides the necessary information to add third-party block storage to EMC ViPR. ViPR uses the OpenStack Block Storage (Cinder) Service to add third-party block storage systems to ViPR. Once OpenStack block storage is added to ViPR, all the storage systems, being managed by OpenStack are discovered, and registered in ViPR.

This article also describes the important REST API calls to manage and configure the storage after it has been added, and discovered in ViPR.

This article applies to EMC ViPR 2.0.

This article is part of a series

Storage systems can be added to ViPR at anytime, however, if you are setting up the ViPR virtual data center for the first time, follow these steps.
  1. Authenticate with the ViPR REST API
  2. Add physical assets to ViPR:
  3. Create ViPR virtual assets:
    1. Create and configure a virtual array
    2. Create virtual pools:

Back to Top

Third-party block storage provider installation requirements

ViPR uses the OpenStack Block Storage (Cinder) Service to add third-party block storage systems to ViPR.

OpenStack version support

For supported versions, see the EMC ViPR Support Matrix available on the EMC Community Network (community.emc.com).

Openstack installation is supported on the following platforms:

  • Red Hat Enterprise Linux
  • SUSE Enterprise Linux
  • Ubuntu Linux

For a list of the supported platform versions, see Openstack documentation at: http://docs.openstack.org.

The following two components must be installed. Both components can be installed on the same server, or on separate servers.

For complete installation and configuration details, refer to the OpenStack documentation at: http://docs.openstack.org.

Back to Top

Third-party block storage system support

Third-party storage systems must be configured on the OpenStack Block Storage Controller node (Cinder service).

Supported third-party block storage systems

ViPR operations are supported on any third-party block storage systems, tested by OpenStack, which use Fibre Channel or iSCSI protocols.

Non-supported storage systems

ViPR does not support third-party storage systems using:
  • Proprietary protocols such as Ceph.
  • Drivers for block over NFS.
  • Local drivers such as LVM.
Some of the OpenStack supported storage systems and drivers that are not supported by ViPR are:
  • LVM
  • NetAppNFS
  • NexentaNFS
  • RBD (Ceph)
  • RemoteFS
  • Scality
  • Sheepdog
  • XenAPINFS

Refer to www.openstack.org for information about OpenStack third-party storage systems.

ViPR native driver support and recommendations

ViPR provides limited support of third-party block storage. Therefore, to have full support of all ViPR operations, it is recommended to add or manage the following storage systems with ViPR native drivers, and not to use the OpenStack third-party block storage provider to add these storage systems to ViPR:
  • EMC VMAX
  • EMC VNX for Block
  • EMC VPLEX
  • Hitachi Data Systems (with Fibre Channel only)
Add these storage systems directly in ViPR, using the storage system host address, or the host address of the proprietary storage provider.

Back to Top

Supported ViPR operations

ViPR discovery, and following service operations can be performed on third-party block storage systems:

  • Create Block Volume
  • Export Volume to Host
  • Create a Block Volume for a Host
  • Expand block volume
  • Remove Volume by Host
  • Remove Block Volume
  • Create Full Copy
  • Create Block Snapshot
  • Create volume from snapshot
  • Remove Block Snapshot
Note Image
Using the ViPR, Create VMware Datastore service, to create a datastore from a block volume created by a third-party storage system is not supported. However datastores can be created from third-party block volumes, manually through VMware vCenter.

Back to Top

OpenStack configuration

After the successful installation of Keystone and Cinder services, the Cinder configuration file needs to be modified for the storage systems it will manage. After modifying the configuration file, the volume types need to be created to map to the backend drivers. These volume types will be discovered as storage pools of a specific storage system in ViPR.

OpenStack third-party storage configuration recommendations

It is recommended that before adding the storage provider to ViPR, configure the storage provider with only the storage systems that you want to manage with ViPR through the third-party block storage provider.

When the third-party block storage provider is added to the ViPR Physical Assets, all of the storage systems managed by the OpenStack block storage service, which are supported by ViPR, will be added to ViPR.

Note Image
If you do not configure the storage provider with only the storage systems you want managed through ViPR, it is possible to deregister or delete the storage systems through ViPR after they are added to ViPR. However, it is a better practice to configure the storage provider for ViPR integration prior to adding the storage provider to ViPR.

Back to Top

Cinder service configuration requirements

The cinder configuration file is located in: /etc/cinder/cinder.conf. An entry must be added to the configuration file for each storage system that will be managed.

Cinder does not have any specific standards on backend driver attributes definitions. Refer to the vendor-specific recommendations on how to configure the cinder driver, which may involve installing a vendor specific plugin or CLI.

Back to Top

Storage system (backend) configuration settings

To manage storage systems, Cinder defines backend configurations in individual sections, of the cinder.conf file, which are specific to the storage system type as follows.

Procedure

  1. Uncomment, enabled_backends, which will be commented by default, and add the multiple back-end names. In the following example, NetApp and IBM SVC are added as backend configurations.

    enabled_backends=netapp-iscsi,ibm-svc-fc

  2. Near the end of the file add the storage system specific entries as follows:

    [netapp-iscsi] #NetApp array configuration goes here [ibm-svc-fc] #IBM SVC array configuration goes here

  3. Restart the Cinder service.

    #service openstack-cinder-volume restart

Back to Top

Create volume types in OpenStack

Volume types must be created and mapped for each configured backend driver.

Volume types can be created either through Cinder CLI commands or through the Dashboard ( OpenStack UI ). The following demonstrates the Cinder CLI commands to create volume types (NetApp, IBM SVC), and map them to the backend driver.

cinder --os-username admin --os-tenant-name admin type-create "NetAPP-iSCSI" cinder --os-username admin --os-tenant-name admin type-key "NetAPP-iSCSI" set volume_backend_name=NetAppISCSI cinder --os-username admin --os-tenant-name admin type-create "IBM-SVC-FC" cinder --os-username admin --os-tenant-name admin type-key "IBM-SVC-FC" set volume_backend_name=IBMSVC-FC cinder --os-username admin --os-tenant-name admin extra-specs-list

Validate setup

Validate that OpenStack has been configured correctly to create volumes for each of the added storage systems.
  1. Create volumes for each type of volume created in OpenStack.
    Volumes can be created in the OpenStack UI or the Cinder CLI. The Cinder CLI command to create a volume is:

    cinder --os-username admin --os-tenant-name admin --display-name <volume-name> --volume-type <volume-type-id> <size>

  2. Check that the volumes are getting created on the associated storage system. For example, NetApp-iSCSI type volumes should be created only on the NetApp storage system.

Back to Top

Add Third-party block storage to ViPR

Before you begin

You need the following information:
  • The IP address of the Third-party block storage provider.
  • The username and password for connecting to the storage provider. System administrator privileges are required.
  • The port number used to connect to the storage provider. The default port number for a Third-party block storage provider is 22.
  • The <interface_type> is cinder.
  • Authenticate with the ViPR REST API as a System Administrator.

The EMC ViPR REST API Reference provides a description and complete list of parameters for the REST API methods used in this article.

Procedure

  1. Add third-party block storage by sending a POST /vdc/storage-providers request.
    The request returns a task whose URI can be queried to determine the when the task is complete.
    Request

    POST https://<ViPR_VIP>:4443/vdc/storage-providers Content-Type: application/xml X-SDS-AUTH-TOKEN: <AUTH_TOKEN> <storage_provider_create> <name>TP_east_1</name> <interface_type>cinder</interface_type> <ip_address>192.168.0.0</ip_address> <port_number>22</port_number> <user_name>admin</user_name> <password>Password1</password> <use_ssl>false</use_ssl> </storage_provider_create>

    Response

    HTTP 202 Accepted Content-Type: application/xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <task> <associated_resources/> <description>SCAN_STORAGEPROVIDER</description> <op_id>d96fcba5-9305-4acc-be60-a9aee495d2b5</op_id> <resource> <id>urn:storageos:StorageProvider:c00b486c-141d-4c90-a5c2-0203794b0cf8:vdc1</id> <link rel="self" href="/vdc/storage-providers/urn:storageos:StorageProvider:c00b486c-141d-4c90-a5c2-0203794b0cf8:vdc1"/> <name>TP_east_1</name> </resource> <link rel="self" href="/vdc/storage-providers/urn:storageos:StorageProvider:c00b486c-141d-4c90-a5c2-0203794b0cf8:vdc1/tasks/d96fcba5-9305-4acc-be60-a9aee495d2b5"/> <start_time>1400193977911</start_time> <state>pending</state> </task>

  2. Repeat the query of the Third-party storage provider create task, using the task URL from the response body of the POST request, until the message attribute of the task is Operation completed successfully.
    Request

    GET https://<ViPR_VIP>:4443/vdc/storage-providers/urn:storageos:StorageProvider:c00b486c-141d-4c90-a5c2-0203794b0cf8:vdc1/tasks/d96fcba5-9305-4acc-be60-a9aee495d2b5 Content-Type: application/xml X-SDS-AUTH-TOKEN: <AUTH_TOKEN>

    Response

    HTTP 200 OK Content-Type: application/xml <task> <end_time>1400194038807</end_time> <description>SCAN_STORAGEPROVIDER</description> <message>Operation completed successfully</message> <op_id>d96fcba5-9305-4acc-be60-a9aee495d2b5</op_id> <resource> <id>urn:storageos:StorageProvider:c00b486c-141d-4c90-a5c2-0203794b0cf8:vdc1</id> <link rel="self" href="/vdc/storage-providers/urn:storageos:StorageProvider:c00b486c-141d-4c90-a5c2-0203794b0cf8:vdc1"/> <name>TP_east_1</name> </resource> <link rel="self" href="/vdc/storage-providers/urn:storageos:StorageProvider:c00b486c-141d-4c90-a5c2-0203794b0cf8:vdc1/tasks/d96fcba5-9305-4acc-be60-a9aee495d2b5"/> <start_time>1400193977911</start_time> <state>ready</state> </task>

Back to Top

Third-party block storage port discovery

The OpenStack API does not provide the storage port World Wide Port Name (WWPN) for Fibre Channel connected storage systems, or the IQN for iSCSI connected storage systems. Therefore, ViPR cannot retrieve the storage port WWPNs, or IQNs during discovery.

After ViPR discovers a third-party block storage array, a default storage port is created for the storage system, and appears in the Storage Port page, with the name Default, and storage port identifier: > Openstack+<storagesystemserialnumber>+Port+Default.

Fibre Channel configured storage ports

ViPR export operations cannot be performed on an FC connected storage system, which has been added to ViPR without any WWPNs assigned to the storage port. Therefore, ViPR system administrators must manually add at least one WWPN to the default storage port before performing any export operations on the storage system. WWPNs can only be added to ViPR through the VIPR CLI. For steps to configure the storage port through the ViPR CLI refer to Add WWPNs to FC connected third-party block storage ports.

After the WWPN is added to the storage port, you can perform export operations on the storage system from ViPR. At the time of the export, ViPR reads the export response from the Cinder service. The export response will include the WWPN, which was manually added by the system administrator from the ViPR CLI, and any additional WWPNs listed in the export response. ViPR then creates a storage port for each of the WWPNs listed in the export response during the export operation.

After a successful export operation is performed, the Storage Port page displays any newly created ports, in addition to the Default storage port.

Each time another export operation is performed on the same storage system, ViPR reads the Cinder export response. If the export response presents WWPNs, which are not present in ViPR, then ViPR creates new storage ports for every new WWPN.

iSCSI configured storage ports

The default storage port is used to support the storage system configuration until an export is performed on the storage system. At the time of the export, ViPR reads the export response from the Cinder service, which includes the iSCSI IQN. ViPR then modifies the default storage port's identifier with the IQN received from the Cinder export response.

Each time another export operation is performed on the same storage system, ViPR reads the Cinder export response. If the export response presents an IQN, which is not present in ViPR, then ViPR creates a new storage port.

Back to Top

Add WWPNs to FC connected third-party block storage ports

If the third-party block storage is connected through Fibre Channel (FC), at least one storage port, World Wide Port Name (WWPN) must be manually added to ViPR through the ViPR CLI.

You will need to get at least one valid WWPN for the storage port before continuing.

Use the following CLI commands to add a WWPN to the storage port.

  1. Get the last three digits of the storage system serial number from the list of storage systems.

    C:\Users\<username>viprcli storagesystem list NAME PROVIDER_NAME SYSTEM_TYPE SERIAL_NUMBER IBMSVC-FC_StorwizeSVCDriver+11111111234 myProviderName openstack 11111111234

  2. Get the port network ID for the Default storage port. The storage port network ID (PORT_NETWORK_ID) will be an invalid value.

    C:\Users\<username>viprcli storageport list -t openstack -sn 234 PORT_NAME TRANSPORT_TYPE NETWORK_NAME PORT_NETWORK_ID REGISTRATION_STATUS DEFAULT FC FABRIC_name-fabric <some invalid value> REGISTERED

  3. Add the WWPN (50:01:02:34:05:06:FE:07 in this example) to the storage port.

    C:\Users\<username>viprcli storageport update -t openstack -sn 234 -pn DEFAULT -tt FC -pnwid "50:01:02:34:05:06:FE:07"

  4. Repeat step 2, to validate the value was added to the storage port (PORT_NETWORK_ID).

    C:\Users\<username>viprcli storageport list -t openstack -sn 234 PORT_NAME TRANSPORT_TYPE NETWORK_NAME PORT_NETWORK_ID REGISTRATION_STATUS DEFAULT FC FABRIC_name-fabric 50:01:02:34:05:06:FE:07 REGISTERED

Refer to the EMC ViPR CLI Reference guide for more information.

Back to Top

Network configuration requirements for storage

After the storage system is added to ViPR, it is recommended to add the corresponding SAN switch using POST /vdc/network-systems.

Back to Top

Important REST API calls to manage and configure storage systems

The table shows some important APIs that are used to manage and configure storage systems.