[an error occurred while processing this directive]

ECS 2.0 – ECS OpenStack Swift Object Service API support

Table of Contents

OpenStack Swift API

ECS includes support for the OpenStack Swift API. This article describes the supported operations and describes the mechanisms for authorization and authentication.

The OpenStack Swift Service is made available on the following ports.

The following sections describe supported methods, the ECS extensions, and the mechanism for authentication:

Examples showing the use of the OpenStack Swift API can be found here:
Back to Top

OpenStack Swift supported operations

The following sections list the OpenStack REST API requests that are supported by ECS.

This information is taken from the Object Storage API V1 section of the OpenStack API Reference documentation.

Supported OpenStack Swift calls

The following OpenStack Swift REST API calls are supported in ECS.

Unsupported OpenStack Swift calls

The following OpenStack Swift REST API calls are not supported in ECS.

Back to Top

API Extensions

A number of extensions to the object APIs are supported.

The extensions and the APIs that support them are listed in the following table.

Back to Top

Updating a byte range within an object

An example of using the ECS API extensions to update a byte range of an object is provided below.

First do a GET request on the object named object1 located in bucket1 to review the object. object1 has the value The quick brown fox jumps over the lazy dog.

GET /bucket1/object1 HTTP/1.1
Date: Mon, 17 Jun 2013 20:04:40 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:9qxKiHt2H7upUDPF86dvGp8VdvI=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:04:40 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:04:28 GMT
ETag: 6
Content-Type: application/json
Content-Length: 43
 
The quick brown fox jumps over the lazy dog.

Now you want to update a specific byte range within this object. To do this, the Range header in the object data request must include the start and end offsets of the object that you want to update. 
The format is: Range: bytes=<startOffset>-<endOffset>

In the example below, the PUT request includes the Range header with the value bytes=10-14 indicating that bytes 10,11,12,13,14 are to be replaced by the value sent in the request. Here, the new value green is being sent.

PUT /bucket1/object1 HTTP/1.1
Content-Length: 5
Range: bytes=10-14
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:15:16 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:xHJcAYAEQansKLaF+/4PdLBHyaM=
Accept-Encoding: gzip, deflate, compress
 
green
 
HTTP/1.1 204 No Content
ETag: 10
x-amz-id-2: object1
x-amz-request-id: 027f037c-29ea-4670-8670-de82d0e9f52a
Content-Length: 0
Date: Mon, 17 Jun 2013 20:15:16 GMT

When reading the object again, the new value is now The quick green fox jumps over the lazy dog. (The word brown has been replaced with green.) You have updated a specific byte range within this object.

GET /bucket1/object1 HTTP/1.1
Cookie: JSESSIONID=wdit99359t8rnvipinz4tbtu
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:16:00 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:OGVN4z8NV5vnSAilQTdpv/fcQzU=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:16:00 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:15:16 GMT
ETag: 10
Content-Type: application/json
Content-Length: 43
 
The quick green fox jumps over the lazy dog.
Back to Top

Overwriting part of an object

An example of using the ECS API extensions to overwrite part of an object is provided below.

You can overwrite part of an object by providing only the starting offset in the data request. The data in the request will be written starting at the provided offset. The format is: Range: <startingOffset>-

For example, to write the data brown cat starting at offset 10, you would issue this PUT request:

PUT /bucket1/object1 HTTP/1.1
Content-Length: 9
Range: bytes=10-
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:51:41 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:uwPjDAgmazCP5lu77Zvbo+CiT4Q=
Accept-Encoding: gzip, deflate, compress
 
brown cat
 
HTTP/1.1 204 No Content
ETag: 25
x-amz-id-2: object1
x-amz-request-id: 65be45c2-0ee8-448a-a5a0-fff82573aa3b
Content-Length: 0
Date: Mon, 17 Jun 2013 20:51:41 GMT

When retrieving the object, you can see the final value The quick brown cat jumps over the lazy dog and cat. (green fox has been replaced with brown cat). You have overwritten part of the data in this object at the provided starting offset.

GET /bucket1/object1 HTTP/1.1
Date: Mon, 17 Jun 2013 20:51:55 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:/UQpdxNqZtyDkzGbK169GzhZmt4=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:51:55 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:51:41 GMT
ETag: 25
Content-Type: application/json
Content-Length: 51
 
The quick brown cat jumps over the lazy dog and cat.
Back to Top

Appending data to an object

An example of using the ECS API extensions to append data to an object is provided below.

There may be cases where you need to append to an object, but determining the exact byte offset is not efficient or useful. For this scenario, ECS provides the ability to atomically append data to the object without specifying an offset (the correct offset is returned to you in the response).

A Range header with the special value bytes=-1- can be used to append data to an object. In this way, the object can be extended without knowing the existing object size.

The format is: Range: bytes=-1-

A sample request showing appending to an existing object using a Range value of bytes=-1-. Here the value and cat is sent in the request.

PUT /bucket1/object1 HTTP/1.1
Content-Length: 8
Range: bytes=-1-
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:46:01 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:/sqOFL65riEBSWLg6t8hL0DFW4c=
Accept-Encoding: gzip, deflate, compress
 
and cat
 
HTTP/1.1 204 No Content
ETag: 24
x-amz-id-2: object1
x-amz-request-id: 087ac237-6ff5-43e3-b587-0c8fe5c08732
Content-Length: 0
Date: Mon, 17 Jun 2013 20:46:01 GMT

When retrieving the object again, you can see the full value The quick green fox jumps over the lazy dog and cat. You have appended data to this object.

GET /bucket1/object1 HTTP/1.1
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:46:56 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:D8FSE8JoLl0MTQcFmd4nG1gMDTg=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:46:56 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:46:01 GMT
ETag: 24
Content-Type: application/json
Content-Length: 51
 
The quick green fox jumps over the lazy dog and cat.
Back to Top

Reading multiple byte ranges within an object

An example of using the ECS API extensions to read multiple byte ranges within an object is provided below.

To read two specific byte ranges within the object named object1, you would issue the following GET request for Range: bytes==4-8,41-44. The read response would be for the words quick and lazy.
Note Image
The Amazon S3 API only supports one range when using the HTTP header Range for reading; ECS supports multiple byte ranges.

GET /bucket1/object1 HTTP/1.1
Date: Mon, 17 Jun 2013 20:51:55 -0000
x-emc-namespace: emc
Range: bytes==4-8,41-44
Content-Type: application/octet-stream
Authorization: AWS wuser1:/UQpdxNqZtyDkzGbK169GzhZmt4=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 206 Partial Content
Date: Mon, 17 Jun 2013 20:51:55 GMT
Content-Type: multipart/byteranges;boundary=bound04acf7f0ae3ccc
Last-Modified: Mon, 17 Jun 2013 20:51:41 GMT
Content-Length: 230
 
--bound04acf7f0ae3ccc
Content-Type: application/octet-stream
Content-Range: bytes 4-8/50
quick
--bound04acf7f0ae3ccc
Content-Type: application/octet-stream
Content-Range: bytes 41-44/50
lazy
--bound04acf7f0ae3ccc--
Back to Top

OpenStack Version 1 authentication

To communicate with ECS

Procedure

  1. Acquire a UID and password from ECS. The UID will be an LDAP or Active Directory name. Call the following ECS REST API to generate a password.

    ECS through the OpenStack Swift API, follow these steps:

    Request:
    PUT /object/user-password/myUser@emc.com
        <user_password_create>
        <password>myPassword</password>
        <namespace>EMC_NAMESPACE</namespace>
        </user_password_create>
    Response:
    HTTP 200
  2. Call the OpenStack authentication REST API shown below. Use port 9024 for HTTP, or port 9025 for HTTPS.
    Request:
    GET /auth/v1.0
      X-Auth-User: myUser@emc.com
      X-Auth-Key: myPassword
    Response:
    HTTP/1.1 
       204 No
       Content
       Date: Mon, 12 Nov 2010 15:32:21 GMT
       Server: Apache
    
       X-Storage-Url: https://{hostname}/v1/account
       X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
       Content-Length: 0

Results

If the UID and password are validated by ECS, the storage URL and token are returned in the response header. Further requests are authenticated by including this token. The storage URL provides the host name and resource address. You can access containers and objects by providing the following X-Storage-Url header:
X-Storage-Url: https://{hostname}/v1/{account}/{container}/{object}

The generated token expires 24 hours after creation. If you repeat the authentication request within the 24 hour period using the same UID and password, OpenStack will return the same token. Once the 24 hour expiration period expires, OpenStack will return a new token.

Example

In the following simple authentication example, the first REST call returns an X-Auth-Token. The second REST call uses that X-Auth-Token to perform a GET request on an account.
$ curl -i -H "X-Storage-User: tim_250@sanity.local" -H "X-Storage-Pass: 1fO9X3xyrVhfcokqy3U1UyTY029gha5T+k+vjLqS" 
                                                                           http://ecs.yourco.com:9024/auth/v1.0
 HTTP/1.1 204 No Content
    X-Storage-Url: http://ecs.yourco.com:9024/v1/s3
    X-Auth-Token: 8cf4a4e943f94711aad1c91a08e98435
    Server: Jetty(7.6.4.v20120524)
$ curl -v -X GET -s -H "X-Auth-Token: 8cf4a4e943f94711aad1c91a08e98435" 
                                                      http://ecs.yourco.com:9024/v1/s3
* About to connect() to ecs.yourco.com port 9024 (#0)
    * Trying 203.0.113.10...
    * Adding handle: conn: 0x7f9218808c00
    * Adding handle: send: 0
    * Adding handle: recv: 0
    * Curl_addHandleToPipeline: length: 1
    * - Conn 0 (0x7f9218808c00) send_pipe: 1, recv_pipe: 0
    * Connected to ecs.yourco.com (203.0.113.10) port 9024 (#0)

    > GET /v1/s3 HTTP/1.1
    > User-Agent: curl/7.31.0
    > Host: ecs.yourco.com:9024
    > Accept: */*
    > X-Auth-Token: 8cf4a4e943f94711aad1c91a08e98435
    >
    < HTTP/1.1 204 No Content
    < Date: Mon, 16 Sep 2013 19:31:45 GMT
    < Content-Type: text/plain
    * Server Jetty(7.6.4.v20120524) is not blacklisted
    < Server: Jetty(7.6.4.v20120524)
    <

    * Connection #0 to host ecs.yourco.com left intact

Back to Top

OpenStack Version 2 authentication

ECS includes limited support for OpenStack Version 2 (Keystone) authentication.

Before you begin

OpenStack V2 introduces unscoped tokens. These can be used to query tenant information. An unscoped token along with tenant information can be used to query a scoped token. A scoped token and a service endpoint can be used to authenticate with ECS as described in the previous section describing V1 authentication.

The two articles listed below provide important background information.

Procedure

  1. Retrieve an unscoped token.
    curl -v -X POST -H 'ACCEPT: application/json' -H "Content-Type: application/json" -d '{"auth": 
    {"passwordCredentials" : {"username" : "swift_user", "password" : "123"}}}' http://203.0.113.10:9024/v2.0/tokens 
    		  
    The response looks like the following. The unscoped token is preceded by id.
    {"access": {"token": {"id":"d668b72a011c4edf960324ab2e87438b","expires":"1376633127950"l},"user": 
                     {"name": "sysadmin", "roles":[ ], "role_links":[ ] },"serviceCatalog":[ ] }} , } 
    		  
  2. Retrieve tenant info associated with the unscoped token.
    curl -v http://203.0.113.10:9024/v2.0/tenants -H 'X-Auth-Token: d668b72a011c4edf960324ab2e87438b' 
    		  
    The response looks like the following.
    {"tenants_links":[], "tenants":[{"description":"s3","enabled":true, "name": "s3"}]}
     
    		  
  3. Retrieve scoped token along with storageUrl.
    curl -v -X POST -H 'ACCEPT: application/json' -H "Content-Type: application/json" -d '{"auth": {"tenantName" : "s3", 
                              "token":{"id" : d668b72a011c4edf960324ab2e87438b"}}}' http://203.0.113.10:9024/v2.0/tokens 
    		  
    An example response follows. The scoped token is preceded by id.
    {"access":{"token":{"id":"baf0709e30ed4b138c5db6767ba76a4e
    ","expires":"1376633255485","tenant":{"description":"s3","enabled":true,"name":"s3"}},
    "user":{"name":"swift_admin","roles":[{"name":"member"},{"name":"admin"}],"role_links":[]},
          "serviceCatalog":[{"type":"object-store", "name":"Swift","endpoints_links":[],"endpoint":[{"internalURL":
           "http://203.0.113.10:9024/v1/s3","publicURL":"http://203.0.113.10:9024/v1/s3"}]}]}}
  4. Use the scoped token and service endpoint URL for swift authentication. This step is the same as in V1 of OpenStack.
    curl -v -H "X-Auth-Token: baf0709e30ed4b138c5db6767ba76a4e" http://203.0.113.10:9024/v1/s3/{container}/{object} 
    		  
Back to Top

Authorization on Container

OpenStack Swift authorization targets only containers.

Swift currently supports two types of authorization:

  • Referral style authorization
  • Group style authorization.

ECS 2.0 supports only group-based authorization.

Admin users can perform all operations within the account. Non-admin users can only perform operations per container based on the container's X-Container-Read and X-Container-Write Access Control Lists. The following operations can be granted to non-admin users:

Admin assigns read access to the container

curl -XPUT -v -H 'X-Container-Read: {GROUP LIST}' 
                 -H 'X-Auth-Token: {TOKEN}' 
                 http://127.0.0.1:8080/v1/AUTH_bourne/{container1}" 
		

This command allows users belonging to the GROUP LIST to have read access rights to container1.

After read permission is granted, users belongs to target group(s) can perform below operations:

  • HEAD container - Retrieve container metadata. Only allowed if user is assigned to group that has Tenant Administrator privileges.
  • GET container - List objects within a container
  • GET objects with container - Read contents of the object within the container

Admin assigns write access to the container

curl -XPUT -v -H 'X-Container-Write: {GROUP LIST}' 
                 -H 'X-Auth-Token: {TOKEN}' 
                 http://127.0.0.1:8080/v1/AUTH_bourne/{container1}" 
		

The users in the group GROUP LIST are granted write permission. Once write permission is granted, users belongs to target group(s) can perform following operations:

  • POST container - Set metadata. Start with prefix "X-Container-Meta".
  • PUT objects within container - Write/override objects with container.

Additional information on authorization can be found in: Container Operations

Back to Top
[an error occurred while processing this directive]