ViPR 2.1 - Use the ViPR Object Services

Table of Contents

API overview

The ViPR object services are accessible through industry standard APIs.

The object services support the Amazon S3, EMC Atmos, and OpenStack Swift APIs. Support for each of these APIs is described in the following articles:

In addition, the object APIs are extended to provide additional capabilities, the most important of which is the file access mode, which enables objects to be accessed as files. The object API extensions are listed in:
The object services are accessed on the ViPR data nodes or commodity nodes on the following ports.

Where you have multiple data nodes, you will need to configure a load balancer and the outward facing ports will be as configured on the load balancer.

Before you can use the ViPR object services, ViPR must be configured from the ViPR UI or using the ViPR controller API or CLI. Refer to the ViPR Installation and Configuration Roadmap for more details.

Back to Top

Object SDKs

A number of SDKs are available to support development of solutions using the Data Services object APIs. The SDKs provide support for the ViPR extensions and support development in a range of languages.

The SDKs are made available through the EMC Developer Network:

and GitHub:

Back to Top

API Extensions

A number of extensions to the object APIs are supported.

The extensions and the APIs that support them are listed in the following table.

Back to Top

Appending data to an object

An example of using the ViPR API extensions to append data to an object is provided below.

There may be cases where you need to append to an object, but determining the exact byte offset is not efficient or useful. For this scenario, ViPR provides the ability to atomically append data to the object without specifying an offset (the correct offset is returned to you in the response).

A Range header with the special value bytes=-1- can be used to append data to an object. In this way, the object can be extended without knowing the existing object size.

The format is: Range: bytes=-1-

A sample request showing appending to an existing object using a Range value of bytes=-1-. Here the value and cat is sent in the request. 

PUT /bucket1/object1 HTTP/1.1
Content-Length: 8
Range: bytes=-1-
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:46:01 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:/sqOFL65riEBSWLg6t8hL0DFW4c=
Accept-Encoding: gzip, deflate, compress
 
and cat
 
HTTP/1.1 204 No Content
ETag: 24
x-amz-id-2: object1
x-amz-request-id: 087ac237-6ff5-43e3-b587-0c8fe5c08732
Content-Length: 0
Date: Mon, 17 Jun 2013 20:46:01 GMT

When retrieving the object again, you can see the full value The quick green fox jumps over the lazy dog and cat. You have appended data to this object. 

GET /bucket1/object1 HTTP/1.1
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:46:56 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:D8FSE8JoLl0MTQcFmd4nG1gMDTg=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:46:56 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:46:01 GMT
ETag: 24
Content-Type: application/json
Content-Length: 51
 
The quick green fox jumps over the lazy dog and cat.
Back to Top

Reading multiple byte ranges within an object

An example of using the ViPR API extensions to read multiple byte ranges within an object is provided below.

To read two specific byte ranges within the object named object1, you would issue the following GET request for Range: bytes==4-8,41-44. The read response would be for the words quick and lazy.
Note Image
The Amazon S3 API only supports one range when using the HTTP header Range for reading; ViPR supports multiple byte ranges. 
GET /bucket1/object1 HTTP/1.1
Date: Mon, 17 Jun 2013 20:51:55 -0000
x-emc-namespace: emc
Range: bytes==4-8,41-44
Content-Type: application/octet-stream
Authorization: AWS wuser1:/UQpdxNqZtyDkzGbK169GzhZmt4=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 206 Partial Content
Date: Mon, 17 Jun 2013 20:51:55 GMT
Content-Type: multipart/byteranges;boundary=bound04acf7f0ae3ccc
Last-Modified: Mon, 17 Jun 2013 20:51:41 GMT
Content-Length: 230
 
--bound04acf7f0ae3ccc
Content-Type: application/octet-stream
Content-Range: bytes 4-8/50
quick
--bound04acf7f0ae3ccc
Content-Type: application/octet-stream
Content-Range: bytes 41-44/50
lazy
--bound04acf7f0ae3ccc--
Back to Top

Updating a byte range within an object

An example of using the ViPR API extensions to update a byte range of an object is provided below.

First do a GET request on the object named object1 located in bucket1 to review the object. object1 has the value The quick brown fox jumps over the lazy dog.

GET /bucket1/object1 HTTP/1.1
Date: Mon, 17 Jun 2013 20:04:40 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:9qxKiHt2H7upUDPF86dvGp8VdvI=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:04:40 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:04:28 GMT
ETag: 6
Content-Type: application/json
Content-Length: 43
 
The quick brown fox jumps over the lazy dog.

Now you want to update a specific byte range within this object. To do this, the Range header in the object data request must include the start and end offsets of the object that you want to update. 
The format is: Range: bytes=<startOffset>-<endOffset>

In the example below, the PUT request includes the Range header with the value bytes=10-14 indicating that bytes 10,11,12,13,14 are to be replaced by the value sent in the request. Here, the new value green is being sent. 

PUT /bucket1/object1 HTTP/1.1
Content-Length: 5
Range: bytes=10-14
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:15:16 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:xHJcAYAEQansKLaF+/4PdLBHyaM=
Accept-Encoding: gzip, deflate, compress
 
green
 
HTTP/1.1 204 No Content
ETag: 10
x-amz-id-2: object1
x-amz-request-id: 027f037c-29ea-4670-8670-de82d0e9f52a
Content-Length: 0
Date: Mon, 17 Jun 2013 20:15:16 GMT

When reading the object again, the new value is now The quick green fox jumps over the lazy dog. (The word brown has been replaced with green.) You have updated a specific byte range within this object. 

GET /bucket1/object1 HTTP/1.1
Cookie: JSESSIONID=wdit99359t8rnvipinz4tbtu
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:16:00 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:OGVN4z8NV5vnSAilQTdpv/fcQzU=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:16:00 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:15:16 GMT
ETag: 10
Content-Type: application/json
Content-Length: 43
 
The quick green fox jumps over the lazy dog.
Back to Top

Overwriting part of an object

An example of using the ViPR API extensions to overwrite part of an object is provided below.

You can overwrite part of an object by providing only the starting offset in the data request. The data in the request will be written starting at the provided offset. The format is: Range: <startingOffset>-

For example, to write the data brown cat starting at offset 10, you would issue this PUT request: 

PUT /bucket1/object1 HTTP/1.1
Content-Length: 9
Range: bytes=10-
ACCEPT: application/json,application/xml,text/html,application/octet-stream
Date: Mon, 17 Jun 2013 20:51:41 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:uwPjDAgmazCP5lu77Zvbo+CiT4Q=
Accept-Encoding: gzip, deflate, compress
 
brown cat
 
HTTP/1.1 204 No Content
ETag: 25
x-amz-id-2: object1
x-amz-request-id: 65be45c2-0ee8-448a-a5a0-fff82573aa3b
Content-Length: 0
Date: Mon, 17 Jun 2013 20:51:41 GMT

When retrieving the object, you can see the final value The quick brown cat jumps over the lazy dog and cat. (green fox has been replaced with brown cat). You have overwritten part of the data in this object at the provided starting offset. 

GET /bucket1/object1 HTTP/1.1
Date: Mon, 17 Jun 2013 20:51:55 -0000
x-emc-namespace: emc
Content-Type: application/octet-stream
Authorization: AWS wuser1:/UQpdxNqZtyDkzGbK169GzhZmt4=
Accept-Encoding: gzip, deflate, compress
 
HTTP/1.1 200 OK
Date: Mon, 17 Jun 2013 20:51:55 GMT
Content-Type: application/octet-stream
Last-Modified: Mon, 17 Jun 2013 20:51:41 GMT
ETag: 25
Content-Type: application/json
Content-Length: 51
 
The quick brown cat jumps over the lazy dog and cat.
Back to Top