ECS 2.0 – Address ECS object storage and set the Base URL

Table of Contents

Introduction

Applications that are written to use Amazon S3 can be enabled to use ECS object storage by setting the Base URL parameter. The Base URL is set by default to amazonaws.com. This article describes how to set the Base URL and ensure that requests are routed to ECS.

The following sections describe the addressing scheme supported by ECS, the use of the Base URL parameter, and the mechanism for setting the Base URL parameter.
Back to Top

Bucket addressing

The ECS S3 service provides a number of ways in which to identify the bucket against which the operation defined in a request should be performed.

When using the Amazon S3 service, all buckets names must be unique. However, the ECS S3 service supports the use of a namespace, which can be used in addition to the bucket name and allows buckets in different namespaces to have the same name. By assigning a namespace to each tenant, a tenant can assign bucket names without regard for the names currently used by other tenants. If no namespace is specified in a request, ECS uses the default namespace associated with the tenant to which the user making the request belongs.

The namespace that refers to the location of an object can be specified in the x-emc-namespace header of an HTTP request. ECS also supports extraction of the location from the host header and allows the following Amazon S3 compatible addressing schemes:

Virtual Host Style Addressing

In the virtual host addressing scheme, the bucket name appears in the hostname. For example, the bucket called "mybucket" on host ecs1.yourco.com, would be accessed using:
http://mybucket.ecs1.yourco.com

In addition, ECS also allows the inclusion of a namespace in the address. For example:

<bucketname>.<namespace>.ecs1.yourco.com

To use this style of addressing, you need to configure ECS so that it knows which part of the URL is the bucket name. This is done by configuring the Base URL. In addition, you need to ensure that your DNS system can resolve the address. The following sections provide more information:

Path Based Addressing

In the path based addressing scheme, the bucket name is added to the end of the path. For example:
ecs1.yourco.com/mybucket
When including a namespace, use the following format:
ecs1.yourco.com/mynamespace/mybucket

Back to Top

DNS Configuration

When accessing ECS storage using the S3 service, you will need to ensure that the URL resolves to the address of the ECS data node, or the data node load balancer.

Where your application uses path-style addressing, this is simply a case of ensuring that the base name is resolvable by DNS. For example, if your application normally issues requests in the form ecs1.yourco.com/bucket, you will need to have a DNS entry that resolves ecs1.yourco.com to the IP address of your load balancer used for access to ECS nodes. If you are using the Amazon service this URI will be of the form: s3-eu-west-1.amazonaws.com.

Where your application is using virtual host style addressing, the URL will include the bucket name and can include a namespace. Under these circumstances, you will need to ensure that you include a DNS entry that will resolve the virtual host style address. You can do this by using a wildcard in the DNS entry.

For example, if your application normally issues requests in the form bucket.s3.yourco.com, you will need to have two DNS entries.

  • ecs1.yourco.com
  • *.ecs1.yourco.com

Or, if If you are using an application that previously connected to the Amazon S3 service, using bucket.s3.amazonaws.com, the entries would be:

  • s3.amazonaws.com
  • *.s3.amazonaws.com

These entries allow the base name to be resolved when issuing service-level commands (for example, list buckets) and the virtual host style bucket address to be resolved.

If you are creating an SSL certificate for this service, it should have the wildcard entry on the name of the certificate and the non-wildcard version as a Subject Alternate Name.

Back to Top

Base URL

If you have an S3 application that uses virtual host style addressing and you want to use it to connect to ECS, the Base URL must be set to enable ECS to know which part of the address refers to the bucket and, optionally, namespace. The Base URL can be set using the ECS Portal, or using the ECS Management REST API, and requires the ECS System Administrator role.

The Base URL Management page shows the Base URLs that have been created and how ECS should use the them.

In order that ECS knows how to treat the bucket location prefix, the Base URL must be configured by choosing one of the following options.

  • Use Base URL with namespace
  • Use Base URL without namespace

When processing a request, ECS will:

  1. Try to extract namespace from the x-emc-namespace header. If found, skip the steps below and process the request.
  2. Get the hostname of the URL from the host header and check if the last part of the address matches any of the configured Base URLs.
  3. Where there is a BaseURL match, use the prefix part of the hostname (the part left when the Base URL is removed), to obtain the bucket location.

The following examples demonstrate how ECS handles incoming HTTP requests with different structures.

Example1

Host:          baseball.image.emc.finance.com
BaseURL:       finance.com
Use BaseURL with namespace enabled

Namespace:     emc
Bucket Name:   baseball.image

Example 2

Host:          baseball.image.emc.finance.com
BaseURL:       finance.com
Use BaseURL without namespace enabled

Namespace:     null (Use other methods to determine namespace)
Bucket Name:   baseball.image.emc 
			

Example 3

Host:          baseball.image.emc.finance.com
BaseURL:       not configured

Namespace:     null (Use other methods to determine namespace.)
Bucket Name:   null (Use other methods to determine the bucket name.)

ViPR treats this request as a path-style request.
						

Back to Top

Add a Base URL

This operation is only necessary if you use object clients that encode the location of an object, its namespace and bucket, in a URL. In that case you can specify a base URL that will be used, together with the namespace, as the path to objects in a tenant.

Before you begin

This operation requires the System Admin role in ECS.

You must ensure that the domain specified in a request that uses a URL to specify an object location resolves to the location of the ECS data node or a load balancer that sits in front of the data nodes.

Procedure

  1. At the ECS Portal, select Settings > Object Base URLs.
  2. Select New Base URL.

    The New Base URL page is displayed.

  3. Enter the name of the Base URL. This will provide additional information about the base URL when looking at the base URL table.
  4. Enter the Base URL.
    If your objects location URLs are in the form: https://mybucket.mynamespace.acme.com (that is, bucket.namespace.baseurl ) or https://mybucket.acme.com (that is, bucket.baseurl), the base URL would be acme.com.
    You can specify which format in the Namespace selector.
  5. Choose the format in which your object address is encoded in the URL: with a namespace or without a namespace.
  6. Select Save.
Back to Top

Example of a Base URL

As an example of the use of the Base URL, an object named myphoto.jpg in the images bucket with a base URL of s3.amazonaws.com and a tenant namespace of "ecsdata", would be addressable using the following URL: https://images.ecsdata.s3.amazonaws.com/myphoto.jpg

Base URL example

Back to Top