Package org.cdlib.mrt.s3.openstack

Class OpenstackCloud

java.lang.Object

org.cdlib.mrt.s3.service.CloudStoreAbs

org.cdlib.mrt.s3.openstack.OpenstackCloud

All Implemented Interfaces:

CloudStoreInf

public class OpenstackCloud
extends CloudStoreAbs
implements CloudStoreInf

Specific SDSC Storage Cloud handling

Author:

dloy

Nested Class Summary

Nested classes/interfaces inherited from interface org.cdlib.mrt.s3.service.CloudStoreInf

CloudStoreInf.CloudAPI

Field Summary

Fields

Modifier and Type	Field	Description
protected org.cdlib.mrt.openstack.utility.OpenStackCmdAbs	cmd

Fields inherited from class org.cdlib.mrt.s3.service.CloudStoreAbs

cloudProp, endpointHostname, log4j, logger, MESSAGE, NAME

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	OpenstackCloud(InputStream propStream, org.cdlib.mrt.utility.LoggerInf logger)
protected	OpenstackCloud(Properties prop, org.cdlib.mrt.utility.LoggerInf logger)

Method Summary

Modifier and Type	Method	Description
protected void	addListEntries(org.cdlib.mrt.cloud.CloudList totalList, org.cdlib.mrt.cloud.CloudList list)
CloudResponse	deleteManifest(String container, org.cdlib.mrt.core.Identifier objectID)	Delete cloud manifest
CloudResponse	deleteObject(String container, String key)	Delete object
CloudResponse	deleteObject(String container, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID)	Delete cloud object
InputStream	getManifest(String container, org.cdlib.mrt.core.Identifier objectID)	Retrieve cloud manifest
InputStream	getManifest(String container, org.cdlib.mrt.core.Identifier objectID, CloudResponse response)	Retrieve cloud manifest
void	getObject(String container, String key, File outFile, CloudResponse response)	Retrieve content into a file
InputStream	getObject(String container, String key, CloudResponse response)	Retrieve cloud object
InputStream	getObject(String container, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID)
InputStream	getObject(String container, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID, CloudResponse response)	Retrieve cloud Object
CloudResponse	getObjectList(String container)	Return info on bucket/container
CloudResponse	getObjectList(String container, String prefix)	Return content that matches for the length of the key
CloudResponse	getObjectList(String container, String marker, int limit)	Return a set number of entry metadata based on start key
CloudResponse	getObjectList(String container, org.cdlib.mrt.core.Identifier objectID, Integer versionID, String fileID)
CloudResponse	getObjectListAfter(String bucketName, String afterKey, int limit)	Return list of S3 keys after a start key
CloudResponse	getObjectListOriginal(String container, String prefix)
Properties	getObjectMeta(String bucketName, String key)	Retrieve metadata for file
InputStream	getObjectStreaming(String container, String key, CloudResponse response)	Retrieve cloud object in streaming mode
static OpenstackCloud	getOpenstack(InputStream propStream, org.cdlib.mrt.utility.LoggerInf logger)
static OpenstackCloud	getOpenstackCloud(Properties prop, org.cdlib.mrt.utility.LoggerInf logger)
CloudResponse	getPreSigned(long expirationMinutes, String bucketName, String key, String contentType, String contentDisp)	Returns an S3 presigned URL
InputStream	getRangeStream(String bucketName, String key, long start, long stop, CloudResponse response)	Return part of a cloud object as stream
StateHandler.RetState	getState(String bucketName)	Return state of cloud store manager
CloudStoreInf.CloudAPI	getType()	Return the API type
Boolean	isAlive(String bucketName)	Determine if host:port is available
boolean	isAlphaNumericKey()	Because of earlier SDSC bug only alpha-numerics could be used in a key.
static boolean	isDistributedContainer(String container)	Is this a distributed container?
CloudResponse	putManifest(String container, org.cdlib.mrt.core.Identifier objectID, File inputFile)	Upload cloud manifest
CloudResponse	putObject(String container, String key, File inputFile)	Upload cloud object
CloudResponse	putObject(String container, String key, File inputFile, Properties fileMeta)
CloudResponse	putObject(String container, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID, File inputFile)	Upload cloud object
CloudResponse	putObject(String container, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID, File inputFile, Properties fileMeta)
CloudResponse	putObject(CloudResponse response, File inputFile)
void	restoreObject(String container, String key, File outFile, CloudResponse response)	Restore nearline content
String	setContainer(String container, String key, boolean addNewContainer)	setContainer determines if the passed container is distributed and will automatically append digest characters to generate physical container
protected void	setStorageService()
CloudResponse	validateMd5(String container, String key, String inMd5)	Validate that the md5 digest matches cloud object digest

Methods inherited from class org.cdlib.mrt.s3.service.CloudStoreAbs

dumpException, getCloudProp, handleException, isAliveTest, isValidFile, validateDigest

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.cdlib.mrt.s3.service.CloudStoreInf

getCloudProp, validateDigest

Field Details

cmd

protected org.cdlib.mrt.openstack.utility.OpenStackCmdAbs cmd

Constructor Details

OpenstackCloud

protected OpenstackCloud(InputStream propStream,
 org.cdlib.mrt.utility.LoggerInf logger)
                  throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

OpenstackCloud

protected OpenstackCloud(Properties prop,
 org.cdlib.mrt.utility.LoggerInf logger)
                  throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

Method Details

getOpenstack

public static OpenstackCloud getOpenstack(InputStream propStream,
 org.cdlib.mrt.utility.LoggerInf logger)
                                   throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

getOpenstackCloud

public static OpenstackCloud getOpenstackCloud(Properties prop,
 org.cdlib.mrt.utility.LoggerInf logger)
                                        throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

setStorageService

protected void setStorageService()
                          throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

putObject

public CloudResponse putObject(CloudResponse response,
 File inputFile)
                        throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

putObject

public CloudResponse putObject(String container,
 String key,
 File inputFile)
                        throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Upload cloud object

Specified by:

putObject in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

key - entry key to be added

inputFile - file to be uploaded

Returns:

information about upload

Throws:

org.cdlib.mrt.utility.TException

putObject

public CloudResponse putObject(String container,
 org.cdlib.mrt.core.Identifier objectID,
 int versionID,
 String fileID,
 File inputFile)
                        throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Upload cloud object

Specified by:

putObject in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

objectID - Object identifier

versionID - Version identifier

fileID - File/component identifier that may contain a directory element

inputFile - file to be uploaded

Returns:

information about upload

Throws:

org.cdlib.mrt.utility.TException

putObject

public CloudResponse putObject(String container,
 String key,
 File inputFile,
 Properties fileMeta)
                        throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

putObject

public CloudResponse putObject(String container,
 org.cdlib.mrt.core.Identifier objectID,
 int versionID,
 String fileID,
 File inputFile,
 Properties fileMeta)
                        throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

putManifest

public CloudResponse putManifest(String container,
 org.cdlib.mrt.core.Identifier objectID,
 File inputFile)
                          throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Upload cloud manifest

Specified by:

putManifest in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

objectID - Object identifier

inputFile - manifest file to be uploaded

Returns:

information about upload

Throws:

org.cdlib.mrt.utility.TException

deleteObject

public CloudResponse deleteObject(String container,
 String key)
                           throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Delete object

Specified by:

deleteObject in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

key - cloud object key

Returns:

Throws:

org.cdlib.mrt.utility.TException

deleteObject

public CloudResponse deleteObject(String container,
 org.cdlib.mrt.core.Identifier objectID,
 int versionID,
 String fileID)
                           throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Delete cloud object

Specified by:

deleteObject in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

objectID - Object identifier

versionID - Version identifier

fileID - File/component identifier that may contain a directory element

Returns:

Throws:

org.cdlib.mrt.utility.TException

deleteManifest

public CloudResponse deleteManifest(String container,
 org.cdlib.mrt.core.Identifier objectID)
                             throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Delete cloud manifest

Specified by:

deleteManifest in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

objectID - Object identifier

Returns:

Throws:

org.cdlib.mrt.utility.TException

getObject

public InputStream getObject(String container,
 org.cdlib.mrt.core.Identifier objectID,
 int versionID,
 String fileID,
 CloudResponse response)
                      throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Retrieve cloud Object

Specified by:

getObject in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

objectID - Object identifier

versionID - Version identifier

fileID - File/component identifier that may contain a directory element

response - input stream of retrieved object

Returns:

Throws:

org.cdlib.mrt.utility.TException

getObject

public InputStream getObject(String container,
 String key,
 CloudResponse response)
                      throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Retrieve cloud object

Specified by:

getObject in interface CloudStoreInf

Specified by:

getObject in class CloudStoreAbs

Parameters:

container - s3 bucket - rackspace container

key - cloud object key for object to be returned

response -

Returns:

input stream of retrieved object

Throws:

org.cdlib.mrt.utility.TException

getObjectStreaming

public InputStream getObjectStreaming(String container,
 String key,
 CloudResponse response)
                               throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Retrieve cloud object in streaming mode

Specified by:

getObjectStreaming in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

key - cloud object key for object to be returned

response -

Returns:

input stream of retrieved object

Throws:

org.cdlib.mrt.utility.TException

getObject

public void getObject(String container,
 String key,
 File outFile,
 CloudResponse response)
               throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Retrieve content into a file

Specified by:

getObject in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

key - cloud object key for object to be returned

outFile - file to receive content

response -

Throws:

org.cdlib.mrt.utility.TException

restoreObject

public void restoreObject(String container,
 String key,
 File outFile,
 CloudResponse response)
                   throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Restore nearline content

Specified by:

restoreObject in interface CloudStoreInf

Parameters:

key - cloud key

outFile - target file of response

response -

Throws:

org.cdlib.mrt.utility.TException

getObjectMeta

public Properties getObjectMeta(String bucketName,
 String key)
                         throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Retrieve metadata for file

Specified by:

getObjectMeta in interface CloudStoreInf

Parameters:

bucketName - s3 bucket - rackspace container

key - cloud object key for object to be returned

Returns:

exists: named properties; does not exist: null

Throws:

org.cdlib.mrt.utility.TException

getManifest

public InputStream getManifest(String container,
 org.cdlib.mrt.core.Identifier objectID)
                        throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Retrieve cloud manifest

Specified by:

getManifest in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

objectID - Object identifier

Returns:

input stream of retrieved manifest

Throws:

org.cdlib.mrt.utility.TException

getManifest

public InputStream getManifest(String container,
 org.cdlib.mrt.core.Identifier objectID,
 CloudResponse response)
                        throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Retrieve cloud manifest

Specified by:

getManifest in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

objectID - Object identifier

response -

Returns:

input stream of retrieved object

Throws:

org.cdlib.mrt.utility.TException

getObject

public InputStream getObject(String container,
 org.cdlib.mrt.core.Identifier objectID,
 int versionID,
 String fileID)
                      throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

getObjectList

public CloudResponse getObjectList(String container,
 org.cdlib.mrt.core.Identifier objectID,
 Integer versionID,
 String fileID)
                            throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

getObjectListOriginal

public CloudResponse getObjectListOriginal(String container,
 String prefix)
                                    throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

getObjectList

public CloudResponse getObjectList(String container,
 String prefix)
                            throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Return content that matches for the length of the key

Specified by:

getObjectList in interface CloudStoreInf

Specified by:

getObjectList in class CloudStoreAbs

Parameters:

container - s3 bucket - rackspace container

prefix - prefix of content to be returned

Returns:

CloudResponse with list of matching entries

Throws:

org.cdlib.mrt.utility.TException

addListEntries

protected void addListEntries(org.cdlib.mrt.cloud.CloudList totalList,
 org.cdlib.mrt.cloud.CloudList list)
                       throws org.cdlib.mrt.utility.TException

Throws:

org.cdlib.mrt.utility.TException

getObjectList

public CloudResponse getObjectList(String container,
 String marker,
 int limit)
                            throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Return a set number of entry metadata based on start key

Specified by:

getObjectList in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

marker - content key of the first output

limit - number of returned entries

Returns:

CloudResponse with list of sequential entries

Throws:

org.cdlib.mrt.utility.TException

getObjectList

public CloudResponse getObjectList(String container)
                            throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Return info on bucket/container

Specified by:

getObjectList in interface CloudStoreInf

Parameters:

container - s3 bucket - rackspace container

Returns:

Throws:

org.cdlib.mrt.utility.TException

getObjectListAfter

public CloudResponse getObjectListAfter(String bucketName,
 String afterKey,
 int limit)
                                 throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Return list of S3 keys after a start key

Specified by:

getObjectListAfter in interface CloudStoreInf

Parameters:

bucketName - s3 bucket

afterKey - start return list after this value

limit - number of keys to return

Returns:

cloud response with keys

Throws:

org.cdlib.mrt.utility.TException

getState

public StateHandler.RetState getState(String bucketName)
                               throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Return state of cloud store manager

Specified by:

getState in interface CloudStoreInf

Parameters:

bucketName - bucket container

Returns:

Throws:

org.cdlib.mrt.utility.TException

validateMd5

public CloudResponse validateMd5(String container,
 String key,
 String inMd5)
                          throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Validate that the md5 digest matches cloud object digest

Specified by:

validateMd5 in interface CloudStoreInf

Overrides:

validateMd5 in class CloudStoreAbs

Parameters:

container - s3 bucket - rackspace container

key - cloud object key

inMd5 - md5 digest value

Returns:

Throws:

org.cdlib.mrt.utility.TException

setContainer

public String setContainer(String container,
 String key,
 boolean addNewContainer)
                    throws org.cdlib.mrt.utility.TException

setContainer determines if the passed container is distributed and will automatically append digest characters to generate physical container

Parameters:

container - physical or distributed container

key - delimited component key - distributed extension ins built off first element of key

addNewContainer - true=add new container if not present, false=ignore add

Returns:

physical key

Throws:

org.cdlib.mrt.utility.TException

isDistributedContainer

public static boolean isDistributedContainer(String container)
                                      throws org.cdlib.mrt.utility.TException

Is this a distributed container? Distributed container has a common prefix with trailing trigger

Parameters:

container -

Returns:

true=distributed container, false=physical container

Throws:

org.cdlib.mrt.utility.TException

isAlive

public Boolean isAlive(String bucketName)

Description copied from interface: CloudStoreInf

Determine if host:port is available

Specified by:

isAlive in interface CloudStoreInf

Returns:

true=alive; false=not alive; null=test not performed

getPreSigned

public CloudResponse getPreSigned(long expirationMinutes,
 String bucketName,
 String key,
 String contentType,
 String contentDisp)
                           throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Returns an S3 presigned URL

Specified by:

getPreSigned in interface CloudStoreInf

Parameters:

expirationMinutes - minutes for the signed URL to work

bucketName - cloud container

key - cloud key

contentType - optional ContentType for this file

contentDisp - optional ContentDisposition for this file

Returns:

CloudResponse where response.getReturnURL returns the presigned URL

Throws:

org.cdlib.mrt.utility.TException

getRangeStream

public InputStream getRangeStream(String bucketName,
 String key,
 long start,
 long stop,
 CloudResponse response)
                           throws org.cdlib.mrt.utility.TException

Description copied from interface: CloudStoreInf

Return part of a cloud object as stream

Specified by:

getRangeStream in interface CloudStoreInf

Parameters:

bucketName - cloud container

key - cloud key

start - byte location to start from zero

stop - byte location to stop from zero

response - Res[pmse tp reqiest

Returns:

input stream for segment

Throws:

org.cdlib.mrt.utility.TException

getType

public CloudStoreInf.CloudAPI getType()

Description copied from interface: CloudStoreInf

Return the API type

Specified by:

getType in interface CloudStoreInf

Returns:

AWS-S3, SDSC-Swift, Cloudhost, Pairtree

isAlphaNumericKey

public boolean isAlphaNumericKey()

Description copied from interface: CloudStoreInf

Because of earlier SDSC bug only alpha-numerics could be used in a key. true=alpha-numeric key, false=ASCII (current default)

Specified by:

isAlphaNumericKey in interface CloudStoreInf

Returns: