Package org.cdlib.mrt.s3.pairtree

Class PairtreeCloud

java.lang.Object
org.cdlib.mrt.s3.service.CloudStoreAbs
org.cdlib.mrt.s3.pairtree.PairtreeCloud
All Implemented Interfaces:
CloudStoreInf
public class PairtreeCloud extends CloudStoreAbs implements CloudStoreInf
Specific SDSC Storage Cloud handling
Author:
dloy

  • Constructor Details

    • PairtreeCloud

      protected PairtreeCloud(boolean pairPath, org.cdlib.mrt.utility.LoggerInf logger) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

  • Method Details

    • getPairtreeCloud

      public static PairtreeCloud getPairtreeCloud(boolean pairPath, org.cdlib.mrt.utility.LoggerInf logger) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • getBase

      protected File getBase(String filePath) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • getDir

      protected File getDir(File baseDir, String key) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • getDirRead

      protected File getDirRead(File baseDir, String key) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • buildPairDirectoryRetry

      public static File buildPairDirectoryRetry(File baseDir, String pairKey, int retryCnt) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • putObject

      protected CloudResponse putObject(CloudResponse response, File inputFile) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • putObjectSave

      protected CloudResponse putObjectSave(CloudResponse response, File inputFile) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • putObject

      public CloudResponse putObject(String bucketName, String key, File inputFile) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Upload cloud object
      Specified by:
      putObject in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - entry key to be added
      inputFile - file to be uploaded
      Returns:
      information about upload
      Throws:
      org.cdlib.mrt.utility.TException

    • putObject

      public CloudResponse putObject(String bucketName, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID, File inputFile) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Upload cloud object
      Specified by:
      putObject in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      objectID - Object identifier
      versionID - Version identifier
      fileID - File/component identifier that may contain a directory element
      inputFile - file to be uploaded
      Returns:
      information about upload
      Throws:
      org.cdlib.mrt.utility.TException

    • putObject

      public CloudResponse putObject(String bucketName, String key, File inputFile, Properties cloudProperties) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • putObject

      public CloudResponse putObject(String bucketName, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID, File inputFile, Properties cloudProperties) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • putManifest

      public CloudResponse putManifest(String bucketName, org.cdlib.mrt.core.Identifier objectID, File inputFile) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Upload cloud manifest
      Specified by:
      putManifest in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      objectID - Object identifier
      inputFile - manifest file to be uploaded
      Returns:
      information about upload
      Throws:
      org.cdlib.mrt.utility.TException

    • deleteObject

      public CloudResponse deleteObject(String bucketName, String key) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Delete object
      Specified by:
      deleteObject in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - cloud object key
      Returns:
      Throws:
      org.cdlib.mrt.utility.TException

    • deleteObject

      public CloudResponse deleteObject(String bucketName, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Delete cloud object
      Specified by:
      deleteObject in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      objectID - Object identifier
      versionID - Version identifier
      fileID - File/component identifier that may contain a directory element
      Returns:
      Throws:
      org.cdlib.mrt.utility.TException

    • deleteManifest

      public CloudResponse deleteManifest(String bucketName, org.cdlib.mrt.core.Identifier objectID) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Delete cloud manifest
      Specified by:
      deleteManifest in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      objectID - Object identifier
      Returns:
      Throws:
      org.cdlib.mrt.utility.TException

    • getObject

      public InputStream getObject(String bucketName, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID, CloudResponse response) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Retrieve cloud Object
      Specified by:
      getObject in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      objectID - Object identifier
      versionID - Version identifier
      fileID - File/component identifier that may contain a directory element
      response - input stream of retrieved object
      Returns:
      Throws:
      org.cdlib.mrt.utility.TException

    • getObject

      public InputStream getObject(String bucketName, String key, CloudResponse response) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Retrieve cloud object
      Specified by:
      getObject in interface CloudStoreInf
      Specified by:
      getObject in class CloudStoreAbs
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - cloud object key for object to be returned
      Returns:
      input stream of retrieved object
      Throws:
      org.cdlib.mrt.utility.TException

    • getObjectStreaming

      public InputStream getObjectStreaming(String container, String key, CloudResponse response) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Retrieve cloud object in streaming mode
      Specified by:
      getObjectStreaming in interface CloudStoreInf
      Parameters:
      container - s3 bucket - rackspace container
      key - cloud object key for object to be returned
      Returns:
      input stream of retrieved object
      Throws:
      org.cdlib.mrt.utility.TException

    • getObject

      public void getObject(String bucketName, String key, File outFile, CloudResponse response) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Retrieve content into a file
      Specified by:
      getObject in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - cloud object key for object to be returned
      outFile - file to receive content
      Throws:
      org.cdlib.mrt.utility.TException

    • restoreObject

      public void restoreObject(String container, String key, File outFile, CloudResponse response) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Restore nearline content
      Specified by:
      restoreObject in interface CloudStoreInf
      key - cloud key
      outFile - target file of response
      Throws:
      org.cdlib.mrt.utility.TException

    • getObjectMeta

      public Properties getObjectMeta(String bucketName, String key) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Retrieve metadata for file
      Specified by:
      getObjectMeta in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - cloud object key for object to be returned
      Returns:
      exists: named properties; does not exist: null
      Throws:
      org.cdlib.mrt.utility.TException

    • getManifest

      public InputStream getManifest(String bucketName, org.cdlib.mrt.core.Identifier objectID) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Retrieve cloud manifest
      Specified by:
      getManifest in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      objectID - Object identifier
      Returns:
      input stream of retrieved manifest
      Throws:
      org.cdlib.mrt.utility.TException

    • getManifest

      public InputStream getManifest(String bucketName, org.cdlib.mrt.core.Identifier objectID, CloudResponse response) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Retrieve cloud manifest
      Specified by:
      getManifest in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      objectID - Object identifier
      Returns:
      input stream of retrieved object
      Throws:
      org.cdlib.mrt.utility.TException

    • getObject

      public InputStream getObject(String bucketName, org.cdlib.mrt.core.Identifier objectID, int versionID, String fileID) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • getObjectList

      public CloudResponse getObjectList(String bucketName, org.cdlib.mrt.core.Identifier objectID, Integer versionID, String fileID) throws org.cdlib.mrt.utility.TException
      Throws:
      org.cdlib.mrt.utility.TException

    • getObjectList

      public CloudResponse getObjectList(String bucketName, String key) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Return content that matches for the length of the key
      Specified by:
      getObjectList in interface CloudStoreInf
      Specified by:
      getObjectList in class CloudStoreAbs
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - prefix of content to be returned
      Returns:
      CloudResponse with list of matching entries
      Throws:
      org.cdlib.mrt.utility.TException

    • getObjectList

      public CloudResponse getObjectList(String bucketName, String key, int limit) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Return a set number of entry metadata based on start key
      Specified by:
      getObjectList in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - content key of the first output
      limit - number of returned entries
      Returns:
      CloudResponse with list of sequential entries
      Throws:
      org.cdlib.mrt.utility.TException

    • getObjectList

      public CloudResponse getObjectList(String bucketName) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Return info on bucket/container
      Specified by:
      getObjectList in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket - rackspace container
      Returns:
      Throws:
      org.cdlib.mrt.utility.TException

    • getObjectListAfter

      public CloudResponse getObjectListAfter(String bucketName, String afterKey, int limit) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Return list of S3 keys after a start key
      Specified by:
      getObjectListAfter in interface CloudStoreInf
      Parameters:
      bucketName - s3 bucket
      afterKey - start return list after this value
      limit - number of keys to return
      Returns:
      cloud response with keys
      Throws:
      org.cdlib.mrt.utility.TException

    • getState

      public StateHandler.RetState getState(String bucketName) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Return state of cloud store manager
      Specified by:
      getState in interface CloudStoreInf
      Parameters:
      bucketName - bucket container
      Returns:
      Throws:
      org.cdlib.mrt.utility.TException

    • validateMd5

      public CloudResponse validateMd5(String bucketName, String key, String inMd5) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Validate that the md5 digest matches cloud object digest
      Specified by:
      validateMd5 in interface CloudStoreInf
      Overrides:
      validateMd5 in class CloudStoreAbs
      Parameters:
      bucketName - s3 bucket - rackspace container
      key - cloud object key
      inMd5 - md5 digest value
      Returns:
      Throws:
      org.cdlib.mrt.utility.TException

    • isAlive

      public Boolean isAlive(String bucketName)
      Description copied from interface: CloudStoreInf
      Determine if host:port is available
      Specified by:
      isAlive in interface CloudStoreInf
      Returns:
      true=alive; false=not alive; null=test not performed

    • getPreSigned

      public CloudResponse getPreSigned(long expirationMinutes, String bucketName, String key, String contentType, String contentDisp) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Returns an S3 presigned URL
      Specified by:
      getPreSigned in interface CloudStoreInf
      Parameters:
      expirationMinutes - minutes for the signed URL to work
      bucketName - cloud container
      key - cloud key
      contentType - optional ContentType for this file
      contentDisp - optional ContentDisposition for this file
      Returns:
      CloudResponse where response.getReturnURL returns the presigned URL
      Throws:
      org.cdlib.mrt.utility.TException

    • getRangeStream

      public InputStream getRangeStream(String bucketName, String key, long start, long stop, CloudResponse response) throws org.cdlib.mrt.utility.TException
      Description copied from interface: CloudStoreInf
      Return part of a cloud object as stream
      Specified by:
      getRangeStream in interface CloudStoreInf
      Parameters:
      bucketName - cloud container
      key - cloud key
      start - byte location to start from zero
      stop - byte location to stop from zero
      response - Res[pmse tp reqiest
      Returns:
      input stream for segment
      Throws:
      org.cdlib.mrt.utility.TException

    • getType

      public CloudStoreInf.CloudAPI getType()
      Description copied from interface: CloudStoreInf
      Return the API type
      Specified by:
      getType in interface CloudStoreInf
      Returns:
      AWS-S3, SDSC-Swift, Cloudhost, Pairtree

    • isAlphaNumericKey

      public boolean isAlphaNumericKey()
      Description copied from interface: CloudStoreInf
      Because of earlier SDSC bug only alpha-numerics could be used in a key. true=alpha-numeric key, false=ASCII (current default)
      Specified by:
      isAlphaNumericKey in interface CloudStoreInf
      Returns: