com.oracle.bmc.hdfs.store

Class BmcDataStore

java.lang.Object

com.oracle.bmc.hdfs.store.BmcDataStore

public class BmcDataStore
extends Object

BmcDataStore is a facade to Object Store that provides CRUD operations for objects by Path references.

Statistics are updated only on successful operations, and not on attempted operations.

Constructor Summary

Constructors

Constructor and Description
BmcDataStore(BmcPropertyAccessor propertyAccessor, com.oracle.bmc.objectstorage.ObjectStorage objectStorage, String namespace, String bucket, FileSystem.Statistics statistics)

Method Summary

Modifier and Type	Method and Description
static String	configureParquetCacheString(BmcPropertyAccessor propertyAccessor)
void	createDirectory(Path path) Creates a pseudo directory at the given path.
void	delete(Path path) Deletes the object at the given path.
void	deleteDirectory(Path path) Deletes the directory at the given path.
org.apache.commons.lang3.tuple.Pair<List<FileStatus>,String>	flatListDirectoryRecursive(Path path, String nextToken) A method to list all files/dirs in a given directory in a flat manner.
long	getBlockSizeInBytes() Gets the configured block size in bytes.
ContentSummary	getContentSummary(Path path)
FileStatus	getFileStatus(Path path) Returns the FileStatus for the object at the given path.
static int	getReadAheadSizeInBytes(BmcPropertyAccessor propertyAccessor)
boolean	isEmptyDirectory(Path path) Tests to see if the directory at the given path is considered empty or not.
List<FileStatus>	listDirectory(Path path) Returns the status of each entry in the directory specified.
FSInputStream	openReadStream(FileStatus status, Path path, int bufferSizeInBytes, FileSystem.Statistics statistics) Creates a new FSInputStream that can be used to read the object at the given path.
OutputStream	openWriteStream(Path path, int bufferSizeInBytes, Progressable progress) Creates a new OutputStream that can be written to in order to create a new file.
void	renameDirectory(Path sourceDirectoryPath, Path destinationDirectoryPath) Renames a directory to have a new location or name.
void	renameFile(Path source, Path destination) Renames an object from one name to another.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BmcDataStore

public BmcDataStore(BmcPropertyAccessor propertyAccessor,
                    com.oracle.bmc.objectstorage.ObjectStorage objectStorage,
                    String namespace,
                    String bucket,
                    FileSystem.Statistics statistics)

Method Detail

getReadAheadSizeInBytes

public static int getReadAheadSizeInBytes(BmcPropertyAccessor propertyAccessor)

configureParquetCacheString

public static String configureParquetCacheString(BmcPropertyAccessor propertyAccessor)

renameFile

public void renameFile(Path source,
                       Path destination)
                throws IOException

Renames an object from one name to another. This is a multi-step operation that consists of finding all matching objects, copying them to the destination, and then deleting the original objects.

Parameters:

source - The source to rename, assumed to exist.

destination - The destination, may not exist, will be overwritten

Throws:

IOException - if the operation cannot be completed.

renameDirectory

public void renameDirectory(Path sourceDirectoryPath,
                            Path destinationDirectoryPath)
                     throws IOException

Renames a directory to have a new location or name. This will move all objects that were considered part of the original directory to be in the new one (all objects that matched the path prefix). This is a multi-step operation that consists of finding all matching objects, copying them to the destination, and then deleting the original objects.

Note, source and destination should not be root TODO: destination could be?

Parameters:

sourceDirectoryPath - The source directory to rename, assumed to exist.

destinationDirectoryPath - The destination directory.

Throws:

IOException - if the operation cannot be completed.

delete

public void delete(Path path)
            throws IOException

Deletes the object at the given path.

Parameters:

path - Path of object to delete.

Throws:

IOException - if the operation cannot be completed.

deleteDirectory

public void deleteDirectory(Path path)
                     throws IOException

Deletes the directory at the given path.

Parameters:

path - Path of object to delete.

Throws:

IOException - if the operation cannot be completed.

createDirectory

public void createDirectory(Path path)
                     throws IOException

Creates a pseudo directory at the given path.

Parameters:

path - The path to create a directory object at.

Throws:

IOException - if the operation cannot be completed.

isEmptyDirectory

public boolean isEmptyDirectory(Path path)
                         throws IOException

Tests to see if the directory at the given path is considered empty or not.

Parameters:

path - The directory path.

Returns:

true if the directory is empty, false if not.

Throws:

IOException - if the operation could not be completed.

listDirectory

public List<FileStatus> listDirectory(Path path)
                               throws IOException

Returns the status of each entry in the directory specified.

Parameters:

path - The directory path.

Returns:

A list of file statuses, or empty if the directory was empty.

Throws:

IOException - if the operation could not be completed

flatListDirectoryRecursive

public org.apache.commons.lang3.tuple.Pair<List<FileStatus>,String> flatListDirectoryRecursive(Path path,
                                                                                               String nextToken)
                                                                                        throws IOException

A method to list all files/dirs in a given directory in a flat manner. This is done without using any delimiters in the OSS list objects API.

Parameters:

path - The path to the directory for which the listing needs to be done.

nextToken - This is the token string in order to continue with a next page of results. It should be passed null in the first call to start fresh.

Returns:

A pair containing list of FileStatus objects and a possible token to the next page.

Throws:

IOException

getContentSummary

public ContentSummary getContentSummary(Path path)
                                 throws IOException

Throws:

IOException

getFileStatus

public FileStatus getFileStatus(Path path)
                         throws IOException

Returns the FileStatus for the object at the given path.

Parameters:

path - The path to query.

Returns:

The file status, null if there was no file at this location.

Throws:

IOException - if the operation could not be completed.

openReadStream

public FSInputStream openReadStream(FileStatus status,
                                    Path path,
                                    int bufferSizeInBytes,
                                    FileSystem.Statistics statistics)

Creates a new FSInputStream that can be used to read the object at the given path.

Parameters:

status - The file status for this file.

path - The path to open.

bufferSizeInBytes - The buffer size in bytes (may not be used).

statistics - The FileSystem.Statistics instance to publish metrics into.

Returns:

A new input stream to read from.

openWriteStream

public OutputStream openWriteStream(Path path,
                                    int bufferSizeInBytes,
                                    Progressable progress)

Creates a new OutputStream that can be written to in order to create a new file.

Parameters:

path - The path for the new file.

bufferSizeInBytes - The buffer size in bytes can be configured by setting the config key BmcConstants.MULTIPART_PART_SIZE_IN_MB_KEY. Default value is 128MiB which comes from OCI Java SDK UploadConfiguration

progress - Progressable instance to report progress updates to.

Returns:

A new output stream to write to.

getBlockSizeInBytes

public long getBlockSizeInBytes()

Gets the configured block size in bytes.

Returns:

Block size in bytes.