aiida.repository.backend package

Module for file repository backend implementations.

class aiida.repository.backend.AbstractRepositoryBackend

Bases: object

Class that defines the abstract interface for an object repository.

The repository backend only deals with raw bytes, both when creating new objects as well as when returning a stream or the content of an existing object. The encoding and decoding of the byte content should be done by the client upstream. The file repository backend is also not expected to keep any kind of file hierarchy but must be assumed to be a simple flat data store. When files are created in the file object repository, the implementation will return a string-based key with which the content of the stored object can be addressed. This key is guaranteed to be unique and persistent. Persisting the key or mapping it onto a virtual file hierarchy is again up to the client upstream.

__abstractmethods__ = frozenset({'_put_object_from_filelike', 'erase', 'has_object', 'initialise', 'is_initialised', 'uuid'})

__dict__ = mappingproxy({'__module__': 'aiida.repository.backend.abstract', '__doc__': 'Class that defines the abstract interface for an object repository.\n\n The repository backend only deals with raw bytes, both when creating new objects as well as when returning a stream\n or the content of an existing object. The encoding and decoding of the byte content should be done by the client\n upstream. The file repository backend is also not expected to keep any kind of file hierarchy but must be assumed\n to be a simple flat data store. When files are created in the file object repository, the implementation will return\n a string-based key with which the content of the stored object can be addressed. This key is guaranteed to be unique\n and persistent. Persisting the key or mapping it onto a virtual file hierarchy is again up to the client upstream.\n ', 'uuid': <abc.abstractproperty object>, 'initialise': <function AbstractRepositoryBackend.initialise>, 'is_initialised': <abc.abstractproperty object>, 'erase': <function AbstractRepositoryBackend.erase>, 'is_readable_byte_stream': <staticmethod object>, 'put_object_from_filelike': <function AbstractRepositoryBackend.put_object_from_filelike>, '_put_object_from_filelike': <function AbstractRepositoryBackend._put_object_from_filelike>, 'put_object_from_file': <function AbstractRepositoryBackend.put_object_from_file>, 'has_object': <function AbstractRepositoryBackend.has_object>, 'open': <function AbstractRepositoryBackend.open>, 'get_object_content': <function AbstractRepositoryBackend.get_object_content>, 'get_object_hash': <function AbstractRepositoryBackend.get_object_hash>, 'delete_object': <function AbstractRepositoryBackend.delete_object>, '__dict__': <attribute '__dict__' of 'AbstractRepositoryBackend' objects>, '__weakref__': <attribute '__weakref__' of 'AbstractRepositoryBackend' objects>, '__abstractmethods__': frozenset({'_put_object_from_filelike', 'initialise', 'erase', 'is_initialised', 'has_object', 'uuid'}), '_abc_impl': <_abc_data object>})

__module__ = 'aiida.repository.backend.abstract'

__weakref__

list of weak references to the object (if defined)

_abc_impl = <_abc_data object>

abstract _put_object_from_filelike(handle: BinaryIO) → str

delete_object(key: str)

Delete the object from the repository.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be deleted.

abstract erase() → None

Delete the repository itself and all its contents.

Note

This should not merely delete the contents of the repository but any resources it created. For example, if the repository is essentially a folder on disk, the folder itself should also be deleted, not just its contents.

get_object_content(key: str) → bytes

Return the content of a object identified by key.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

get_object_hash(key: str) → str

Return the SHA-256 hash of an object stored under the given key.

Important

A SHA-256 hash should always be returned, to ensure consistency across different repository implementations.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

abstract has_object(key: str) → bool

Return whether the repository has an object with the given key.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

True if the object exists, False otherwise.

abstract initialise(**kwargs) → None

Initialise the repository if it hasn’t already been initialised.

Parameters

kwargs – parameters for the initialisation.

abstract property is_initialised

Return whether the repository has been initialised.

static is_readable_byte_stream(handle) → bool

open(key: str) → Iterator[BinaryIO]

Open a file handle to an object stored under the given key.

Note

this should only be used to open a handle to read an existing file. To write a new file use the method put_object_from_filelike instead.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

yield a byte stream object.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

put_object_from_file(filepath: Union[str, pathlib.Path]) → str

Store a new object with contents of the file located at filepath on this file system.

Parameters

filepath – absolute path of file whose contents to copy to the repository.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

put_object_from_filelike(handle: BinaryIO) → str

Store the byte contents of a file in the repository.

Parameters

handle – filelike object with the byte content to be stored.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

abstract property uuid

Return the unique identifier of the repository.

class aiida.repository.backend.DiskObjectStoreRepositoryBackend(container)

Bases: aiida.repository.backend.abstract.AbstractRepositoryBackend

Implementation of the AbstractRepositoryBackend using the disk-object-store as the backend.

__abstractmethods__ = frozenset({})

__init__(container)

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.repository.backend.disk_object_store'

__str__() → str

Return the string representation of this repository.

_abc_impl = <_abc_data object>

_put_object_from_filelike(handle: BinaryIO) → str

Store the byte contents of a file in the repository.

Parameters

handle – filelike object with the byte content to be stored.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

property container

delete_object(key: str)

Delete the object from the repository.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be deleted.

erase()

Delete the repository itself and all its contents.

get_object_hash(key: str) → str

Return the SHA-256 hash of an object stored under the given key.

Important

A SHA-256 hash should always be returned, to ensure consistency across different repository implementations.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

FileNotFoundError – if the file does not exist.

has_object(key: str) → bool

Return whether the repository has an object with the given key.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

True if the object exists, False otherwise.

initialise(**kwargs) → None

Initialise the repository if it hasn’t already been initialised.

Parameters

kwargs – parameters for the initialisation.

property is_initialised

Return whether the repository has been initialised.

open(key: str) → Iterator[BinaryIO]

Open a file handle to an object stored under the given key.

Note

this should only be used to open a handle to read an existing file. To write a new file use the method put_object_from_filelike instead.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

yield a byte stream object.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

property uuid

Return the unique identifier of the repository.

class aiida.repository.backend.SandboxRepositoryBackend

Bases: aiida.repository.backend.abstract.AbstractRepositoryBackend

Implementation of the AbstractRepositoryBackend using a sandbox folder on disk as the backend.

__abstractmethods__ = frozenset({})

__del__()

Delete the entire sandbox folder if it was instantiated and still exists.

__init__()

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.repository.backend.sandbox'

__str__() → str

Return the string representation of this repository.

_abc_impl = <_abc_data object>

_put_object_from_filelike(handle: BinaryIO) → str

Store the byte contents of a file in the repository.

Parameters

handle – filelike object with the byte content to be stored.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

delete_object(key: str)

Delete the object from the repository.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be deleted.

erase()

Delete the repository itself and all its contents.

has_object(key: str) → bool

Return whether the repository has an object with the given key.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

True if the object exists, False otherwise.

initialise(**kwargs) → None

Initialise the repository if it hasn’t already been initialised.

Parameters

kwargs – parameters for the initialisation.

property is_initialised

Return whether the repository has been initialised.

open(key: str) → Iterator[BinaryIO]

Open a file handle to an object stored under the given key.

Note

this should only be used to open a handle to read an existing file. To write a new file use the method put_object_from_filelike instead.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

yield a byte stream object.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

property sandbox

Return the sandbox instance of this repository.

property uuid

Return the unique identifier of the repository.

Note

A sandbox folder does not have the concept of a unique identifier and so always returns None.

Submodules

Class that defines the abstract interface for an object repository.

The scope of this class is intentionally very narrow. Any backend implementation should merely provide the methods to store binary blobs, or “objects”, and return a string-based key that unique identifies the object that was just created. This key should then be able to be used to retrieve the bytes of the corresponding object or to delete it.

class aiida.repository.backend.abstract.AbstractRepositoryBackend

Bases: object

Class that defines the abstract interface for an object repository.

The repository backend only deals with raw bytes, both when creating new objects as well as when returning a stream or the content of an existing object. The encoding and decoding of the byte content should be done by the client upstream. The file repository backend is also not expected to keep any kind of file hierarchy but must be assumed to be a simple flat data store. When files are created in the file object repository, the implementation will return a string-based key with which the content of the stored object can be addressed. This key is guaranteed to be unique and persistent. Persisting the key or mapping it onto a virtual file hierarchy is again up to the client upstream.

__abstractmethods__ = frozenset({'_put_object_from_filelike', 'erase', 'has_object', 'initialise', 'is_initialised', 'uuid'})

__dict__ = mappingproxy({'__module__': 'aiida.repository.backend.abstract', '__doc__': 'Class that defines the abstract interface for an object repository.\n\n The repository backend only deals with raw bytes, both when creating new objects as well as when returning a stream\n or the content of an existing object. The encoding and decoding of the byte content should be done by the client\n upstream. The file repository backend is also not expected to keep any kind of file hierarchy but must be assumed\n to be a simple flat data store. When files are created in the file object repository, the implementation will return\n a string-based key with which the content of the stored object can be addressed. This key is guaranteed to be unique\n and persistent. Persisting the key or mapping it onto a virtual file hierarchy is again up to the client upstream.\n ', 'uuid': <abc.abstractproperty object>, 'initialise': <function AbstractRepositoryBackend.initialise>, 'is_initialised': <abc.abstractproperty object>, 'erase': <function AbstractRepositoryBackend.erase>, 'is_readable_byte_stream': <staticmethod object>, 'put_object_from_filelike': <function AbstractRepositoryBackend.put_object_from_filelike>, '_put_object_from_filelike': <function AbstractRepositoryBackend._put_object_from_filelike>, 'put_object_from_file': <function AbstractRepositoryBackend.put_object_from_file>, 'has_object': <function AbstractRepositoryBackend.has_object>, 'open': <function AbstractRepositoryBackend.open>, 'get_object_content': <function AbstractRepositoryBackend.get_object_content>, 'get_object_hash': <function AbstractRepositoryBackend.get_object_hash>, 'delete_object': <function AbstractRepositoryBackend.delete_object>, '__dict__': <attribute '__dict__' of 'AbstractRepositoryBackend' objects>, '__weakref__': <attribute '__weakref__' of 'AbstractRepositoryBackend' objects>, '__abstractmethods__': frozenset({'_put_object_from_filelike', 'initialise', 'erase', 'is_initialised', 'has_object', 'uuid'}), '_abc_impl': <_abc_data object>})

__module__ = 'aiida.repository.backend.abstract'

__weakref__

list of weak references to the object (if defined)

_abc_impl = <_abc_data object>

abstract _put_object_from_filelike(handle: BinaryIO) → str

delete_object(key: str)

Delete the object from the repository.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be deleted.

abstract erase() → None

Delete the repository itself and all its contents.

Note

This should not merely delete the contents of the repository but any resources it created. For example, if the repository is essentially a folder on disk, the folder itself should also be deleted, not just its contents.

get_object_content(key: str) → bytes

Return the content of a object identified by key.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

get_object_hash(key: str) → str

Return the SHA-256 hash of an object stored under the given key.

Important

A SHA-256 hash should always be returned, to ensure consistency across different repository implementations.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

abstract has_object(key: str) → bool

Return whether the repository has an object with the given key.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

True if the object exists, False otherwise.

abstract initialise(**kwargs) → None

Initialise the repository if it hasn’t already been initialised.

Parameters

kwargs – parameters for the initialisation.

abstract property is_initialised

Return whether the repository has been initialised.

static is_readable_byte_stream(handle) → bool

open(key: str) → Iterator[BinaryIO]

Open a file handle to an object stored under the given key.

Note

this should only be used to open a handle to read an existing file. To write a new file use the method put_object_from_filelike instead.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

yield a byte stream object.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

put_object_from_file(filepath: Union[str, pathlib.Path]) → str

Store a new object with contents of the file located at filepath on this file system.

Parameters

filepath – absolute path of file whose contents to copy to the repository.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

put_object_from_filelike(handle: BinaryIO) → str

Store the byte contents of a file in the repository.

Parameters

handle – filelike object with the byte content to be stored.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

abstract property uuid

Return the unique identifier of the repository.

Implementation of the AbstractRepositoryBackend using the disk-objectstore as the backend.

class aiida.repository.backend.disk_object_store.DiskObjectStoreRepositoryBackend(container)

Bases: aiida.repository.backend.abstract.AbstractRepositoryBackend

Implementation of the AbstractRepositoryBackend using the disk-object-store as the backend.

__abstractmethods__ = frozenset({})

__init__(container)

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.repository.backend.disk_object_store'

__str__() → str

Return the string representation of this repository.

_abc_impl = <_abc_data object>

_put_object_from_filelike(handle: BinaryIO) → str

Store the byte contents of a file in the repository.

Parameters

handle – filelike object with the byte content to be stored.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

property container

delete_object(key: str)

Delete the object from the repository.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be deleted.

erase()

Delete the repository itself and all its contents.

get_object_hash(key: str) → str

Return the SHA-256 hash of an object stored under the given key.

Important

A SHA-256 hash should always be returned, to ensure consistency across different repository implementations.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

FileNotFoundError – if the file does not exist.

has_object(key: str) → bool

Return whether the repository has an object with the given key.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

True if the object exists, False otherwise.

initialise(**kwargs) → None

Initialise the repository if it hasn’t already been initialised.

Parameters

kwargs – parameters for the initialisation.

property is_initialised

Return whether the repository has been initialised.

open(key: str) → Iterator[BinaryIO]

Open a file handle to an object stored under the given key.

Note

this should only be used to open a handle to read an existing file. To write a new file use the method put_object_from_filelike instead.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

yield a byte stream object.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

property uuid

Return the unique identifier of the repository.

Implementation of the AbstractRepositoryBackend using a sandbox folder on disk as the backend.

class aiida.repository.backend.sandbox.SandboxRepositoryBackend

Bases: aiida.repository.backend.abstract.AbstractRepositoryBackend

Implementation of the AbstractRepositoryBackend using a sandbox folder on disk as the backend.

__abstractmethods__ = frozenset({})

__del__()

Delete the entire sandbox folder if it was instantiated and still exists.

__init__()

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.repository.backend.sandbox'

__str__() → str

Return the string representation of this repository.

_abc_impl = <_abc_data object>

_put_object_from_filelike(handle: BinaryIO) → str

Store the byte contents of a file in the repository.

Parameters

handle – filelike object with the byte content to be stored.

Returns

the generated fully qualified identifier for the object within the repository.

Raises

TypeError – if the handle is not a byte stream.

delete_object(key: str)

Delete the object from the repository.

Parameters

key – fully qualified identifier for the object within the repository.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be deleted.

erase()

Delete the repository itself and all its contents.

has_object(key: str) → bool

Return whether the repository has an object with the given key.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

True if the object exists, False otherwise.

initialise(**kwargs) → None

Initialise the repository if it hasn’t already been initialised.

Parameters

kwargs – parameters for the initialisation.

property is_initialised

Return whether the repository has been initialised.

open(key: str) → Iterator[BinaryIO]

Open a file handle to an object stored under the given key.

Note

this should only be used to open a handle to read an existing file. To write a new file use the method put_object_from_filelike instead.

Parameters

key – fully qualified identifier for the object within the repository.

Returns

yield a byte stream object.

Raises

• FileNotFoundError – if the file does not exist.

• OSError – if the file could not be opened.

property sandbox

Return the sandbox instance of this repository.

property uuid

Return the unique identifier of the repository.

Note

A sandbox folder does not have the concept of a unique identifier and so always returns None.

aiida.repository package aiida.restapi package