7 DBFS Content API

DBFS Content API Role

Access to the Content operational and administrative API (packages, types, tables, and so on) is available through the dbfs_role. This role can be granted to all users as needed.

Path Name Constants and Types

PATH_MAX is the maximum length of an absolute path name visible to clients. NAME_MAX is the maximum length of any individual component of an absolute path name visible to clients. These constants are modeled after their SecureFile Store counterparts.

PL/SQL types path_t and name_t are portable aliases for strings that can represent path names and component names.

Content Properties

Every path name in a store is associated with a set of properties. For simplicity and generality, each property is identified by a string name, has a string value (possibly null if not set or undefined or unsupported by a specific store implementation) and a value typecode (a numeric discriminant for the actual type of value held in the value string.

Coercing property values to strings has the advantage of making the various interfaces uniform and compact (and can even simplify implementation of the underlying stores), but has the disadvantage of the potential for information loss during conversions to and from strings.

It is expected that clients and stores use well-defined database conventions for these conversions, and use the typecode field (explained below) as appropriate.

PROPNAME_MAX is the maximum length of a property name, and PROPVAL_MAX is the maximum length of the string value of a property.

PL/SQL types propname_t and propval_t are portable aliases for strings that can represent property names and values. A typecode is a numeric value (see the various constants defined in dbms_types) representing the true type of a string-coerced property value. Not all typecodes defined in dbms_types are necessarily even supportable in stores. Simple scalar types (numbers, dates, timestamps, and so on) can be depended upon by clients and must be implemented by stores.

Since standard RDBMS typecodes are positive integers, the DBFS Content API allows negative integers to represent client-defined types by negative typecodes. These typecodes do not conflict with standard typecodes, and will be made persistent and returned to the client as needed, but need not be interpreted by the DBFS Content API or any particular store. Portable client applications should not use user-defined typecodes as a back-door way of passing information to specific stores.

The dbms_dbfs_content_property_t object type describes a single (name, value, typecode) property tuple. All properties: standard, optional, and user-defined, are described via such tuples.

dbms_dbfs_content_properties_t is a variable-sized array of property tuples. These two types are used by both the client-facing APIs and by the store providers for the DBFS Content API.

Path Name Types

Stores can store and provide access to four types of entities:

type_file: A regular file storing content (a logically linear sequence of bytes accessing as a BLOB).

type_directory: A container of other path name types, including file types.

type_link: A symbolic link that is, an uninterpreted string value associated with a path name). Since symbolic links may represent path names that fall outside the scope of any given store (or even the entire aggregation of stores managed by the DBFS Content API), or may not even represent path names, clients must be careful in creating symbolic links, and stores must be careful in trying to resolve these links internally.

type_reference: A hard link (that is, an always valid path name alias) to content.

Not all stores need to implement all of directories, links, or references (see the next section).

Store Features

In order to provide a common programmatic interface to as many different types of stores as possible, the DBFS Content API leaves some of the behavior of various operations to individual store providers to define and implement. However, it is still important to provide client-side programmers with an API that is sufficiently rich and conducive to portable applications.

The DBFS Content API achieves this by allowing different store providers (and different stores) to describe themselves via a "feature set" (a bit mask indicating which features they support and which ones they do not).

Using the feature set, it is possible, although tricky, for client applications to compensate for the feature deficiencies of specific stores by implementing additional logic on the client side, and deferring complex operations to stores capable of supporting them.

feature_folders:

Set if the store supports folders (or directories) as part of hierarchical path names.

feature_foiat:

Set if implicit folder operations within the store (performed as part of a client-requested operation) runs inside autonomous transactions. In general, the use of autonomous transactions is a compromise between (a) simplicity in the implementation and client-controlled transaction scope for all operations, at the cost of greatly reduced concurrency (feature_foiat not set), and (b) more complex implementation and smaller client-controlled transaction scope, at the benefit of greatly increased concurrency (feature_foiat set).

Access to read-only (or read-mostly) stores should not be greatly affected by this feature.

feature_acls:

Set if the store supports access control lists (ACLs) and internal authorization and checking based on these ACLs. ACLs are standard properties, but a store may do nothing more than store and retrieve the ACLs without interpreting them in any way.

feature_links, feature_link_deref:

Set if the store supports symbolic links, and if certain types of symbolic links (specifically non-absolute path names) can be internally resolved by the store itself.

feature_references:

Set if the store supports hard links.

feature_locking:

Set if the store supports user-level locks (read-only, write-only, read-write) that can be applied on various items of the store, and if the store uses these lock settings to control various types of accesses to the locked items. User-level locks are orthogonal to transaction locks and persist beyond the scope of any specific transaction, session, or connection -- this implies that the store itself may not be able to clean up after dangling locks, and client-applications need to perform any "garbage collection."

feature_lock_hierarchy:

Set if the store allows a user-lock to control access to the entire sub-tree under the locked path name. A simpler locking model would have locking semantics apply only to a specific path name, and depend on the locks placed on its parents or children (unless the requested operation would implicitly need to modify these parents or children).

feature_lock_convert:

Set if the store supports upgrade and downgrade of locks from one mode to another.

feature_versioning:

Set if the store supports at least a linear versioning and version management. Different versions of the same path name are identified by monotonic version numbers, with a version-nonqualified path name representing the latest version.

feature_version_path:

Set if the store supports a hierarchical namespace for different versions of a path name.

feature_soft_deletes:

Set if the store supports a "soft-delete," that is, the ability to delete a path name and make it invisible to normal operations, but retain the ability to restore the path name later (as long as it has not been overwritten by a new create operation). The store also supports purging soft-deleted path names (making them truly deleted), and navigation modes that show soft-deleted items.

feature_hashing:

Set if the store automatically computes and maintains some type of a secure hash of the contents of a path name (typically a type_file path).

feature_hash_lookup:

Set if the store allows "content-based addressing," that is, the ability to locate a content item based, not on its path name, but on its content hash.

feature_filtering:

Set if the store allows clients to pass a filter function (a PL/SQL function conforming to the signature below) that returns a logical boolean indicating if a given store item satisfies a selection predicate. Stores that support filtering may be able to more efficiently perform item listing, directory navigation, and deletions by embedding the filtering logic inside their implementation. If filtering is not supported, clients can retrieve more items than necessary and perform the filtering checks themselves, although less efficiently.

A filter predicate is a function with the following signature:

function filterFunction(
    path        in      varchar2,
    store_name  in      varchar2,
    opcode      in      integer,
    item_type   in      integer,
    properties  in      dbms_dbfs_content_properties_t,
    content     in      blob)
    return  integer;

Any PL/SQL function conforming to this signature can examine the contents and properties of a store item, and decide if the item satisfies the selection criterion for the current operation. A return value of true (non-zero) will result in the DBFS Content API processing the item as part of the current operation; a return value of false (zero or null) will result in the item being skipped entirely from processing.

feature_searching:

Set if the store allows clients to pass a text-search filter query to locate type_file path names based on their content. Stores that support searching may use indexes to accelerate such searches; otherwise, clients need to build their own indexes, or else search a potentially larger set of items to locate the ones of interest for the current search.

feature_asof:

Set if the store allows clients to use a "flashback" timestamp in query operations (non-mutating getPath, list, search).

feature_provider_props:

Set if the store allows per-operation properties (that control the behavior of the store with respect to the current operation, as opposed to properties associated with individual items).

Lock Types

Stores that support locking should implement three types of locks: lock_read_only, lock_write_only, lock_read_write.

User locks (one of the three types) can be associated with a user-supplied lock_data -- this is not interpreted by the store, but can be used by client applications for their own purposes (for example, the user data could indicate the time at which the lock was placed, assuming some part of the client application is interested in later using this information to control its actions, for example, garbage collecting stale locks or explicitly break locks).

In the simplest locking model, a lock_read_only prevents all explicit modifications to a path name (but allows implicit modifications, and changes to parent/child path names). A lock_write_only prevents all explicit reads to the path name (but allows implicit reads, and reads to parent/child path names). A lock_read_write allows both.

All locks are associated with a "principal" performing the locking operation; stores that support locking are expected to preserve this information, and use it to perform read/write lock checking (see opt_locker).

More complex lock models: multiple read-locks, lock-scoping across path name hierarchies, lock conversions, group-locking, and so on, are possible but currently not defined by the DBFS Content API.

Standard Properties

Standard properties are well-defined, mandatory properties associated with all content path names that all stores must support (in the manner described by the DBFS Content API), with some concessions (for example, a read-only store need not implement a modification_time or creation_time; stores created against tables with a fixed schema may choose reasonable defaults for as many of these properties as needed, and so on).

All standard properties informally use the std: namespace. Clients and stores should avoid using this namespace to define their own properties since this can cause conflicts in the future.

The menu of standard properties is expected to be fairly stable over time.

std_access_time (TYPECODE_TIMESTAMP in UTC):

The time of last access of the contents of a path name.

std_acl (TYPECODE_VARCHAR2):

The access control list (in standard ACL syntax) associated with the path name.

std_canonical_path (TYPECODE_VARCHAR2):

The canonical store-specific path name of an item, suitably cleaned up (leading/trailing "/" collapsed, trimmed, and so on).

std_change_time (TYPECODE_TIMESTAMP in UTC):

The time of last change to the metadata of a path name.

std_children (TYPECODE_NUMBER):

The number of child directories/folders a directory/folder path has (this property should be available in providers that support feature_folders).

std_content_type (TYPECODE_VARCHAR2):

The client-supplied mime-type(s) (in standard RFC syntax) describing the (typically type_file) path name. The content type is not necessarily interpreted by the store.

std_creation_time (TYPECODE_TIMESTAMP in UTC):

The time at which the item was created (once set, this value never changes for the lifetime of the path name).

std_deleted (TYPECODE_NUMBER as a boolean):

Set to a non-zero number if the path name has been "soft-deleted" (see above for this feature), but not yet purged.

std_guid (TYPECODE_NUMBER):

A store-specific unique identifier for a path name. Clients must not depend on the GUID being unique across different stores, but a given (<store-name>, <store-specific-pathname>) has a stable and unique GUID for its lifetime.

std_length (TYPECODE_NUMBER):

The length of the content (BLOB) of a type_file or type_reference path, or the length of the referent of a type_link symbolic link. Directories do not have a well-defined length and stores are free to set this property to zero, null, or any other value they choose.

std_modification_time (TYPECODE_TIMESTAMP in UTC):

The time of last change to the data associated with a path name. Change to the content of a type_file or type_reference path, the referent of the type_link path, and addtion/deletion of immediate children in a type_directory path, all constitute data changes.

std_owner (TYPECODE_VARCHAR2):

A client-supplied (or implicit) owner name for the path name. The owner name may be used (along with the current "principal") for access checks by stores that support ACLs and/or locking.

std_parent_guid (TYPECODE_NUMBER):

A store-specific unique identifier for the parent of a path name. Clients must not depend on the GUID being unique across different stores, but a given (<store-name>, <store-specific-pathname>) has a stable and unique GUID for its lifetime.

std_parent_guid (pathname == std_guid(parent(pathname)))

std_referent (TYPECODE_VARCHAR2):

The content of the symbolic link of a type_link path; null otherwise. As mentioned before, the std_referent can be an arbitrary string and must not necessarily be interpreted as path name by clients (or such interpretation should be done with great care).

Optional Properties

Optional properties are well-defined but non-mandatory properties associated with all content path names that all stores are free to support (but only in the manner described by the DBFS Content API). Clients should be prepared to deal with stores that support none of the optional properties.

All optional properties informally use the "opt:" namespace. Clients and stores should avoid using this namespace to define their own properties since this can cause conflicts in the future.

The menu of optional properties is expected to be expand over time.

opt_hash_type (TYPECODE_NUMBER):

The type of hash provided in the opt_hash_value property; see dbms_crypto for possible options.

opt_hash_value (TYPECODE_RAW):

The hash value of type opt_hash_type describing the content of the path name.

opt_lock_count (TYPECODE_NUMBER):

The number of (compatible) locks placed on a path name. If different principals are allowed to place compatible (read) locks on a path, the opt_locker must specify all lockers (with repeats so that lock counts can be correctly maintained).

opt_lock_data (TYPECODE_VARCHAR2):

The client-supplied user-data associated with a user lock, uninterpreted by the store.

opt_locker (TYPECODE_VARCHAR2):

The implicit or client-specified principal(s) that applied a user lock on a path name.

opt_lock_status (TYPECODE_NUMBER):

One of the values (lock_read_only, lock_write_only, lock_read_write) describing the type of lock currently applied on a path name.

opt_version (TYPECODE_NUMBER):

A sequence number for linear versioning of a path name.

opt_version_path (TYPECODE_VARCHAR2):

A version path name for hierarchical versioning of a path name.

Property Access Flags

DBFS Content API methods to get and set properties can use combinations of property access flags to fetch properties from different namespaces in a single API call.

prop_none:

Used when the client is not interested in any properties, and is invoking the content access method for other reasons (path name existence/lockability validation, data access, and so on.)

prop_std:

Used when the client is interested in the standard properties; all standard properties are retrieved if this flag is specified.

prop_opt:

Used when the client is interested in the optional properties; all optional properties are retrieved if this flag is specified.

prop_usr:

Used when the client is interested in the user-defined properties; all user-defined properties are retrieved if this flag is specified.

prop_all:

An alias for the combination of all standard, optional, and user-defined properties.

prop_data:

Used when the client is interested only in data access, and does not care about properties.

prop_spc:

Used when the client is interested in a mix-and-match of different subsets of various property namespaces; the names of the specific properties to fetch are passed into the DBFS Content API method call as arguments, and only these property values are fetched and returned to the client. This is useful in cases where there are a very large number of properties potentially accessible, but the client is interested in only a small number of them (and knows the names of those properties beforehand).

prop_spc is applicable only to the various "getPath" operations. Other operations that specify properties will simply ignore prop_spc specifications.

Exceptions

DBFS Content API operations can raise any one of the following top-level exceptions.

Clients can program against these specific exceptions in their error handlers without worrying about the specific store implementations of the underlying error signalling code.

Store service providers, for their part, should do their best to trap and wrap any internal exceptions into one of the following exception types, as appropriate.

Property Bundles

The property_t record type describes a single (value, typecode) property value tuple; the property name is implied (see properties_t in the next code snippet).

properties_t is a name-indexed hash table of property tuples. The implicit hash-table association between the index and the value allows the client to build up the full dbms_dbfs_content_property_t tuples for a properties_t.

There is an approximate correspondence between dbms_dbfs_content_property_t and property_t -- the former is a SQL object type that describes the full property tuple, while the latter is a PL/SQL record type that describes only the property value component.

There is an approximate correspondence between dbms_dbfs_content_properties_t and properties_t -- the former is a SQL nested table type, while the latter is a PL/SQL hash table type.

Dynamic SQL calling conventions force the use of SQL types, but PL/SQL code may be implemented more conveniently in terms of the hash-table types.

DBFS Content API provides convenient utility functions to convert between dbms_dbfs_content_properties_t and properties_t (see next code snippet).

Store Descriptors

A store_t is a record that describes a store registered with, and managed by the DBFS Content API (see the Administrative APIs in the next section).

A mount_t is a record that describes a store mount point and its properties.

Clients can query the DBFS Content API for the list of available stores, determine which store will handle accesses to a given path name, and determine the feature set for the store.