6 DBFS File System Client

DBFS Prerequisites

• dbfs_client is only available on Linux and Linux.X64 platforms.

• The dbfs_client host must have the Oracle client libraries installed.

• The dbfs_client host must have the kernel-devel package installed to configure and build FUSE.

• The dbfs_client host must have the FUSE Linux package installed before mounting a DBFS file system.

Installing FUSE

• Download the kernel-devel package from your Linux distributor that matches your Linux release.

• Download FUSE 2.7.3 package from http://fuse.sourceforge.net/.

• Install kernel-devel package. For example:

  # rpm -i kernel-devel-2.6.18-8.el5.i686.rpm
  

• Determine the kernel directory. The kernel directory is usually /usr/src/kernels/`uname -r`-`uname -p`

• Install FUSE.

  $ tar -xzvf fuse-2.7.3.tar.gz    
  $ cd [fuse_src_dir]
  $ ./configure --prefix=/usr --with-kernel=[your kernel dir]
  $ make
  $ sudo su
  # make install
  # /sbin/depmod
  # /sbin/modprobe fuse
  # chmod 666 /dev/fuse
  # echo "/sbin/modprobe fuse" >> /etc/rc.modules
  

DBFS Installation Home

The DBFS installation home contains the SQL (.plb extension) scripts for the content store, and the dbfs_client executable.

Creating a File System

Grant dbfs_role role to the user. Only the users who are granted dbfs_role role can create file systems. Create a file system by running dbfs_create_filesystem.sql while logged in as the user that will own the file system.

$ sqlplus @$ORACLE_HOME/rdbms/admin/dbfs_create_filesystem.sql <tablespace name>
     <file system name>

The following example creates a file system called staging_area in the tablespace dbfs_tbspc. The tablespace has been previously created.

$ sqlplus @$ORACLE_HOME/rdbms/admin/dbfs_create_filesystem.sql dbfs_tbspc
     staging_area

dbfs_create_filesystem.sql creates a partitioned file system. Partitioning is the best performing and scalable way to create a file system in DBFS. Partitioning creates multiple physical segments in the database and files are distributed randomly in these partitions.

Space cannot be shared between partitions. So it is possible for one partition to run out of space even when other partitions have space. This is usually not an issue if file system size is big compared to the size of the individual files. However if file sizes are a big percentage of the file system size, it can cause ENOSPC error to happen even if the file system is not full.

Another implication of partitioning is that a "rename" operation can require the file to be rewritten. This can be expensive if file is a big file.

dbfs_create_filesystem_advanced.sql can be used to create a non-partitioned file system.

Dropping a File System

Drop the file system by running dbfs_drop_filesystem.sql.

$ sqlplus @$ORACLE_HOME/rdbms/admin/dbfs_drop_filesystem.sql  <file system name>

Mounting the DBFS Store

Run the dbms_client program to mount the DBFS store. Ensure that LD_LIBRARY_PATH has the correct path to the Oracle client libraries before calling this program. The dbfs_client will not return until the file system is unmounted.

Usage: dbfs_client <db_user>@<db_server> [options] <mount point>
 
    db_user: Name of Database user that owns DBFS content store filesystem(s)
    db_server:   A valid connect string Oracle database server
                 (for example, hrdb_host:1521/hrservice).
    mount point: Path to mount Database File System(s).
                 All the file systems owned by the database user will be seen at
                 the mount point.
 
DBFS options:
  -o direct_io   Bypasses the Linux page cache.  Gives much better performance for
                 large files. 
                 Programs in the file system cannot be executed with this option.
                 This option is recommended when DBFS is used as an ETL staging
                 area.
  -o wallet      Run dbfs_client in background. Wallet must be configured to get
                 credentials. 
  -o failover    DBFS Client fails over to surviving database instance with no
                 data loss. 
                 Some performance cost on writes, especially for small files.
  -o allow_root  Allows root access to the filesystem. 
                 This option requires setting 'user_allow_other' parameter in
                 /etc/fuse.conf
  -o allow_other    Allows other users access to the filesystem.
                    This option requires setting 'user_allow_other' parameter in
                    /etc/fuse.conf
  -o rw             Mount the filesystem read-write [Default]
  -o ro             Mount the filesystem read-only. Files cannot be modified.
  -o trace_level=N  Trace Level: 1->DEBUG, 2->INFO, 3->WARNING, 4->ERROR,
                    5->CRITICAL [Default: 4]
  -o trace_file     <file> | 'syslog'
  -h help
  -V version

For example, to mount a file system using dbfs_client by entering the password on the command prompt:

$ dbfs_client ETLUser@DBConnectString /mnt/dbfs  
  password: xxxxxxx

The following example mounts a file system and frees the terminal. It reads the password from a file:

$ nohup dbfs_client ETLUser@DBConnectString /mnt/dbfs  < passwordfile.f &

$ ls -l /mnt/dbfs
drwxrwxrwx  10 root root 0 Feb  9 17:28 staging_area

For a more secure method of specifying the password, see"Using Oracle Wallet with DBFS Client".

Unmounting a File System

Run fusermount to unmount file systems.

$ fusermount -u <mount point>

Mounting DBFS Through fstab

File systems are commonly configured using the fstab mechanism in Linux. To mount DBFS through /etc/fstab, Oracle Wallet needs to be used for authentication. Run the following operations as root:

1. $ ln -s /sbin/mount.dbfs $ORACLE_HOME/bin/dbfs_client
   2. Create a new Linux group called fuse.
   3. Add the Linux user that will be running the DBFS Client to the fuse group.
   4. Add the following line to /etc/fstab
       /sbin/mount.dbfs#db_user@db_server mount_point fuse rw, user, noauto 0 0

       For example: /sbin/mount.dbfs#/@DBConnectString /mnt/dbfs fuse rw, user,
        noauto 0 0

The Linux user can mount the DBFS file system using the standard Linux mount command. For example:

$ mount /mnt/dbfs

Note that FUSE does not currently support automount.

Restrictions on Mounted File Systems

DBFS supports most file system operations with the exception of ioctl, locking, memory-mapped files, Async IOs, O_DIRECT file opens, and hard links. For performance reasons, DBFS does not update the file access time every time a file is read.

Using DBFS

All DBFS content store paths must be preceded by "dbfs:". For example: dbfs:/staging_area/file1. All database path names specified must be absolute paths. To run DBFS commands, specify --command to the DBFS client.

dbfs_client <db_user>@<db_server> --command <command> [switches] [arguments]
 
      command:         Command to be executed. For example,ls,cp,mkdir,rm
      switches:        Switches are described below for each command
      arguments:       File names or directory names

dbfs_client returns a nonzero value in case of a failure.

Listing a Directory

ls lists the contents of a directory.

dbfs_client <db_user>@<db_server> --command ls [switches] target
 
Switches:
       -a         Show all files including '.' and '..'
       -l         Use a long listing format. In addition to the name of each file,
                  print the file type, permissions, and size.
       -R         List subdirectories recursively
 
Examples:
$ dbfs_client ETLUser@DBConnectString --command  ls dbfs:/staging_area/directory1

$ dbfs_client ETLUser@DBConnectString --command  ls -l -a -R
   dbfs:/staging_area/directory1 

Copying Files and Directories

cp copies files or directories from source location to destination location. It also supports recursive copy of directories.

dbfs_client <db_user>@<db_server> --command cp [switches] source destination
 
Switches:
       -R,-r  Copy a directory and its contents recursively into the destination
              directory

The following example copies the contents of the local directory, 1-Jan-2009-dump recursively into a directory in DBFS:

$ dbfs_client ETLUser@DBConnectString --command cp -R  1-Jan-2009-dump
    dbfs:/staging_area/

The following example copies the file hello.txt from DBFS to a local file Hi.txt:

$ dbfs_client ETLUser@DBConnectString --command cp dbfs:/staging_area/hello.txt 
    /tmp/Hi.txt

Removing Files and Directories

rm removes a file or directory. It also supports recursive delete of directories.

dbfs_client <db_user>@<db_server> --command rm [switches] target
 
Switches:
      -R, -r      Removes a directory and its contents recursively.
 
Examples:
$ dbfs_client ETLUser@DBConnectString --command rm 
    dbfs:/staging_area/srcdir/hello.txt

$ dbfs_client ETLUser@DBConnectString --command rm -R 
    dbfs:/staging_area/directory1

Creating a Directory with mkdir

mkdir creates a new directory.

dbfs_client <db_user>@<db_server> --command mkdir directory_name
 
Example:

$ dbfs_client ETLUser@DBConnectString --command mkdir
    dbfs:/staging_area/directory2

Using Oracle Wallet with DBFS Client

An Oracle Wallet allows the DBFS client to mount a DBFS store without the user having to enter a password. Please refer to Oracle Database Advanced Security Administrator's Guide for more information about creation and management of wallets.

Create Oracle Wallet:

1. Create a directory for the wallet. For example:

   mkdir $HOME/oracle/wallet
   

2. Create an auto-login wallet.

   mkstore -wrl $HOME/oracle/wallet -create
   

3. Add the wallet location in the client's sqlnet.ora file:

   vi $TNS_ADMIN/sqlnet.ora
   WALLET_LOCATION = (SOURCE = (METHOD = FILE) (METHOD_DATA = (DIRECTORY =
    $HOME/oracle/wallet) ) ) 
   

4. Add the following parameter in the client's sqlnet.ora file:

   vi $TNS_ADMIN/sqlnet.ora
   SQLNET.WALLET_OVERRIDE = TRUE
   

Create credentials:

1. mkstore -wrl <wallet_location> -createCredential <db_connect_string>
    <username> <password>
   

   For example:

   mkstore -wrl $HOME/oracle/wallet -createCredential DBConnectString scott tiger
   

Add the connection alias to your tnsnames.ora file.

Now you can use dbfs_client with Oracle Wallet. For example:

$ dbfs_client -o wallet /@DBConnectString /mnt/dbfs 

The "/@" syntax means to use the wallet.

File System Security Model

Security in the DBFS is primarily managed by the database, not by the operating system security model. Access to a database file system requires a database login as a database user that has privileges on the tables underlying the file system. Access to the file system can be granted to other users by the database administrator. This implies that different database users may have different read or update privileges to the file system as determined by database administrator. The database administrator has access to all the files stored in the DBFS.

On the client machine, access to a DBFS mount point is limited to the operating system user that mounts the file system. However this does not limit the number of users who can access DBFS. Many users can separately mount the same DBFS file system.

Operating system file level permission checking is done by Linux when the DBFS is mounted as a file system, but it is not done by the DBFS when using the command interface or when using the PL/SQL interface directly. In this case only database privilege checking is performed.

The operating system user mounting the file system can allow root access to the file system by specifying the allow_root option. This option requires a user_allow_other field to be present in /etc/fuse.conf. For example:

# Allow users to specify the 'allow_root' mount option.
user_allow_other

The operating system user mounting the file system can allow users access to the file system by specifying the allow_other option. This option requires a user_allow_other field to be present in /etc/fuse.conf.

Oracle does not recommend DBFS to be run as a root user.

DBFS Diagnostics

dbfs_client supports multiple levels of tracing to help diagnose any problems. dbfs_client can either output traces to a file or to /var/log/messages using the syslog daemon on Linux. When tracing to a file, it keeps two trace files on disk. dbfs_client rotates the trace files automatically and limits disk usage to 20 MB.

By default, tracing is turned off except for critical messages which are always logged to /var/log/messages.

If dbfs_client is not able to connect to the Oracle Database, enable tracing using trace_level and trace_file options. Tracing will print additional messages to log file for easier debugging.

DBFS uses Oracle RDBMS Database for storing files. Sometimes Oracle server issues will get propagated to dbfs_client as errors. If there is a dbfs_client error, please see the Oracle server logs to see if that is the root cause.

DBFS Client Failover

In case of a failure of one database instance, dbfs_client can failover to one of the other existing database instances. For dbfs_client failover to work correctly, modify the Oracle database service and specify failover parameters. For example:

exec DBMS_SERVICE.MODIFY_SERVICE(service_name => 'service_name',
                                    aq_ha_notifications => true,
                                    failover_method => 'BASIC',
                                    failover_type => 'SELECT',
                                    failover_retries => 180,
                                    failover_delay => 1);

To insure no data loss during failover of the DBFS connection after a failure of the back-end Oracle database instance, specify the "-o failover" mount option. In this case, cached writes may be lost if the client dies, but back-end failover to other RAC instances or standby databases will not cause writes to be lost.

$ dbfs_client <db_user>@<db_server> -o failover /mnt/dbfs 

Sharing and Caching

It is possible to have multiple copies of dbfs_client accessing the same shared file system. The sharing and caching semantics are similar to NFS. Like NFS, the default mode caches writes on the client and flushes them after a timeout or when the file being modified is closed. Also like NFS, writes to a file are only visible to clients that open the file after the writer closed the file. This behavior is commonly referred to as close-to-open cache consistency.

To bypass client side write caching, specify O_SYNC when the file is opened. Writes in the cache can be forced to disk by calling fsync.

Backup of DBFS

DBFS can be backed up two ways. One way is to back up the tables underlying the file system at the database level. An alternate way is to use a normal file system backup tool such as Oracle Secure Backup to backup the files through a mount point.

An advantage of backing up the tables at the database level is that the files in the file system will always be consistent with the relational data in the database. A full restore and recover of the database will also fully restore and recover the file system with no data loss. If a point in time recovery of the database is ever done, then the files will be recovered to the exact same time. As usual with database backup, modifications that happen during the backup will not affect the consistency of a restore. The entire restored file system will always be consistent as of a point in time.

The advantage of backing up the file system using a normal file system backup tool is that individual files can more easily be restored from backup. In this method any changes made to the restored files after the last backup will be lost.

The allow_root mount option should be specified if backups are scheduled using the Oracle Secure Backup Administrative Server.

Small File Performance of DBFS

Like any shared file system, the performance of DBFS for small files will lag the performance of a local file system. Each file data or metadata operation in DBFS must go through the FUSE user mode file system, and then be forwarded across the network to the database. Therefore, each operation that is not cached on the client takes a few milliseconds to run in DBFS.

For operations that involve an input/output (IO) to disk, the overhead is masked by the wait for the disk IO. Larger IOs have a lower percentage overhead than smaller IOs. The network overhead is more noticeable for operations that don't issue a disk IO.

Therefore, compared to a local file system, operations that operate on a few files will not see a noticeable overhead. Operations that operate on thousands of small files will see a larger overhead. For example, listing a single directory or looking at a single file will see near instantaneous response. Searching across a directory tree with many thousands of files you will see a larger relative overhead.

Note that the performance of DBFS on file systems with tens of thousands of files is not fully tuned in the current revision.

DBFS Advanced Features

DBFS leverages the advanced features available with SecureFiles -- compression, deduplication, and encryption. For example, DBFS can be configured as a compressed file system. At the time of creating a file system, you can specify the set of features (compression, deduplication, or encryption) that you choose to enable for the file system.

$ sqlplus @dbfs_create_filesystem_advanced <tablespace name> <file system name>
   <compress-high | compress-medium | compress-low | nocompress> 
   <deduplicate | nodeduplicate> <encrypt | noencrypt> <partition | non-partition>