9 Using Oracle In-Memory Database Cache in a RAC Environment

This chapter describes how to use Oracle In-Memory Database Cache in an Oracle Real Application Clusters (RAC) environment. It includes the following topics:

How Oracle In-Memory Database Cache works in a RAC environment
Restrictions on using Oracle In-Memory Database Cache in a RAC environment
Setting up Oracle In-Memory Database Cache in a RAC environment

How Oracle In-Memory Database Cache works in a RAC environment

Oracle RAC enables multiple Oracle instances to access one Oracle database with shared resources, including all data files, control files, PFILEs and redo log files that reside on cluster-aware shared disks. RAC handles read/write consistency and load balancing while providing high availability.

Fast Application Notification (FAN) is a RAC feature that was integrated with Oracle Call Interface (OCI) in Oracle Database 10g Release 2. FAN publishes information about changes in the cluster to applications that subscribe to FAN events. FAN prevents unnecessary operations such as the following:

Attempts to connect when services are down
Attempts to finish processing a transaction when the server is down
Waiting for TCP/IP timeouts

See Oracle Real Application Clusters Administration and Deployment Guide for more information about RAC and FAN.

Oracle In-Memory Database Cache uses OCI integrated with FAN to receive notification of Oracle events. Without FAN, it can take several minutes for Oracle In-Memory Database Cache to receive notification of an Oracle failure. Oracle In-Memory Database Cache can recover quickly from Oracle failures without user intervention.

Oracle In-Memory Database Cache also uses Transparent Application Failover (TAF), which is a feature of Oracle Net Services that enables you to specify how you want applications to reconnect after a failure. See Oracle Database Net Services Administrator's Guide for more information about TAF.

OCI applications can use one of the following types of Oracle Net failover functionality:

None: No failover functionality is used. This can also be explicitly specified to prevent failover from happening. This is the default failover functionality.
Session: If an application's connection is lost, a new connection is automatically created for the application. This type of failover does not attempt to recover selects.
Select: This type of failover enables applications that began fetching rows from a cursor before failover to continue fetching rows after failover.

The behavior of Oracle In-Memory Database Cache depends on the actions of TAF and how TAF is configured. By default, TAF and FAN callbacks are installed if you are using Oracle In-Memory Database Cache in an Oracle RAC environment. If you do not want TAF and FAN capabilities, set the RACCallback DSN attribute to 0.

Table 9-1 shows the behaviors of Oracle In-Memory Database Cache operations in a RAC environment with different TAF failover types.

Table 9-1 Behavior of Oracle In-Memory Database Cache operations in a RAC environment

Operation	TAF Failover Type	Behavior After a Failed Connection on the Oracle Database
Automatic refresh	None	The cache agent automatically stops, restarts and waits until a connection can be established on the Oracle database. This behavior is the same as in a non-RAC environment. No user intervention is needed.
Automatic refresh	Session	One of the following occurs: All failed connections are recovered. Automatic refresh operations that were in progress will be rolled back and retried. If TAF times out or cannot recover the connection, the cache agent automatically stops, restarts and waits until a connection can be established on the Oracle database. In all cases, no user intervention is needed.
Automatic refresh	Select	One of the following occurs: Automatic refresh operations resume from the point of connection failure. Automatic refresh operations that were in progress will be rolled back and retried. If TAF times out or cannot recover the connection, the cache agent automatically stops, restarts and waits until a connection can be established on the Oracle database. In all cases, no user intervention is needed.
AWT	None	The receiver thread of the replication agent for the AWT cache group exits. A new thread is spawned and tries to connect to the Oracle database. No user intervention is needed.
AWT	Session, Select	One of the following occurs: If the connection is recovered and there are uncommitted DML operations in the transaction, the transaction will be rolled back and then reissued. If the connection is recovered and there are no uncommitted DML operations, new operations can be issued without rolling back. In all cases, no user intervention is needed.
SWT, propagate, flush, and passthrough	None	The application is notified of the connection loss. The cache agent disconnects from the Oracle database and the current transaction is rolled back. All modified session attributes are lost. During the next passthrough operation, the cache agent tries to reconnect to the Oracle database. This behavior is the same as in a non-RAC environment. No user intervention is needed.
SWT, propagate, flush and passthrough SWT, propagate and flush	Session Select	One of the following occurs: The connection to the Oracle database is recovered. If there were open cursors, DML or lock operations on the lost connection, an error is returned and the user must roll back the transaction before continuing. Otherwise, the user can continue without rolling back. If TAF times out or cannot recover the connection, the application is notified of the connection loss. The cache agent disconnects from the Oracle database and the current transaction is rolled back. All modified session attributes are lost. During the next passthrough operation, the cache agent tries to reconnect to the Oracle database. In this case, no user intervention is needed.
Passthrough	Select	The connection to the Oracle database is recovered. If there were DML or lock operations on the lost connection, an error is returned and the user must roll back the transaction before continuing. Otherwise, the user can continue without rolling back.
Load and refresh	None	The application receives a loss of connection error.
Load and refresh	Session	One of the following occurs: The load or refresh operation succeeds. An error is returned stating that a fetch operation on Oracle cannot be executed.
Load and refresh	Select	One of the following occurs: If the Oracle cursor is open and the cursor is recovered, or if the Oracle cursor is not open, then the load or refresh operation succeeds. An error is returned if TAF was unable to recover either the session or open Oracle cursors. Note: An error is less likely to be returned than if the TAF failover type is Session.

Restrictions on using Oracle In-Memory Database Cache in a RAC environment

Oracle In-Memory Database Cache's support of RAC has the following restrictions:

Oracle In-Memory Database Cache behavior is limited to RAC, FAN and TAF capabilities. For example, if all nodes for a service fail, the service will not be restarted. Oracle In-Memory Database Cache waits for the user to restart the service.
TAF does not recover ALTER SESSION operations. The user is responsible for restoring changed session attributes after a failover.
Oracle In-Memory Database Cache uses OCI integrated with FAN. This interface automatically spawns a thread to wait for an Oracle event. This is the only Oracle In-Memory Database Cache feature that spawns a thread in a TimesTen direct link application. Adapt your application to account for this thread creation. If you do not want the extra thread, set the RACCallback DSN attribute to 0 so that TAF and FAN will not be used.

Setting up Oracle In-Memory Database Cache in a RAC environment

Install Oracle RAC and Oracle In-Memory Database Cache. Ensure that the cache administration user has the SELECT ANY DICTIONARY privilege.

There are two TimesTen environment variables that control TAF timeouts:

TT_ORA_FAILOVER_TIMEOUT: TAF timeout, in minutes, for the application. Applies to SWT cache groups, user managed cache groups that use the PROPAGATE cache table attribute, and the use of the passthrough feature. The default is 5 minutes.
TT_AGENT_ORA_FAILOVER_TIMEOUT: TAF timeout, in minutes, for the replication agent. Applies to AWT cache groups. The default is 5 minutes.

Make sure that the TimesTen daemon and cache agent are started. The following Oracle components should also be started:

Oracle instances
Oracle listeners
Oracle service that will be used for Oracle In-Memory Database Cache

Then perform the following tasks:

Verify that the RACCallback DSN attribute is set to 1 (default).
Use the DBMS_SERVICE.MODIFY_SERVICE function or Oracle Enterprise Manager to enable publishing of FAN events. This changes the value in the AQ_HA_NOTIFICATIONS column of the Oracle ALL_SERVICES view to YES.

See Oracle Database PL/SQL Packages and Types Reference for more information about the DBMS_SERVICE Oracle PL/SQL package.

Enable TAF on the Oracle service used for Oracle In-Memory Database Cache with one of the following methods:
Create a service for TimesTen in the Oracle tnsnames.ora file with the following settings:
LOAD_BALANCE=ON (optional)
FAILOVER_MODE=(TYPE=SELECT) or FAILOVER_MODE=(TYPE=SESSION)
Use the DBMS_SERVICE.MODIFY_SERVICE function to set the TAF failover type.

See Oracle Database Net Services Administrator's Guide for more information about enabling TAF.

If you have a TimesTen direct link application, link it with a thread library so that it will receive FAN notifications. FAN spawns a thread to monitor for failures.