5 Replaying a Database Workload

5.1.1 Restoring the Database

Before a workload can be replayed, the application data state should be logically equivalent to that of the capture system at the start time of workload capture. This minimizes data divergence during replay. The method for restoring the database depends on the backup method that was used before capturing the workload. For example, if RMAN was used to back up the capture system, you can use RMAN DUPLICATE capabilities to create the test database. For more information, see "Prerequisites for Capturing a Database Workload".

After the database is created with the appropriate application data on the test system, perform the system change you want to test, such as a database or operating system upgrade. The primary purpose of Database Replay is to test the effect of system changes on a captured workload. Therefore, the system changes you make should define the test you are conducting with the captured workload.

5.1.2 Resetting the System Time

It is recommended that the system time on the replay system host be changed to a value that approximately matches the capture start time just before replay is started. Otherwise, an invalid data set may result when replaying time-sensitive workloads. For example, a captured workload that contains SQL statements using the SYSDATE and SYSTIMESTAMP functions may cause data divergence when replayed on a system that has a different system time. Resetting the system time will also minimize job scheduling inconsistencies between capture and replay.

5.2.1 Setting Up the Replay Directory

The captured workload must have been preprocessed and copied to the replay system. A directory object for the directory to which the preprocessed workload is copied must exist in the replay system.

5.2.2 Resolving References to External Systems

A captured workload may contain references to external systems, such as database links or external tables. Typically, you should reconfigure these external interactions to avoid impacting other production systems during replay. External references that need to be resolved before replay a workload include:

• Database links

  It is typically not desirable for the replay system to interact with other databases. Therefore, you should reconfigure all database links to point to an appropriate database that contains the data needed for replay.

• External tables

  All external files specified using directory objects referenced by external tables need to be available to the database during replay. The content of these files should be the same as during capture, and the filenames and directory objects used to define the external tables should also be valid.

• Directory objects

  You should reconfigure any references to directories on the production system by appropriately redefining the directory objects present in the replay system after restoring the database.

• URLs

  URLs/URIs that are stored in the database need to be configured so that Web services accessed during the workload capture will point to the proper URLs during replay. If the workload refers to URLs that are stored in the production system, you should isolate the test system network during replay.

• E-mails

  To avoid resending E-mail notifications during replay, any E-mail server accessible to the replay system should be configured to ignore requests for outgoing E-mails.

5.2.3 Remapping Connections

During workload capture, connection strings used to connect to the production system are captured. In order for the replay to succeed, you need to remap these connection strings to the replay system. The replay clients can then connect to the replay system using the remapped connections.

For Oracle Real Application Clusters (Oracle RAC) databases, you can map all connection strings to a load balancing connection string. This is especially useful if the number of nodes on the replay system is different from the capture system. Alternatively, if you want to direct workload to specific instances, you can use services or explicitly specify the instance identifier in the remapped connection strings.

5.2.4 Specifying Replay Options

After the database is restored and connections are remapped, you can set the following replay options as appropriate:

• Preserving COMMIT Order

• Controlling Session Connection Rate

• Controlling Request Rate Within a Session

5.2.5 Using Filters with Workload Replay

By default, all captured database calls are replayed during workload replay. You can use workload filters to specify which database calls to include in or exclude from the workload during workload replay.

Workload replay filters are first defined and then added to a replay filter set so they can be used in a workload replay. There are two types of workload filters: inclusion filters and exclusion filters. Inclusion filters enable you to specify database calls that will be replayed. Exclusion filters enable you to specify database calls that will not be replayed. You can use either inclusion filters or exclusion filters in a workload replay, but not both. The workload filter is determined as an inclusion or exclusion filter when the replay filter set is created.

5.2.6 Setting Up Replay Clients

The replay client is a multithreaded program (an executable named wrc located in the $ORACLE_HOME/bin directory) where each thread submits a workload from a captured session. Before replay begins, the database will wait for replay clients to connect. At this point, you need to set up and start the replay clients, which will connect to the replay system and send requests based on what has been captured in the workload.

Before starting replay clients, ensure that the:

• Replay client software is installed on the hosts where it will run

• Replay clients have access to the replay directory

• Replay directory contains the preprocessed workload capture

• Replay user has the correct user ID, password, and privileges (the replay user needs the DBA role and cannot be the SYS user)

• Replay clients are not started on a system that is running the database

• Replay clients read the capture directory on a file system that is different from the one on which the database files reside

  To do this, copy the capture directory to the system where the replay client will run. After the replay is completed, you can delete the capture directory.

After these prerequisites are met, you can proceed to set up and start the replay clients using the wrc executable. The wrc executable uses the following syntax:

wrc [user/password[@server]] MODE=[value] [keyword=[value]]

The parameters user and password specify the username and password used to connect to the host where the wrc executable is installed. The parameter server specifies the server where the wrc executable is installed. The parameter mode specifies the mode in which to run the wrc executable. Possible values include replay (the default), calibrate, and list_hosts. The parameter keyword specifies the options to use for the execution and is dependent on the mode selected. To display the possible keywords and their corresponding values, run the wrc executable without any arguments.

The following sections describe the modes that you can select when running the wrc executable:

• Calibrating Replay Clients

• Starting Replay Clients

• Displaying Host Information

%> wrc mode=calibrate replaydir=./replay

Workload Replay Client: Release 11.2.0.0.2 - Production on Fri May 1
13:06:33 2009
 
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
 
Report for Workload in: /oracle/replay/
-----------------------
 
Recommendation:
Consider using at least 21 clients divided among 6 CPU(s).
 
Workload Characteristics:
- max concurrency: 1004 sessions
- total number of sessions: 1013
 
Assumptions:
- 1 client process per 50 concurrent sessions
- 4 client process per CPU
- think time scale = 100
- connect time scale = 100
- synchronization = TRUE

%> wrc system/password@test mode=replay replaydir=./replay

%> wrc mode=list_hosts replaydir=./replay

Workload Replay Client: Release 11.2.0.0.2 - Production on Fri May 1 13:44:48 2009
 
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
 
Hosts found:
Capture:
        prod1
        prod2
Replay 1:
        test1
Replay 2:
        test1
        test2
Replay 3:
        testwin

5.4.1 Monitoring an Active Workload Replay

This section describes how to monitor an active workload replay using Enterprise Manager.

To monitor an active workload replay:

1. On the Software and Support page, under Real Application Testing, click Database Replay.

   The Database Replay page appears.

2. Under Active Capture and Replay, select the workload replay you want to monitor and click View.

   The View Workload Replay page appears.

   
   Description of the illustration dbr_replay_view_active.gif
   
   

3. Once the workload replay is completed, you can assess various types of information about the replay using this page, as described in "Viewing a Completed Workload Replay".

   To stop the workload replay manually, click Stop Replay. To return to the Database Replay page, click OK.

5.4.2 Viewing a Completed Workload Replay

This section describes how to view a completed workload replay using Enterprise Manager.

To view a completed workload replay:

1. On the Software and Support page, under Real Application Testing, click Database Replay.

   The Database Replay page appears.

2. In the Go to Task column, click the icon that corresponds to the Replay Workload task.

   The Replay Workload page appears.

3. In the Directory Object list, select the directory that contains the replayed workload that you want to view.

   After the directory is selected, the Replay Workload page will be refreshed to display the Capture Summary and the Replay History sections.

4. The Replay History section displays previous replays of the workload capture. To view details about a replay, select the replay and click View.

   The View Workload Replay page appears.

5. Under Summary, information about the workload replay is displayed.

6. To view the workload profile, click the Workload Profile tab.

   There are two types of charts that are available under the Workload Profile tab:

   • Elapsed Time Comparison

     To view this chart, in the Chart Type field, select Elapsed Time Comparison.

     
     Description of the illustration dbr_elapsed_time_comp.gif
     
     

     The Elapsed Time Comparison chart shows how much time it has taken to replay the same workload compared to the elapsed time during the workload capture. If the Replay bar is shorter than the Capture bar, the replay system is processing the workload faster than the capture system.

   • User Call Progress

     To view this chart, in the Chart Type field, select User Call Progress.

     
     Description of the illustration dbr_user_call_progress.gif
     
     

     The User Call Progress chart shows how much time it has taken to replay the same workload compared to the elapsed time during the workload capture in terms of user calls. If the Replay line is above or to the left of the Capture line, the replay system is processing the workload faster than the capture system.

     If the workload capture was performed on an older version of Oracle Database, then the Capture line may not appear in the User Call Progress chart because the user call data may be unavailable. In this case, import the user call data by clicking the Import User Call Data button, which will appear next to the Chart Type list.

   Under Divergence, any error and data discrepancies between the replay system and the capture system are displayed as diverged database calls during replay. The percentage of total calls that diverged can be used as a measure of the replay quality. To view details about the diverged calls, click the link that corresponds to the type of diverged call in the Number of Calls column to bring up the Diverged Calls During Replay page. The Diverged Calls During Replay page shows the most relevant set of replayed calls that diverged from the workload captured by grouping them based on common attribute values and specified filter conditions. To view details about a particular diverged call—such as the call attributes, SQL text, and bind variables—click the corresponding link in the SQL ID column to bring up the Replay Diverged Statement page.

   To view a detailed comparison of the workload during capture and replay, expand Detailed Comparison.

   
   Description of the illustration dbr_replay_view_detail.gif
   
   

   The Detailed Comparison section displays the following information:

   • Duration

     The duration that was captured in a workload is compared to the amount of time it took to replay the workload. In the Capture column, the duration of the time period that was captured is shown. In the Replay column, the amount of time it took to replay the workload is shown. The Percentage of Capture column shows the percentage of the captured duration that it took to replay the workload. If the value is under 100 percent, the replay system processed the workload faster than the capture system. If the value is over 100 percent, the replay system processed the workload slower than the capture system.

   • Database time

     The database time that is consumed in the time period that was captured is compared to the amount of database time that is consumed when replaying the workload.

   • Average active sessions

     The number of average active sessions captured in the workload is compared to the number of average active session that are replayed.

   • User calls

     The number of user calls captured in the workload is compared to the number of user calls that are replayed.

   To view the workload replay report, click View Workload Replay Report. For information about using the Workload Replay report, see "Reviewing Workload Replay Reports".

7. To view the connection strings used in the capture and the replay systems, click the Connection Mappings tab.

8. To view replay parameters used by the workload replay, click the Replay Parameters tab.

9. To run a report, click the Report tab.

   
   Description of the illustration dbr_replay_view_report.gif
   
   

   There are several types of reports you can run for a completed workload replay:

   • Workload Replay

     The Workload Replay report contains information that can be used to measure data and performance divergence between the capture system and the replay system. To run this report, under Workload Replay Report, click Run Report. For information about using the Workload Replay report, see "Reviewing Workload Replay Reports".

   • Replay Compare Period

     The Replay Compare Period report can be used to compare one workload replay to its capture or to another replay of the same capture. Before running this report, AWR data for the captured or replayed workload must have been previously exported. To run this report, under Compare Period Report, select the first and second workload captures or replays you want to compare and click Run Replay Compare Period Report.

   • AWR Compare Period

     The AWR Compare Period report can be used to compare the AWR data in one workload capture or replay with another. Before running this report, AWR data for the captured or replayed workload must have been previously exported. To run this report, under Compare Period Report, select the first and second workload captures or replays you want to compare and click Run AWR Compare Period Report. If AWR data is not previously exported from the captured or replayed workload, you will be prompted to import the AWR data before continuing. For more information about the AWR Compare Period report, see Oracle Database 2 Day + Performance Tuning Guide.

   • AWR

     The AWR report shows the AWR data contained in a workload that was captured or replayed. Before running this report, AWR data must have been previously exported from the captured or replayed workload. To run this report, under AWR Report, select the workload capture or replay for which you want to generate an AWR report and click Run Report. If AWR data is not previously exported from the captured or replayed workload, you will be prompted to import the AWR data before continuing. For more information about the AWR report, see Oracle Database Performance Tuning Guide.

   • ASH

     The ASH report contains active session history (ASH) information for a specified duration of a workload that was captured or replayed. Before running this report, AWR data must have been previously exported from the captured or replayed workload. To run this report, under ASH Report, select the workload capture or replay for which you want to generate an ASH report. Specify the duration using the Start Date, Start Time, End Date, and End Time fields. You can also apply filters using the Filter field. Once the duration and filters are specified, click Run Report. If AWR data is not previously exported from the captured or replayed workload, you will be prompted to import the AWR data before continuing. For more information about the ASH report, see Oracle Database 2 Day + Performance Tuning Guide.

   The Report window opens while the report is being generated. Once the report is generated, you can save the report by clicking Save to File.

10. To return to the Database Replay page, click OK.

5.5.1 Initializing Replay Data

After the workload capture is preprocessed and the test system is properly prepared, the replay data can be initialized. Initializing replay data loads the necessary metadata into tables required by workload replay. For example, captured connection strings are loaded into a table where they can be remapped for replay.

To initialize replay data, use the INITIALIZE_REPLAY procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY (replay_name => 'dec06_102',
                           replay_dir => 'dec06');
END;
/

In this example, the INITIALIZE_REPLAY procedure loads preprocessed workload data from the dec06 directory into the database.

The INITIALIZE_REPLAY procedure in this example uses the following parameters:

• The replay_name required parameter specifies a replay name that can be used with other APIs to retrieve settings and filters of previous replays.

• The replay_dir required parameter specifies the directory that contains the workload capture that will be replayed.

5.5.2 Connection Remapping

After the replay data is initialized, connection strings used in the workload capture need to be remapped so that user sessions can connect to the appropriate databases and perform external interactions as captured during replay. To view connection mappings, use the DBA_WORKLOAD_CONNECTION_MAP view. For information about connection remapping, see "Remapping Connections".

To remap connections, use the REMAP_CONNECTION procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.REMAP_CONNECTION (connection_id => 101,
                           replay_connection => 'dlsun244:3434/bjava21');
END;
/

In this example, the connection that corresponds to the connection ID 101 will use the new connection string defined by the replay_connection parameter.

The REMAP_CONNECTION procedure in this example uses the following parameters:

• The connection_id required parameter is generated when initializing replay data and corresponds to a connection from the workload capture.

• The replay_connection optional parameter specifies the new connection string that will be used during workload replay.

5.5.3 Setting Workload Replay Options

After the replay data is initialized and the connections are appropriately remapped, you need to prepare the database for workload replay. For information about workload replay preparation, see "Steps for Replaying a Database Workload".

To prepare workload replay on the replay system, use the PREPARE_REPLAY procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY (synchronization => TRUE);
END;
/

In this example, the PREPARE_REPLAY procedure prepares a replay that has been previously initialized. The COMMIT order in the workload capture will be preserved.

The PREPARE_REPLAY procedure uses the following parameters:

• The synchronization required parameter determines if synchronization will be used during workload replay.

  If this parameter is set to SCN, the COMMIT order in the captured workload will be preserved during replay and all replay actions will be executed only after all dependent COMMIT actions have completed. The default value is SCN.

  If this parameter is set to OBJECT_ID, all replay actions will be executed only after all relevant COMMIT actions have completed. Relevant COMMIT actions must meet the following criteria:

  • Issued before the given action in the workload capture

  • Modified at least one of the database objects for which the given action is referencing, either implicitly or explicitly

  Setting this parameter to OBJECT_ID allows for more concurrency during workload replays for COMMIT actions that do not reference the same database objects as the workload capture.

  You can disable this option by setting the parameter to OFF, but the replay will likely yield significant data divergence. However, this may be desirable if the workload consists primarily of independent transactions, and divergence during unsynchronized replay is acceptable.

• The connect_time_scale parameter scales the elapsed time from when the workload capture started to when the session connects with the specified value and is interpreted as a % value. Use this parameter to increase or decrease the number of concurrent users during replay. The default value is 100.

• The think_time_scale parameter scales the elapsed time between two successive user calls from the same session and is interpreted as a % value. Setting this parameter to 0 will send user calls to the database as fast as possible during replay. The default value is 100.

• The think_time_auto_correct parameter corrects the think time (based on the think_time_scale parameter) between calls when user calls take longer to complete during replay than during capture. This parameter can be set to either TRUE or FALSE. Setting this parameter to TRUE reduces the think time if the workload replay is taking longer than the workload capture. The default value is TRUE.

• The scale_up_multiplier parameter defines the number of times the workload is scaled up during replay. Each captured session will be replayed concurrently for as many times as specified by this parameter. However, only one session in each set of identical replay sessions will execute both queries and updates. The rest of the sessions will only execute queries.

For more information about setting these parameters, see "Specifying Replay Options".

5.5.4 Defining Workload Replay Filters and Replay Filter Sets

This section describes how to add and remove workload replay filters, and how to create and use replay filter sets. For information about using workload filters and replay filter sets with workload replay, see "Using Filters with Workload Replay".

This section contains the following topics:

• Adding Workload Replay Filters

• Deleting Workload Replay Filters

• Creating a Replay Filter Set

• Using a Replay Filter Set

BEGIN
  DBMS_WORKLOAD_REPLAY.ADD_FILTER (
                           fname => 'user_ichan',
                           fattribute => 'USER',
                           fvalue => 'ICHAN');
END;
/

BEGIN
  DBMS_WORKLOAD_REPLAY.DELETE_FILTER (fname => 'user_ichan');
END;
/

BEGIN
  DBMS_WORKLOAD_REPLAY.CREATE_FILTER_SET (
                           replay_dir => 'apr09',
                           filter_set => 'replayfilters',
                           default_action => 'INCLUDE');
END;
/

BEGIN
  DBMS_WORKLOAD_REPLAY.USE_FILTER_SET (filter_set => 'replayfilters');
END;
/

5.5.5 Starting a Workload Replay

Before starting a workload replay, you must first:

• Preprocess the captured workload, as described in "Preprocessing a Database Workload Using APIs"

• Initialize the replay data, as described in "Initializing Replay Data"

• Specify the replay options, as described in "Setting Workload Replay Options"

• Start the replay clients, as described in "Starting Replay Clients"

To start a workload replay, use the START_REPLAY procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.START_REPLAY ();
END;
/

5.5.6 Pausing a Workload Replay

To pause a workload replay that is in progress, use the PAUSE_REPLAY procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.PAUSE_REPLAY ();
END;
/

Pausing a workload replay will halt all subsequent user calls issued by the replay clients until the workload replay is either resumed or cancelled. User calls that are already in progress will be allowed to complete. This option enables you to temporarily stop the replay to perform a change and observe its impact for the remainder of the replay.

5.5.7 Resuming a Workload Replay

To resume a workload replay that is paused, use the RESUME_REPLAY procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.RESUME_REPLAY ();
END;
/

5.5.8 Cancelling a Workload Replay

To cancel a workload replay, use the CANCEL_REPLAY procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.CANCEL_REPLAY ();
END;
/

5.5.9 Exporting AWR Data for Workload Replay

Exporting AWR data enables detailed analysis of the workload. This data is also required if you plan to run the AWR Compare Period report on a pair of workload captures or replays.

To export AWR data, use the EXPORT_AWR procedure:

BEGIN
  DBMS_WORKLOAD_REPLAY.EXPORT_AWR (replay_id => 1);
END;
/

In this example, the AWR snapshots that correspond to the workload replay with a replay ID of 1 are exported. The EXPORT_AWR procedure uses the replay_id required parameter, which specifies the ID of the replay whose AWR snapshots will be exported. This procedure will work only if the corresponding workload replay was performed in the current database and the AWR snapshots that correspond to the original replay time period are still available.

5.6.1 Retrieving Information About Diverged Calls

During replay, any error and data discrepancies between the replay system and the capture system are recorded as diverged calls.

To retrieve information about a diverged call—including its SQL identifier, SQL text, and bind values—call the GET_DIVERGING_STATEMENT function using the following parameters:

• Set the replay_id parameter to the ID of the replay in which the call diverged

• Set the stream_id parameter to the stream ID of the diverged call

• Set the call_counter parameter to the call counter of the diverged call

To view these information about a diverged call, use the DBA_WORKLOAD_REPLAY_DIVERGENCE view. The following example illustrates a function call:

EXEC DBMS_WORKLOAD_REPLAY.GET_DIVERGENT_STATEMENT(replay_id => 1, -
       stream_id => 3, - 
       call_counter => 5);

5.6.2 Monitoring Workload Replay Using Views

This section summarizes the views that you can display to monitor workload replay. You need DBA privileges to access these views.

• The DBA_WORKLOAD_CAPTURES view lists all the workload captures that have been captured in the current database.

• The DBA_WORKLOAD_FILTERS view lists all workload filters for workload captures defined in the current database.

• The DBA_WORKLOAD_REPLAYS view lists all the workload replays that have been replayed in the current database.

• The DBA_WORKLOAD_REPLAY_DIVERGENCE view enables you to view information about diverged calls, such as the replay identifier, stream identifier, and call counter.

• The DBA_WORKLOAD_REPLAY_FILTER_SET view lists all workload filters for workload replays defined in the current database.

• The DBA_WORKLOAD_CONNECTION_MAP view lists the connection mapping information for workload replay.

• The V$WORKLOAD_REPLAY_THREAD view lists information about all sessions from the replay clients.