RMAN provides several features not available when you make user-managed backups with
operating system commands.

• Frequently executed operations can be stored as scripts in the database.
• With block change tracking enabled in the database RMAN can limit incremental backups to recording only those blocks that have changed since the previous backup.
• RMAN can be used to manage the size of backup pieces and save time by parallelizing the backup operation.
• RMAN can recover an individual corrupt data block or set of data blocks within a data file rather than restoring and recovering the entire data file.
• RMAN operations can be integrated with the Oracle Database Scheduler to automate
  backup operations.
• You can use RMAN to detect block corruption. The information relating to the block
  corruption that is detected during backup can be obtained by using the
  V$BACKUP_CORRUPTION and V$COPY_CORRUPTION dynamic views.
• RMAN provides performance enhancements such as:
          – Automatic parallelization of backup, restore, and recovery operations
          – No generation of extra redo during online database backups
          – Backups can be restricted to limit reads per file, per second to avoid interfering   with OLTP work
          -  Prevention of flooding of any one file with reads and writes while still keeping a tape drive streaming, using multiplexing
• RMAN has a media management API to work seamlessly with third-party media
  management tools interfacing with storage devices providing increased speed and
  reliability.
• Under the user-managed method you need to keep track of all database files and backups. In a recovery situation you must locate backups for each data file, copy them to the correct place using operating system commands, and choose which redo log files to apply. RMAN manages these tasks automatically.

Beginning with Release 11g, the alert log file is written as XML formatted and as a text file (as in previous releases). The default location of both these files is the new ADR home (Automatic Diagnostic Respository, another dump dest in 11g).

The database, Automatic Storage Management (ASM), Cluster Ready Services (CRS), and other Oracle products or components store all diagnostic data in the ADR.

The ADR is set by using the DIAGNOSTIC_DEST initialization parameter. If this parameter is omitted, then, the default location of ADR is, ‘$ORACLE_HOME/log’

The location of an ADR home is given by the following path, which starts at the ADR base directory:

$ORACLE_BASE/diag/product_type/product_id/insance_id

v$diag_info can be queried to check ADR locations, including ADR BASE and ADR home.

Within the ADR home directory are subdirectories:

alert       – XML formatted alertlog
trace      - Files and text alert.log file
cdump – core files

message_text can have multiple lines of ORA- errors and related tracing file information, but for monitoring only need first error code.

SELECT        record_id, substr(message_text, instr(message_text,'ORA-'),
              instr(message_text,chr(10),1,2)-instr(message_text,'ORA-') ) message_text
FROM          <strong>X$DBGALERTEXT</strong>
WHERE         originating_timestamp > sysdate-1
              AND message_text like '%ORA-%'
ORDER BY  record_id desc

Query to get errors for the past day:

</span>
<pre>SELECT  distinct    originating_timestamp,   message_text
FROM    X$DBGALERTEXT
WHERE   originating_timestamp > sysdate-1
AND     (  message_text = 'ORA-00600'  OR
           message_text like '%Fatal%'
        )

Query to get errors for the last 10 minutes:

</span>
<pre>SELECT         rownum       “line”,    message_text “error”
FROM           X$DBGALERTEXT
WHERE          originating_timestamp > (sysdate – 10/1440)  AND
               message_text like ‘%ORA-%’
ORDER BY       originating_timestamp;

Links:

Oracle 11g Reading Alert Log Via SQL

Note: 1072547.1 – How To Edit, Read, and Query the Alert.Log

Oracle 11g Automatic Diagnostic Repository

A whole database backup includes all data files and at least one control file.

Full backups make a copy of every data block within the files being backed up that contains data.

Partial database backups may include zero or more tablespaces, zero or more data files, and may or may not include a control file.

Incremental backups make a copy of all data blocks that have changed since some previous backup. Oracle Database 10g supports two levels of incremental backup (0-1). A level 0 or baseline backup is equivalent to a full backup and contains all data blocks. A level 1 incremental backs up all database blocks changed since the level 0 backup. To restore using incremental backups, the baseline backup must first be restored, and then the incremental.

Offline backups or Cold backups(also known as consistent backups) are taken while the database is not open. They are consistent because at the time of the backup, the SCN data file headers matches the SCN in the control files.

Online backups (also known as hot or inconsistent backups) are taken while the database is open. The backups are inconsistent because with the database open there is no guarantee that the data files are synchronized with the control files. Inconsistent backups require recovery in order to be used. For online backup, ARCHIVELOG mode should be enabled.

Modes of Backups

i. ARCHIVELOG

If the database is in noarchive log mode, online redo logs can be overwritten without making sure they are saved. This implies that a database cannot be recovered fully (it can be rolled forward to the last transaction) even if backups were made.

ii. NOARCHIVELOG

If the database is in log archive mode, the database makes sure that online redo logs are not overwritten before they have been archived. Database can be recovered to the last

Types of Backups

i. Physical Backup

Physical backups are backups of the physical files used in storing and recovering your database, such as datafiles, control files, and archived redo logs.

ii. Logical Backup

Logical backups contain logical data (tables or stored procedures) exported from a database with an Oracle export utility and stored in a binary file, for later re-importing into a database using the corresponding Oracle import utility.

Backup Types based on Physical Backup

For performing backup and recovery based on physical backups, you have two solutions available:

• Recovery Manager (RMAN), a tool (with command-line client and Enterprise Manager GUI interfaces) that integrates with sessions running on the Oracle server to perform a range of backup and recovery activities, as well as maintaining a repository of historical data about your backups
• The traditional user-managed backup and recovery, where you directly manage the files that make up your database with a mixture of host operating system commands and SQL*Plus backup and recovery-related capabilities

Recovery Manager isthe preferred solution for database backup and recovery. It can perform the same types of backup and recovery available through user-managed methods more easily, provides a common interface for backup tasks across different host operating systems, and offers a number of backup techniques not available through user-managed methods.

Image Copies and Backup Sets

Image copies are duplicates of data or archived log files (similar to simply copying the files using OS commands).

Backup sets are copies of one or more data or archived log files. With backup sets, empty data locks are not stored, thereby causing backup sets to use less space on disk or tape. Backup sets an be compressed to further reduce the space requirements of the backup.

Image copies must be backed up to disk whereas Backup sets can be sent to disk or directly to tape.

Database Architecture                   

An Oracle database server consists of an Oracle database and an Oracle instance.

Oracle Database architecture is based on physical and logical structures. 

Oracle Database Physical Structure

Oracle database consists of physical files to store information. Basically there are three types of physical files required for database:

Core Files:

• Data Files                    Contains all the data of database
• Control Files              Contains data about database(metadata)
• Redo Log Files          Contains all changes made to data, allows recovery

Other Files

• Parameter File         Defines how instance will be configured when it starts up
• Password File           Allow users to connect remotely to the database
• Archive Log Files   Contains history of redo log files
• Trace Files                 Contains messages written by server and background process
• Alert File                    Special trace file, chronological log of messages and errors              

Oracle Database Logical Structure

Oracle database is divided into logical storage units known as Tablespaces, which can be used to group related logical structures together. Logical structure mainly consists of following structures starting with the bigger unit and going down to the smallest.

• Tablespace
• Segments
• Extents
• Data blocks

Tablespace

Each database is logically divided into one or more tablespaces. One or more datafiles are explicitly created for each tablespace to physically store the data of all logical structures in a tablespace. The combined size of the datafiles in a tablespace is the total storage capacity of the tablespace.

Segment

Tablespace consists of one or more segments. A segment is a set of extents allocated for a certain logical structure stored in the same tablespace. The segments can be of one of following types:

• Data segment  (stores user data within the database)
• Index Segment (store Indexes)
• Temporary Segment ( stores rollback segment)
• Rollback/Undo segment (created when sql statement needs a temporary work area)

Extent

An extent is a specific number of contiguous data blocks, obtained in a single allocation, used to store a specific type of information in one data file.

Data Blocks

Oracle database data is stored in data blocks(at the lowest logical level).  One data block corresponds to a specific number of bytes of physical database space on disk(default 8K). The standard block size is specified by the DB_BLOCK_SIZE initialization parameter.

Oracle Data blocks are the smallest unit of I/O in the database and they are mapped to OS block on the storage device

Oracle Instance

Oracle instance is comprised of memory structures and background processes that are required to perform different tasks to keep the instance and database running. Instance does not exist until database is started, which involves reading initialization parameter file and configuring instance according to the information within the parameter file.

Memory Structures

Instance has two major memory structures:

• System Global Area (SGA):  stores information in memory shared by processes in Oracle
• Program Global Area (PGA): contains information private to a particular process

Background Processes

Oracle creates a set of background processes for an instance that  manage the memory structures, asynchronously perform I/O to write data to disk, and do general housekeeping.
The most common background processes are the following:

• System monitor (SMON): Performs crash recovery when the instance starts after a failure
• Process monitor (PMON): Performs process cleanup when a user process fails
• Database writer (DBWn): Writes modified blocks from the database buffer cache to the files on disk
• Checkpoint (CKPT): Signals DBWn at checkpoints and updates all of the data files and control files of the database to indicate the most recent checkpoint
• Log writer (LGWR): Writes redo log entries to disk
• Archiver (ARCn): Copies the redo log files to archival storage when the log files are full or a log switch occurs

C:\Users\ACE>sqlplus
SQL*Plus: Release 11.2.0.1.0 Production on Sat Mar 5 19:52:01 2011

Enter user-name: sys as sysdba
Enter password:
Connected.
SQL>  shutdown immediate
ORA-24324: service handle not initialized
ORA-24323:  value not allowed
ORA-01090: shutdown in progress - connection is not  permitted

SQL> shutdown abort
ORA-01031: insufficient privileges

SQL> select instance from v$instance;
select instance from  v$instance
* ERROR at line 1: ORA-01012: not logged on
Process ID:  0
Session ID: 0 Serial number: 0

In this scenario it did “connect / as sysdba”, but in actually failed to connect. Then, when I
tried to shutdown abort, you got ORA-1031 cause you weren’t actually connected.
It’s interesting to note that if you connect as sysdba, and the connection is successful, you see something like:”Connected to:Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 – 64bit
Production With the Partitioning, OLAP and Data Mining options”If you fail to connect, cause process table is full, you’ll  get: “Connected.”
So, “Connected.” means you *failed* to connect!

ORA-01012: not logged on (check this link for another reason why Oracle shows “not logged on” error)

Although shutdow abort causes instance recovery but it becomes necessary if you want to accomplish a repetitive task at a given time in a specific time. You can combine it with restricted startup and then again shutdown with immediate option and better run it through a script.

#!/bin/ksh
# Source the db.env to set the Oracle environment

export ORACLE_SID
ORACLE_HOME=`cat /etc/oratab|grep ^$ORACLE_SID:|cut -f2 -d':'`
export ORACLE_HOME
PATH=$ORACLE_HOME/bin:$PATH
export PATH

shutdown abort;
startup restrict;
shutdown immediate;

Debugging a hung database

Starting with Oracle 11g,  SQL*Plus has a new option called “prelim” . This option is very useful for running oradebug and other utilities that do not require a real connection to the database.

C:\ACE> sqlplus –prelim

SQL>

or

SQL> set _prelim on
SQL> connect / as sysdba

Run oradebug commands to diagnose a hung database issue using the

new hanganalyze utility:

SQL> oradebug hanganalyze

Wait for some time to identify process state changes.

SQL> oradebug hanganalyze

Open a separate SQL session and immediately generate a system state dump.

SQL> alter session set events 'immediate trace name SYSTEMSTATE level 10';

Further Reading:

Fix Oracle Hanging

Monitoring Methodologies

• Reactive
• Proactive

Reactive monitoring is not a recommend approach but at times it becomes inevitable. Oracle 10g provide tools that aid in proactively monitoring database. Server-generated alerts and Automated Database Diagnostic Monitor(AADM) are going to be discussed in another post.

Database and Instance Metrics

Performance measurements are commonly referred as databases metrics which is basically the rate of change of cumulative statistic that need to be monitored. Metrics are indicators of the health of various database targets. Excerpt from Oracle Database 10g New Features by Rampant TechPress

“The Metrics are the statistics derived from base statistics. They represent the delta values between the snapshot periods. Metrics are used by internal components (clients) for system health monitoring, problem detection and self-tuning. There are hundreds of different measurement points available to monitor database system.

Each metric is also associated with a metric name. You can query the view v$metricname to find the names of all the metrics.

SQL> select METRIC_NAME, METRIC_UNIT from v$metricname; "

Viewing Metrics

The new MMON background process collects database metrics continuously and automatically saves them in the SGA for one hour.It also transfers the memory version of AWR stats to disk on regular basis(in snapshots). Performance metrics are available through

• Data Dictionary
• Dynamic Performance Views
• Optimizer Statistics

Data Dictionary Metrics

Data dictionary metrics provide information about space consumption and object status. Database indexes and stored procedures both need to be VALID to be used.

Unusable indexes always require DBA intervention to restore them to a valid state. Invalid
PL/SQL objects will normally recompile automatically the first time they are called, but
sometimes require DBA intervention in cases where the automatic recompilation fails.

Compiling PL/SQL procedures and Packages

Indexes can become unusable due to normal maintenance operations on tables. Unusable indexes can be rebuild to make them valid.

Rebuilding Indexes

Viewing In-Memory Metrics

All system related metrics saved in memory and they can be viewed using dynamic performance views. Most of the cumulative statistics can be viewed through following:

• V$SYSSTAT
• V$SESSSTAT
• V$SYSMETRIC
• V$SYSMETRIC_HISTORY

Example of few system metrics maintained in V$SYSMETRIC view:

• Buffer Cache Hit Ratio
• CPU Usage Per Sec
• Disk Sort Per Sec
• Host CPU Utilization (%)
• Library Cache Hit Ratio
• SQL Service Response Time
• Shared Pool Free (%)

Viewing Saved Metrics

After every 60 min, MMON places metric info from SGA to disk through AWR snapshot mechanism. The metric data that AWR collects through MMON is permanently stored in DBA_HIST_* views like

• DBA_HIST_SERVICE_NAME
• DBA_HIST_SESSMETRIC_HISTORY

Optimizer Statistics

Optimizer statistics for tables and indexes are stored in the data dictionary. These statistics are not intended to provide real-time data. They provide the optimizer a statistically correct snapshot of data storage and distribution which the optimizer uses to make decisions on how to access data.

Metrics collected include:

• Size of the table or index in database blocks
• Number of rows
• Average row size and chain count (tables only)
• Height and number of deleted leaf rows (indexes only)

As data is inserted, deleted, and modified these values change. The performance impact of
maintaining real-time data distribution statistics would be prohibitive, so these statistics are updated by periodically gathering statistics on tables and indexes.

SQL*Loader reads from a data file(s)(usually a flat file) and a control file(file that contains description of the data to be loaded and it has nothing to do with database control files) . During loading , process information regarding loading is written to log file, bad rows are to bad file and discarded rows to the discard file.

SQL*Loader environment

SQL*Loader Control File

The control file is a text file that contains DDL instructions and provides the following information to SQL*Loader:

• The name and location of the input data file
• The format of the records in the input data file
• The name of the table(s) to be loaded
• Table and field list information
• SQL*Loader configuration(memory management, record rejection criteria, interrupted .
• The names and locations of bad file and  discard file

SQL*Loader Options:

Following are few commonly used Options in control file:

 SKIP = n (Number of records to be skipped before loading)
  ERRORS = n (Number of erroneous records allowed )
 ROWS = n (Number of rows before committing )
  DIRECT = {TRUE | FALSE}
 PARALLEL = {TRUE | FALSE}

If there is a small amount of data then its possible for the control file to contain the actual data to be loaded. This is achieved by using BEGINDATA clause before the actual data in the control file.

Case Study for simple control file including data

SQL*Loader Input Data Files

SQL*Loader reads data from one or more files organized as records in one of three formats:

• Fixed-record format
• Variable-record format
• Stream-record format

Fixed-Record Format

A file is in fixed-record format when all records in a data file are the same byte length.

INFILE  ‘filename’ “fix  n”

The following example shows a control file that specifies a fixed-record format data file. The data file contains 2 physical records. The first record is [0001, abcd], which is exactly nine

bytes long (using a single-byte character set) and the carriage return is the tenth byte.

Control File:

load data

infile ‘example.dat’    “fix 10″

into table example

fields terminated by ’,’

(col1, col2)

input.dat:

0001,abcd

0002,fghi

Variable-Record Format

A file is in variable-record format when the length of each record in a character field is included at the beginning of each record in the data file.
INFILE  ‘filename’ “var  n”   (n is the number of bytes in record length field)

The following example tells SQL*Loader to look for data in the

input.dat data file and to expect variable-record format where the record length fields are 3

bytes long. The example.dat data file consists of two physical records. The first is specified

to be 009 ( nine) bytes long, the second is 010 bytes long (including a one-character

newline).

Control File:

load data

infile ‘example.dat’     “var 3“

into table example

fields terminated by ’,’ optionally enclosed by ’”’

(col1 char(5),col2 char(7))

input.dat:

009hello,cd,

010world,im,

Stream-Record Format

A file is in stream-record format when the records are not specified by size; instead SQL*Loader forms records by scanning for the record terminator.

INFILE  ‘filename’ “str terminator_string”

The following example illustrates loading data in stream-record format where the terminator string is specified using a character string, ’|\n’.

Control File:

load data

infile ‘example.dat’   “str ’|\n’“

into table example

fields terminated by ’,’ optionally enclosed by ’”’

(col1 char(5), col2 char(7))

input.dat:

hello,world,|

james,bond,|

* Filler fields in control file are fields that have names but they are not loaded into the table.

Loading Methods

SQL*Loader provides two methods for loading data:

• Conventional path

• Direct path

Conventional Path Load

Conventional path load builds an array of rows to be inserted and uses the SQL INSERT statement to load the data. During conventional path loads, input records are parsed based on field specifications, and an array of records is built and inserted into the table specified in the control file.

Direct Path Load

A direct path load builds blocks of data in memory and saves these blocks directly into the extents allocated for the table being loaded. Online redo log file entries are not generated unless the database is in ARCHIVELOG mode. Direct path loads use the field specifications to build whole Oracle blocks of data, and write the blocks directly to Oracle data files above the high water mark.

The high-water mark is the highest point to which data has been written into the table so far. Direct path load bypasses the database buffer cache and accesses the SGA only for extent management and adjustment of the high-water mark.

Comparing Direct and Conventional Path Loads

Comparing Direct and Conventional Path Loads

Method of Saving Data

Conventional path loads use SQL processing and a database COMMIT for saving data. The

insertion of an array of records is followed by a commit operation.Direct path loads use data saves to write blocks of data to Oracle data files. The following features differentiate a data save from a COMMIT:

• During a data save, only full database blocks are written to the database.

• The blocks are written after the high-water mark of the table.

• After a data save, the high-water mark is moved.

• Internal resources are not released after a data save.

• A data save does not end the transaction.

• Indexes are not updated at each data save.

Logging Changes

Conventional path loading generates redo entries similar to any DML statement. When using a direct path load, redo entries are not generated if:
• The database is in NOARCHIVELOG mode

• The database is in ARCHIVELOG mode, but logging is disabled.

Logging can be disabled by setting the NOLOGGING attribute for the table or by using the UNRECOVERABLE clause in the control file.

Conventional Vs Direct Path Load (from Shailesh’s Blog )

Useful  Links:

Maximizing SQL*Load Performance( from Burleson Consulting )

Oracle FAQ on Orafaq

Intro to SQL*Loader by O’Reilly

Oracle SQL Loader Concepts on Oracle Site

ORA-12154: TNS:could not resolve service name

SQL> connect SYSTEM/manager@DEV
ERROR: ORA-12154: TNS:could not resolve service name

The most common TNS error mostly due to multiple ORACLE HOME, tnsnames.ora and sqlnet.ora conflicts.

• Check that the instance or database name is being typed in correctly. Check ORACLE_SID,ORACLE_HOME
• Make sure listener is up and running

lsnrctl > status           if down then

lsnrctl > start

• PING the server to see if network is up

ping 168.192.1.22

• TNSPING the server and then try connecting

tnsping DEV

• Check the tns names.ora file to validate that the instance name has been entered correctly. Make sure that all entries are properly terminated with parenthesis.
• Validate the domain name entry in sqlnet.ora is not conflicting with the full database name resolution.
• If nothings works, delete and recreate the entry to eliminate any character or control character issues.

For more details on ORA-12154 causes and actions

TNS-03505: Failed to resolve name

Message 3511 not found; product=network; facility=TNS
TNS-03505: Failed to resolve name

This is quite similar to the above error so you have to again check the environmental variables like ORACLE_HOME and PATH and make sure their values are set correctly.
The issue usually occurs when the tns entiries we provide has some unwanted characters or spaces etc.Please also check the tns entry(alias) is pointing to the correct database with sid hostname.

For more details on TNS-03505 reasons and remedy actions

ORA-12537: TNS: connection closed error

It happens if the connection has been closed most likely due to the following reasons:

• Firewall has disconnected idle connections
• Configuration problem in sqlnet.ora or listener.ora
• Time out occurred on client connection
• Path name that is too long for the Oracle TNS client on windows

Try the following to fix this issue:

1. Install the exe in a shorter named directory.
2. Modify the TNS Names entry so that it is shorter.
3. Replace domain names with IP Addresses if that shortens and vice versa.
4. Remove any parameters from the entry that are not needed.
5. Patch the Database Listener to the most recent version

For more details on ORA-12537 reasons and remedy actions

Troubleshooting Tips from the Field for Oracle Net Services

Here are some tips you may find helpful when you are having difficulty diagnosing network problems from Oracle documentation:

• Use the node or network address during configuration instead of the name of the server computer

  This eliminates any internal lookup problems and make the connection slightly faster.

• If you are using TCP/IP addresses, use the IP address rather than host name

  For example, change the (HOST=server_name) line in the tnsnames.ora file with the internet address, for example (HOST=198.32.3.5).

• Perform a loopback test

  Perform a loopback test on the server as described in “Testing Configuration on the Database Server”. If the test passes, ftp the tnsnames.ora and sqlnet.ora files to the client.

• Check what is between you and the server

  If it is a wide area network (WAN), identify any intermediate systems that may not work correctly. If all computers are fine, the problem may be a timing issue.

• Verify whether or not there is a timing issue

  Timing issues are associated with an ORA-12535 error in the client log files.

  To resolve this, try speeding up the connection by using exact addresses instead of names and increase the INBOUND_CONNECT_TIMEOUT_listener_name parameter in the listener.ora file. The default value for this parameter is 10 seconds.

• Determine which Oracle applications are failing

  SQL*Plus may work, but CASE tools may not. If you determine the problem is a data volume issue, try to transfer a large (5 MB) file with the base connectivity.

Resolving most common Error messages for Oracle Net Services

Parameter Files

Oracle must read either an initialization parameter file or a server
parameter file(SPFILE) to start an instance. These files contain list of initialization parameters and a value for each parameter.

• Server Parameter Files

• Initialization Parameter Files 

In the platform-specific default location, Oracle Database locates your initialization parameter file by examining filenames in the following order:

• spfile$ORACLE_SID.ora
• spfile.ora
• init$ORACLE_SID.ora

Initialization Parameters

Initialization parameters are divided into two groups: basic and advanced. In the majority of cases, it is necessary to set and tune only the basic parameters to get reasonable performance. In rare situations, modification to the advanced parameters may be needed for optimal performance.

Complete List of Initialization Parameters (10g)

Basic Parameters

There are about 29 basic parameters. Only a few of these parameters must be explicitly set because the default values are adequate in the majority of cases.

Advanced Parameters

Other than basic parameters, several parameters are used for database tuning and other advance services. Few examples of parameters

SESSION_CACHED_CURSORS

CURSOR_SHARING

DB_KEEP_CACHE_SIZE

OPEN_LINKS

AUDIT_FILE_DEST

JAVA_POOL_SIZE

LARGE_POOL_SIZE

STREAMS_POOL_SIZE

QUERY_REWRITE_ENABLED

Hidden/Undocumented Parameters

Hidden parameters should only be used as the last resort and preferably when advised by Oracle Support.

_abort_recovery_on_join

_alert_message_purge

_backup_io_pool_size

_cursor_plan_enabled

_log_checkpoint_recovery_check

Complete List of Hidden Parameters(10g)

Viewing and Changing Parameters Settings

• SHOW PARAMETERS command from SQL*PLUS can be used to to view parameter values
  • SQL> SHOW PARAMTERS CONTROL_FILES
    
    
• V$PARAMETER, V$PARAMETER2 views is used to display current parameter values
• V$SPPARAMETER view is used to display contents of server parameter file.

CHANGING PARAMETER VALUES 

  PFILE can be edited with any text editor but the SPFILE is a binary file and its parameter values can be changed using “ALTER SYSTEM SET” and “ALTER SYSTEM RESET”commands.

Removing a Parameter from SPFILE: 

Parameter Setting to SPFILE / Memory/ Both

SQL> ALTER SYSTEM SET open_cursors=300 SCOPE=SPFILE;

SQL> ALTER SYSTEM SET timed_statistics=TRUE SCOPE=BOTH SID='*';

The scope parameter can be set to SPFILE, MEMORY or BOTH. Memory is default if PFILE was used at STARTUP and BOTH is default if SPFILE was used at STARTUP.

• MEMORY:  The parameter value is set for the current instance only.
• SPFILE: The parameter value is updated in SPFILE and takes effect at next STARTUP.
• BOTH: The parameter value is set for current instance and persists to the SPFILE

SHUTDOWN [NORMAL | TRANSACTIONAL | IMMEDIATE | ABORT ]

SHUTDOWN NORMAL

Normal is the default shutdown mode.Normal database shutdown proceeds with the
following conditions:

• No new connections can be made.
• The Oracle server waits for all users to disconnect before completing the shutdown.
• Database and redo buffers are written to disk.
• Background processes are terminated, and the SGA is removed from memory.
• Oracle closes and dismounts the database before shutting down the instance.
• The next startup does not require an instance recovery.

SHUTDOWN TRANSACTIONAL

A transactional shutdown prevents clients from losing work. A transactional database

shutdown proceeds with the following conditions:

• No client can start a new transaction on this particular instance.
• A client is disconnected when the client ends the transaction that is in progress.
• When all transactions have finished, a shutdown occurs immediately.
• The next startup does not require an instance recovery.

SHUTDOWN IMMEDIATE

Immediate database shutdown proceeds with the following conditions:

• Current SQL statements being processed by Oracle are not completed.
• The Oracle server does not wait for the users, who are currently connected to the database, to disconnect.
• Oracle rolls back active transactions and disconnects all connected users.
• Oracle closes and dismounts the database before shutting down the instance.
• The next startup does not require an instance recovery.

SHUTDOWN ABORT

If no shutdown option works then database instance can be shutdown using abort option. Aborting an instance proceeds with the following conditions:

• Current SQL statements being processed by the Oracle server are immediately terminated.
• Oracle does not wait for users currently connected to the database to disconnect.
• Database and redo buffers are not written to disk.
• Uncommitted transactions are not rolled back.
• The instance is terminated without closing the files.
• The database is not closed or dismounted.
• The next startup requires instance recovery, which occurs automatically.

All shutdown options except ABORT are clean shutdown where no instance recovery is required. Shutdown abort is a dirty shutdown as it requires instance recovery before getting back up.

Oracle Streams captures database changes at a source database, stages the changes, propagates the changes to one or more destination databases, and then applies the changes at the destination database(s). Using Oracle Streams, enterprise systems can capture, propagate, and apply information as follows:

• Within an Oracle database
• Between two Oracle databases
• Among multiple Oracle databases
• Between an Oracle database and a non-Oracle database

Oracle9i Database introduced the Change Data Capture (CDC) feature.

Oracle 10g introduces Asynchronous Change Data Capture, which instead of using triggers uses the database log files to capture changes and apply them to collection tables. Asynchronous requires both source and destination databases to be Oracle 10g.

Streams Features

• Distinct capture/propagation/apply services
• Message Queueing (via AQ)
• Publish Subscribe
• Rules based Capture/Transform/Apply
• Database integration
• Easy configuration
• Flexible
• Inline transformations

Streams Overview

There are three basic tasks of a stream

• Capture
• Staging /Propagation
• Apply / Consumption

Capture Process:

• A capture process captures a database change(DML or DDL changes from Redo logs) and converts it into a specific message format called a logical change record (LCR) or user messages.
• After capturing an LCR, a capture process enqueues a message containing the LCR into a queue(local or remote).
• Captured LCRs can be sent to queues in the same database or other databases by propagations.
• A capture process running on source database is a local capture process while if it runs on a remote database, the capture process is a downstream capture process. (Check Local and Downstream Capture for Details

Further Readings:  Local and DownStream Capture(10g) , Summary of Information Capture Information in Oracle 11g

Staging Process:

• Oracle Streams uses queues to stage messages. Staged messages can be consumed or propagated, or both.
• Staged messages can be consumed by an apply process, a messaging client, or a user application.
• A running apply process implicitly dequeues messages, but messaging clients and user applications explicitly dequeue messages.

Further Readings: Message Propagation between Queues,

Propagation Process:

• Oracle Streams uses  Oracle scheduler jobs to propagate messages between two queues.
• A propagation is always between a source queue and a destination queue.
• Only one propagation is allowed between a particular source queue and a particular destination queue.
• A single source queue can propagate messages to multiple destination queues, and a single destination queue can receive messages from multiple source queues. A single propagation can propagate LCRs and user messages.
• A propagation either propagates or discards messages based on rules defined.
• A propagation can be queue-to-queue or queue-to-database link. Propagation to the remote queue uses database links over Oracle Net Services.

Apply/Consumption Process:

• An apply process dequeues messages (LCR or user messages) from a specific queue. 
• An apply process either applies each message directly or passes it as a parameter to an apply handler. 
• Auser-enqueued message dequeued by an apply process is of type ANYDATA and can contain any message, including an LCR or a user message.
• An apply process can only dequeue messages from an ANYDATA queue, not a typed queue.

Further Readings: Message Processing with Apply Process

Streams Process Architecture

In simple five steps the whole Streams process is as follows:

• Capture changes at a database(source/target)
• Enqueue events in a queue
• Propagate events from one queue to another
• Dequeue events
• Apply events at a database

Streams Rules

• Rule is a database object that enables a client to perform an action when an event occurs and a condition is satisfied.
• Rules are used to control which information to share and where to share it.
• Rules can be used during capture, propagate and apply process.
• Rule consists of Rule condition and optionally evaluation context and Action extent . It can be defined in three levels:
1. Table
2. Schema
3. Global

Database Configuration

Following parameters should be set at each site:

• COMPATIBLE
• PROCESSES
• SHARED_POOL_SIZE
• STREAMS_POOL_SIZE
• TIMED_STATISTICS

DB parameters that must be set at source site:

• LOG_ARCHIVE_DEST_n
• LOG_ARCHIVE_DEST_STATE_n
• REMOTE_ARCHIVE_ENABLE
• UNDO_RETENTION

DB parameters specific to propagation:

• GLOBAL_NAMES
• JOB_QUEUE_PROCESSES

Databases capturing changes via Streams must be in archived mode whereas databases applying changes are not required in archived mode.

Oracle Streams Administration Tools

Oracle-Supplied PL/SQL Packages

DBMS_STREAMS_ADM Package

DBMS_CAPTURE_ADM Package

DBMS_PROPAGATION_ADM Package

DBMS_APPLY_ADM Package

DBMS_STREAMS_MESSAGING Package

DBMS_RULE_ADM Package

DBMS_RULE Package

DBMS_STREAMS Package

DBMS_STREAMS_TABLESPACE_ADM

DBMS_STREAMS_AUTH Package

Streams Data Dictionary Views

Streams Static Data Dictionary Views

• DBA_APPLY
• DBA_CAPTURE
• DBA_RULES
• DBA_STREAMS_RULES
• DBA_STREAMS_SCHEMA_RULES
• DBA_STREAMS_TABLE_RULES

Streams Dynamic Performance Views

• V$BUFFERED_QUEUES

• V$BUFFERED_SUBSCRIBERS

• V$PROPAGATION_RECEIVER

• V$PROPAGATION_SENDER

• V$RULE

• V$RULE_SET

• V$STREAMS_CAPTURE

• V$STREAMS_POOL_ADVICE

• V$STREAMS_TRANSACTION

Oracle Enterprise Manager Console

Oracle provides a Streams tool in the Oracle Enterprise Manager Console to help configure , administer and monitor Streams environments.

There are different states in which database can start up. The following scenarios describe different stages of starting up an instance starting with shutdown.

SHUTDOWN

NOMOUNT

MOUNT

OPEN

Starting the Instance (NOMOUNT)

An instance is started in the NOMOUNT stage only during database creation or the re-creation
of control files.
Starting an instance includes the following tasks:

• Reading the initialization file from $ORACLE_HOME/dbs in the following order:
        – First spfileSID.ora
        – If not found then, spfile.ora
        – If not found then, initSID.ora
• Allocating the SGA
• Starting the background processes
• Opening the alertSID.log file and the trace files

STARTUP NOMOUNT

Specifying the PFILE parameter with STARTUP overrides the default behavior

STARTUP NOMOUNT PFILE = /u01/oracle/dbs/init.ora

Mounting the Database (MOUNT)

To perform specific maintenance operations, you start an instance and mount a database but
do not open the database.

For example, the database must be mounted but not open during the following tasks:
• Renaming data files (data files for an offline tablespace can be renamed when the
database is open)
• Enabling and disabling online redo log file archiving options
• Performing full database recovery
Mounting a database includes the following tasks:
• Associating a database with a previously started instance
• Locating and opening the control files specified in the parameter file
• Reading the control files to obtain the names and status of the data files and online
redo log files. However, no checks are performed to verify the existence of the data
files and online redo log files at this time.

STARTUP MOUNT

Opening the Database (OPEN)

Normal database operation means that an instance is started and the database is mounted and
open. Opening the database includes the following tasks:

• Opening the online data files
• Opening the online redo log files

If any of the data files or online redo log files are not present when you attempt to open the
database, the Oracle server returns an error. Oracle server verifies that all the data files and online redo log
files can be opened and checks the consistency of the database. If necessary, the SMON
background process initiates instance recovery.

STARTUP

Restricting Access to an Instance at Startup

You can start an instance, and optionally mount and open a database, in restricted mode so that the instance is available only to administrative personnel (not general database users). Use this mode of instance startup when you need to accomplish one of the following tasks:

• Perform an export or import of data

• Perform a data load (with SQL*Loader)

• Temporarily prevent typical users from using data

• Perform certain migration or upgrade operations

STARTUP RESTRICT

Forcing Instance to Start / Instance with Media Recovery

         You cannot shut down the current instance with the SHUTDOWN NORMAL, SHUTDOWN IMMEDIATE, or SHUTDOWN TRANSACTIONAL commands. Use the STARTUP command with the FORCE clause:

STARTUP FORCE

If media recovery is required, start by using the STARTUP command with the RECOVER clause:

STARTUP OPEN RECOVER

Altering Database Availability

You can alter the availability of a database. You may want to do this in order to restrict access for maintenance reasons or to make the database read only. The following sections explain how to alter the availability of a database:

Database can be altered to mount state from nomount state but cannot be altered directly to open state.

ALTER DATABASE MOUNT;

To open a mounted database, use the ALTER DATABASE statement with the OPEN clause:

ALTER DATABASE OPEN;

Opening a database in read-only mode enables you to query an open database while eliminating any potential for online data content changes.

ALTER DATABASE OPEN READ ONLY;

You can also open a database in read/write mode as follows which is the default mode:

ALTER DATABASE OPEN READ WRITE;

set lines 100
col name format a60
select	name
,	floor(space_limit / 1024 / 1024) "Size MB"
,	ceil(space_used  / 1024 / 1024) "Used MB"
from	v$recovery_file_dest
order by name
/

To fix the problem, you need to either make the flash recovery area larger, or remove some files from it.

If you have the disk space available, make the recovery area larger:

alter system set db_recovery_file_dest_size=<size> scope=both
/

To remove files you must use RMAN. Manually moving or deleting files will have no effect as oracle will be unaware. The obvious choice is to backup and remove some archive log files. However, if you usually write your RMAN backups to disk, this could prove tricky. RMAN will attempt to write the backup to the flash recovery area…which is full. You could try sending the backup elsewhere using a command such as this:

rman target / catalog user/pass@rmancat

run {
allocate channel t1 type disk;
backup archivelog all delete input format '/<temp backup location>/arch_%d_%u_%s';
release channel t1;
}

This will backup all archive log files to a location of your choice and then remove them.

http://www.shutdownabort.com/errors/ORA-19809.php

wspprd.__db_cache_size=587202560
wspprd.__java_pool_size=16777216
wspprd.__large_pool_size=16777216
wspprd.__shared_pool_size=939524096
wspprd.__streams_pool_size=33554432
*.compatible=’10.2.0.3.0′
*.db_block_size=8192
*.db_cache_size=524288000
*.db_domain=”
*.db_file_multiblock_read_count=16
*.db_files=1024
*.db_recovery_file_dest=’/wspprd/oracle/product/10.2.0/flash_recovery_area’
*.db_recovery_file_dest_size=2147483648
*.job_queue_processes=10
*.open_cursors=1600
*.pga_aggregate_target=314572800
*.pre_page_sga=TRUE
*.processes=600
*.sessions=665
*.sga_max_size=1610612736
*.sga_target=1610612736
*.shared_pool_size=536870912
*.db_writer_processes=20
*.sort_area_size=65536
*.cursor_space_for_time=TRUE
*.session_cached_cursors=500
*.log_checkpoints_to_alert=TRUE
*.log_checkpoint_interval=0
*.log_checkpoint_timeout=0
*.fast_start_mttr_target=3600
*.log_buffer=419430400

Related articles by Zemanta

• Creating Oracle Database for Websphere Portal (mohibalvi.wordpress.com)

SQL Server 2008 allows you to create four different types of backups:

• Full Backups
• Differential Backups
• Transaction log Backups
• Filegroup Backups

Full Backups

A full backup captures all pages within a database that contain data. Pages that do not

contain data are not included in the backup.

Because it is more common to back up a database than to restore one, the backup engine

is optimized for the backup process. When a backup is initiated, the backup engine grabs

pages from the data files as quickly as possible, without regard to the order of pages. Because

the backup process is not concerned with the ordering of pages, multiple threads can be used

to write pages to your backup device.

Because changes canbe made to the database while a backup is running, SQL Server needs to be able to accommodate the changes while also ensuring that backups are consistent for restore purposes. To ensure both concurrent access and backup consistency, SQL Server performs the steps of the backup procedure as follows:

1. Locks the database, blocking all transactions

2. Places a mark in the transaction log

3. Releases the database lock

4. Extracts all pages in the data fi les and writes them to the backup device

5. Locks the database, blocking all transactions

6. Places a mark in the transaction log

7. Releases the database lock

8. Extracts the portion of the log between the marks and appends it to the backup

The only operations that are not allowed during a full backup are

• Adding or removing a database file
• Shrinking a database

The only two parameters required for a backup are the name of the database and the

backup device. When you specify a disk backup device, a directory and a fi le name can

be specifi ed. If a directory is not specifi ed, SQL Server performs a backup to disk and writes

the file to the default backup directory configured for the instance. Although most backups

are written to a single disk file or a single tape device, you can specify up to 64 backup

devices. When you specify more than one backup device, SQL Server stripes the backup

across each of the devices specified.

Related articles by Zemanta

• SQL Server 2008 Backup (mohibalvi.wordpress.com)
• Get the SQL Server 2008 R2 August Community Technology Preview (msdn.microsoft.com)

Use -x option to debug a shell script

Run a shell script with -x option. This is valid for bash shell scripts as well.

$ ksh -x tesh.sh
$ ksh -v test.sh

Use of set builtin command

korn shell offers debugging options which can be turn on or off using set command.

=> set -x : Display commands and their arguments as they are executed.

=> set -v : Display shell input lines as they are read.

You can use above two command in shell script itself:

<span style="color: rgb(0, 0, 153); font-weight: bold;">#!/bin/ksh</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">clear</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;"># turn on debug mode</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">set -x</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">for f in *</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">do</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">   file $f</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">done</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;"># turn OFF debug mode</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">set +x</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">ls</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;"># more commands</span>

You can replace standard

#!/bin/ksh

with (for debugging)

#!/bin/ksh -xv

Use of intelligent DEBUG function

Add special variable _DEBUG. Set to `on’ when you need to debug a script:

_DEBUG="on"

Put the following function at the beginning of the script:

function DEBUG()<br />{<br /> [ "$_DEBUG" == "on" ] &&  $@ || :<br />}

Now wherever you need debugging simply use DEBUG function

DEBUG echo "File is $filename"

OR

DEBUG set -x

Cmd1

Cmd2

DEBUG set +x

When debugging done and before moving a script to production set _DEBUG to off

No need to delete debug lines.

_DEBUG="off" # set to anything but not to 'on'

Sample script:

<span style="color: rgb(0, 0, 153); font-weight: bold;">#!/bin/bash</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">_DEBUG="on"</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">function DEBUG()</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">{</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;"> [ "$_DEBUG" == "on" ] &&  $@ || :</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">}</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">DEBUG echo 'Reading files'</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">for i in *</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">do</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">  grep 'something' $i > /dev/null</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">  [ $? -eq 0 ] && echo "Found in $i file" || :</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">done</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">DEBUG set -x</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">a=2</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">b=3</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">c=$(( $a + $b ))</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">DEBUG set +x</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">echo "$a + $b = $c"</span><br />

Save and run the script:

$ ./script.sh

Output:

<span style="color: rgb(0, 0, 153); font-weight: bold;">Reading files</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">Found in xyz.txt file</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">+ a=2</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">+ b=3</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">+ c=5</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">+ DEBUG set +x</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">+ '[' on == on ']'</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">+ set +x</span><br style="color: rgb(0, 0, 153); font-weight: bold;" /><span style="color: rgb(0, 0, 153); font-weight: bold;">2 + 3 = 5</span><br />

Now set DEBUG to off

_DEBUG="off"

Run script:

$ ./script.sh

Output:

<span style="font-weight: bold; color: rgb(0, 0, 153);">Found in xyz.txt file</span><br style="font-weight: bold; color: rgb(0, 0, 153);" /><span style="font-weight: bold; color: rgb(0, 0, 153);">2 + 3 = 5</span><br />

You can also try to use DEBUG as an alias instead of function.

Links

Korn Shell Debugger by OReilly

Bash Debugger Project

Powered by ScribeFire.

Join Types by Sachin

Previously, i copied most of the important points from his post for my own request but now deleting as the author does not like it.

Related articles by Zemanta

• Optimization of Joins
  

</pre>1) SELECT distinct D.deptno, D.dname
    FROM dept D, emp E
        WHERE E.deptno = D.deptno
        ORDER BY D.deptno;

2) SELECT D.deptno, D.dname
    FROM dept D
        WHERE EXISTS
                 (
                 SELECT 1
                 FROM emp E
                 WHERE E.deptno = D.deptno
                 )
        ORDER BY D.deptno;

<span style="color: rgb(0, 0, 0); font-family: verdana;"><span>In general, you should phrase the queries in the manner  that says it best. If one set of joins were particularly efficient to use in all  cases, Oracle would not have implemented the rest of them! </span></span><br style="color: rgb(0, 0, 0); font-family: verdana;" /><br style="color: rgb(0, 0, 0); font-family: verdana;" /><span style="color: rgb(0, 0, 0); font-family: verdana;"><span>In general, you use a join when you need data from more  than one table in the ultimate SELECT list. Here you need data only from DEPT,  so it is unlikely that you would consider a join. You would instead consider  using either WHERE EXISTS or WHERE IN. (The cost-based optimizer [CBO] sees them  as more or less equivalent—you might, too.)</span></span><br style="color: rgb(0, 0, 0); font-family: verdana;" />
<span>I would write the query as either</span>

<pre>select deptno, dname
  from dept
 where exists
   ( select NULL
       from emp
      where emp.deptno
         = dept.deptno )
 order by deptno;

or

select deptno, dname<br />  from dept<br /> where deptno in <br />( select deptno <br />    from emp )<br /> order by deptno;

In both cases, the optimizer would employ either a semi join, which you cannot specify but the optimizer may perform (it stops joining after the first hit), or an index probe using NESTED LOOPS into EMP to test for row existence.

This example demonstrates, in general, what happens. When the optimizer detects that DEPT is small and EMP is large, it index-probes the large EMP table for each row in DEPT. On the other hand, if it deems DEPT to be large, the optimizer will do a pure semi join in bulk, with no indexes. I’m using a copy of the EMP and DEPT tables for the test:

SQL> create table emp as<br />  2  select * from scott.emp;<br />Table created.<br /><br />SQL> create table dept as<br />  2  select * from scott.dept;<br />Table created.<br /><br />SQL> create index emp_deptno_idx<br />  2  on emp(deptno);<br />Index created.

Then I set the first representative statistics. I use SET_TABLE_STATS to make EMP appear large, with 1,000,000 rows, and DEPT appear small, with only 100 rows. Additionally, I tell the optimizer about the index I envision having on EMP(DEPTNO). Because Oracle Database 10g Release 2 computes statistics on an index creation by default, I start by removing any existing statistics on this index and putting in place statistics that might be more representative:

 <br />SQL> begin<br />  2   dbms_stats.set_table_stats<br />  3   ( user,<br />  4    'EMP',<br />  5    numrows => 1000000,<br />  6    numblks => 100000 );<br />  7   dbms_stats.delete_index_stats<br />  8   ( user,<br />  9    'EMP_DEPTNO_IDX' );<br /> 10   dbms_stats.set_index_stats<br /> 11   ( user,<br /> 12    'EMP_DEPTNO_IDX',<br /> 13     numrows => 1000000,<br /> 14     numdist => 10000,<br /> 15     numlblks =>10000 );<br /> 16   dbms_stats.set_column_stats<br /> 17   ( user,<br /> 18    'EMP',<br /> 19    'DEPTNO',<br /> 20     DISTCNT => 10000 );<br /> 21   dbms_stats.set_table_stats<br /> 22   ( user,<br /> 23    'DEPT',<br /> 24     numrows=> 100,<br /> 25     numblks => 100 );<br /> 26  end;<br /> 27  /

When using the CBO with a large EMP and a small DEPT, Oracle Database might use a plan such as the one in Listing 1. The optimizer opts to read every row in DEPT and then performs an index range scan—the index probe into the EMP table—to see if the row exists in the other table. Because I have relatively few index probes to perform (about 100 is what the optimizer thinks) and because the EMP table is large, the optimizer uses this plan in both the WHERE IN and WHERE EXISTS cases.

Code Listing 1: CBO plan with large EMP and small DEPT

SQL> set autotrace traceonly explain<br />SQL> select deptno, dname<br />  2    from dept where deptno  in<br />  3    ( select deptno<br />  4        from emp )<br />  5   order by deptno;<br /><br />Execution Plan<br />-----------------------------------------------------------<br />Plan hash value: 3383088615<br /><br />-----------------------------------------------------------<br />|  Id |	Operation	| Name	        |   Rows | Bytes  |<br />-----------------------------------------------------------<br />|   0 |SELECT STATEMENT	|		|   100	 |  3500  |<br />|   1 |SORT ORDER BY	|		|   100	 |  3500  |<br />|   2 |NESTED LOOPS SEMI|		|   100	 |  3500  | <br />|   3 |TABLE ACCESS FULL| DEPT	        |   100	 |  2200  | <br />|*  4 |INDEX RANGE SCAN	| EMP_DEPTNO_IDX|  1000K |    12M |<br />-----------------------------------------------------------<br /><br />Predicate Information (identified by operation id):<br />-----------------------------------------------------------<br /><br />   4 - access("DEPTNO"="DEPTNO")<br /><br />SQL> select deptno, dname<br />  2    from dept where exists<br />  3    ( select null<br />  4        from emp<br />  5       where emp.deptno =<br />  6            dept.deptno )<br />  7   order by deptno;<br /><br />Execution Plan<br />------------------------------------------------------------<br />Plan hash value: 3383088615<br /><br />------------------------------------------------------------<br />|  Id |	Operation	 | Name	         |   Rows | Bytes  |<br />------------------------------------------------------------<br />|   0 |	SELECT STATEMENT |	         |   100  |  3500  |<br />|   1 | SORT ORDER BY	 |		 |   100  |  3500  |<br />|   2 | NESTED LOOPS SEMI|		 |   100  |  3500  |<br />|   3 | TABLE ACCESS FULL| DEPT	         |   100  |  2200  |<br />|*  4 | INDEX RANGE SCAN | EMP_DEPTNO_IDX|  1000K |    12M |<br />------------------------------------------------------------<br /><br />Predicate Information (identified by operation id):<br />------------------------------------------------------------<br /><br />   4 - access("EMP"."DEPTNO"="DEPT"."DEPTNO")

Compare that with the RBO plan, shown in Listing 2. When using the RBO, the developer must ask, “What size will EMP and DEPT be?” and, depending on the answer, choose either WHERE IN or WHERE EXISTS to get an “optimal” plan. The RBO comes up with two different plans based solely on how the question was asked, not on how large the amount of data was. This is one of the major advantages of the CBO over the RBO: Developers don’t have to know how to write the best query in all cases—which is not to say that they don’t need to learn SQL! They just do not have to figure out minutiae such as where they should use WHERE IN versus WHERE EXISTS.

Code Listing 2: RBO plan

SQL> select /*+ RULE */<br />  2    deptno, dname<br />  3    from dept where deptno  in<br />  4    ( select deptno<br />  5        from emp )<br />  6   order by deptno;<br /><br />Execution Plan<br />-------------------------------------<br />Plan hash value: 1932208647<br /><br />-------------------------------------<br />|  Id |	Operation	 | Name	    |<br />-------------------------------------<br />|   0 | SELECT STATEMENT |	    |<br />|   1 | MERGE JOIN       |	    |<br />|   2 | SORT JOIN	 |	    |<br />|   3 | TABLE ACCESS FULL| DEPT     |<br />|*  4 | SORT JOIN	 |	    |<br />|   5 | VIEW		 | VW_NSO_1 |			<br />|   6 | SORT UNIQUE	 |          |<br />|   7 | TABLE ACCESS FULL| EMP	    |<br />-------------------------------------<br /><br />Predicate Information (identified by operation id):<br />-------------------------------------<br /><br />   4 - access("DEPTNO"="$nso_col_1")<br />       filter("DEPTNO"="$nso_col_1")<br /><br />Note<br />----------<br />   - rule based optimizer used (consider using cbo)<br /><br />SQL> select /*+ RULE */<br />  2    deptno, dname<br />  3    from dept where exists<br />  4    ( select null<br />  5        from emp<br />  6       where emp.deptno =<br />  7            dept.deptno )<br />  8   order by deptno;<br /><br />Execution Plan<br />----------------------------------------<br />Plan hash value: 4109416194<br /><br />----------------------------------------<br />| Id | Operation	| Name	       |<br />----------------------------------------<br />|  0 | SELECT STATEMENT	|	       |<br />|  1 | SORT ORDER BY	|	       |<br />|* 2 | FILTER		|	       |<br />|  3 | TABLE ACCESS FULL| DEPT 	       |<br />|* 4 | INDEX RANGE SCAN	| EMP_DEPTNO_ID|<br />----------------------------------------<br /><br />Predicate Information (identified by operation id):<br />----------------------------------------<br /><br />   2 - filter( EXISTS (SELECT 0 FROM "EMP" "EMP" WHERE<br />                "EMP"."DEPTNO"=:B1))<br />   4 - access("EMP"."DEPTNO"=:B1)

Now, back to the CBO. As the amount of data in DEPT increases—and the optimizer decides that it will have to perform significantly more index probes into the EMP table—the query plans change. They go from using NESTED LOOPS and INDEX RANGE SCANS to more-efficient bulk operations. To see this, I tell the optimizer that DEPT is much larger now

SQL> begin<br />  2     dbms_stats.set_table_stats<br />  3     ( user,<br />  4      'DEPT',<br />  5       numrows=> 100000,<br />  6       numblks => 10000 );<br />  7  end;<br />  8  /

and then rerun the queries and review the resulting plans, as shown in Listing 3. As you can see, the optimizer, in both cases, now chooses to perform full scans and a nice big hash semi join—a join that stops after the first match—instead of using an index probe, because it would have to use the index 100,000 times.

Code Listing 3: CBO plan with larger DEPT

SQL> set autotrace traceonly explain<br />SQL> select deptno, dname<br />  2    from dept where deptno  in<br />  3    ( select deptno<br />  4        from emp )<br />  5   order by deptno;<br /><br />Execution Plan<br />-----------------------------------------------------------------<br />Plan hash value: 3127359958<br /><br />-----------------------------------------------------------------<br />| Id |	Operation	    | Name	     |	 Rows |  Bytes	|	<br />-----------------------------------------------------------------<br />|  0 |	SELECT STATEMENT    |		     |   100K |  3417K	|	<br />|  1 |  SORT ORDER BY	    |		     |   100K |  3417K	|<br />|* 2 |  HASH JOIN SEMI	    |		     |   100K |  3417K	|<br />|  3 |  TABLE ACCESS FULL   | DEPT	     |   100K |  2148K	|	<br />|  4 |  INDEX FAST FULL SCAN| EMP_DEPTNO_IDX |   1000K|    12M	|<br />-----------------------------------------------------------------<br />Predicate Information (identified by operation id):<br />-----------------------------------------------------------------<br /><br />   2 - access("DEPTNO"="DEPTNO")<br /><br />SQL> select deptno, dname<br />  2    from dept where exists<br />  3    ( select null<br />  4        from emp<br />  5       where emp.deptno =<br />  6            dept.deptno )<br />  7   order by deptno;<br /><br />Execution Plan<br />---------------------------------------------------------------<br />Plan hash value: 3127359958<br /><br />---------------------------------------------------------------<br />| Id |	Operation	    | Name	    |	Rows |   Bytes|	<br />---------------------------------------------------------------<br />|  0 |	SELECT STATEMENT    |		    |   100K |  3417K |	<br />|  1 |  SORT ORDER BY	    |		    |   100K |  3417K | <br />|* 2 |  HASH JOIN SEMI	    |		    |   100K |  3417K | <br />|  3 |  TABLE ACCESS FULL   | DEPT	    |   100K |  2148K |	<br />|  4 |  INDEX FAST FULL SCAN| EMP_DEPTNO_IDX|   1000K|    12M |	<br />---------------------------------------------------------------<br /><br />Predicate Information (identified by operation id):<br />---------------------------------------------------------------<br /><br />   2 - access("EMP"."DEPTNO"="DEPT"."DEPTNO")

Another important thing to note about these examples is that the RBO never performed a semi join, as the CBO did. The RBO has a very limited set of access paths and join operations available—it simply doesn’t know how to do a semi join and won’t use that join operation. Instead it performs operations such as the SORT DISTINCT it did in Listing 2, which is much less efficient than the cost-based plan in this case. Another reason to abandon the RBO in favor of the CBO!

Why Does My Plan Change?
I have noticed that gathering statistics against tables where none of the data has changed can cause the query plans against those tables to change. For example, one day I gather statistics, run some queries, and save the query plans in a report. The next day, I gather statistics (but I have not modified any of the data in any table) and I discover that the plans are different. What could cause that? The new query plans were better, but I would like to understand how this could be.

This change is likely due to the default value of the METHOD_OPT parameter used with DBMS_STATS. In Oracle Database 10g, the METHOD_OPT parameter defaults to a SIZE AUTO. After you ran a query, the database remembered the predicates and updated a dictionary table, SYS.COL_USAGE$. Then, the next time you ran DBMS_STATS to gather statistics on these tables, DBMS_STATS queried that table to find out what columns should have histograms collected automatically, based on past query workload. It looked at your predicates and said, “Hmm, these columns are candidates for histograms based on the queries the end users have been running.”

You can see this easily with a relatively small example. I start out by creating a small table with some skewed data in a column named ID and gathering the default set of statistics on it. The data in the ID column is such that for the values 0 through 4, about 20 percent of the table will be retrieved, but for the value 99, only a single row will be returned. I’m using this exaggerated example just to see the plans change:

SQL> create table t<br />  2  as<br />  3  select mod(rownum,5) id, a.*<br />  4    from all_objects a;<br />Table created.<br />SQL> update t<br />  2     set id = 99<br />  3   where rownum = 1;<br />1 row updated.<br /><br />SQL> create index t_idx on t(id);<br />Index created.<br /><br />SQL> begin<br />  2   dbms_stats.gather_table_stats<br />  3   ( user, 'T' );<br />  4  end;<br />  5  /<br /><br />SQL> select column_name, count(*)<br />  2    from user_tab_histograms<br />  3   where table_name = 'T'<br />  4     and column_name = 'ID'<br />  5   group by column_name;<br /><br />COLUMN_NAME   COUNT(*)<br />-----------   ------------<br />ID                  2

So, right now, the ID column doesn’t have complete histograms, by default—just two buckets, as shown by the query against USER_TAB_HISTOGRAMS. To understand the skewed nature of the data, I need a histogram with more than just two buckets; the two entries in USER_TAB_HISTOGRAMS tell the optimizer only the high and low values right now.

The optimizer knows the high value (99), the low value (0), the number of distinct values (6 in this case), and the number of rows in my table T (50,119 when I tested). Given those facts, the optimizer will believe that WHERE ID = will return about 50,119/6, or 8,353 rows. Sure enough, when I query with ID=1 or ID=99, I observe the results in Listing 4.

Code Listing 4: CBO plans when ID=1 and ID=99

SQL> set autotrace traceonly explain<br />SQL> select *<br />  2    from t<br />  3   where id = 1;<br /><br />----------------------------------------------------------------------------<br />|  Id |	Operation	 | Name	|  Rows	|  Bytes|   Cost (%CPU)	|  Time	   |<br />----------------------------------------------------------------------------<br />|   0 |	SELECT STATEMENT |	|  8353	|   783K|   163	(2)	| 00:00:02 |<br />|*  1 | TABLE ACCESS FULL| T	|  8353	|   783K|   163	(2)	| 00:00:02 |<br />----------------------------------------------------------------------------<br /><br />SQL> select *<br />  2    from t<br />  3   where id = 99;<br /><br />Execution Plan<br />----------------------------------------------------------------------------<br />Plan hash value: 1601196873<br /><br />----------------------------------------------------------------------------<br />|  Id |	Operation	 | Name	|   Rows |  Bytes|  Cost (%CPU)	| Time	   |<br />----------------------------------------------------------------------------<br />|   0 |	SELECT STATEMENT |	|   8353 |   783K|   163 (2)	| 00:00:02 |<br />|*  1 | TABLE ACCESS FULL| T	|   8353 |   783K|   163 (2)	| 00:00:02 |<br />----------------------------------------------------------------------------

Now, I immediately gather statistics, using the same command as before:

SQL> begin<br />  2   dbms_stats.gather_table_stats<br />  3   ( user, 'T' );<br />  4  end;<br />  5  /<br />SQL> select column_name, count(*)<br />  2    from user_tab_histograms<br />  3   where table_name = 'T'<br />  4     and column_name = 'ID'<br />  5   group by column_name;<br /><br />COLUMN_NAME   COUNT(*)<br />------------  -----------<br />ID                  5

Note, however, that I have more than two buckets for my histograms. DBMS_STATS, using the AUTO setting, gathered more information here. If I were to query the SYS.COL_USAGE$ table right now, I would discover that a new row was added, indicating that I have some queries in my system that use equality predicates against this particular database column. That is the “magic” that caused DBMS_STATS to change how it gathered statistics and caused a dramatic change in my resulting query plans, as shown in Listing 5.

Code Listing 5: New CBO plans when ID=1 and ID=99

  2    from t<br />  3   where id = 1;<br /><br />Execution Plan<br />----------------------------------------------------------------------------<br />Plan hash value: 1601196873<br /><br />----------------------------------------------------------------------------<br />| Id |	Operation	 | Name	|  Rows	|  Bytes|  Cost (%CPU)	| Time	   |<br />----------------------------------------------------------------------------<br />|  0 |	SELECT STATEMENT |	|  10260|   961K|   164	(2)	| 00:00:02 |<br />|* 1 |  TABLE ACCESS FULL| T	|  10260|   961K|   164	(2)	| 00:00:02 |<br />----------------------------------------------------------------------------<br /><br />SQL> select *<br />  2    from t<br />  3   where id = 99;<br /><br />Execution Plan<br />----------------------------------------------------------------------------------------<br />Plan hash value: 470836197<br /><br />----------------------------------------------------------------------------------------<br />| Id |  Operation	           | Name    | Rows |  Bytes|  Cost (%CPU)  |  Time    |<br />----------------------------------------------------------------------------------------<br />|  0 |  SELECT STATEMENT	   |	     | 1    |    96	|   2	(0) | 00:00:01 |<br />|  1 |  TABLE ACCESS BY INDEX ROWID|	T    | 1    |    96	|   2	(0) | 00:00:01 |<br />|* 2 |  INDEX RANGE SCAN	   |	T_IDX| 1    |   	|   1	(0) | 00:00:01 |<br />----------------------------------------------------------------------------------------

Note how the cardinalities are very different—8,353 changed to either 10,260 or 1. The resulting change in estimated cardinality led to a change in the overall cost of the first query plan and resulted in an entirely different (but better) query plan in the second case.

This is a significant fact for a DBA to be aware of. As the queries submitted by your end users change over time, the AUTO feature of statistics gathering could change what statistics are gathered over time as well. Unless you understand this feature, enabled by default in Oracle Database 10g, it can look as if inexplicable magic is happening in your database.

Powered by ScribeFire.

Database Character Set Statement of Direction

A list of character sets has been compiled in Table A-4, "Recommended ASCII Database Character Sets" and Table A-5, "Recommended EBCDIC Database Character Sets" that Oracle Corporation strongly recommends for usage as the database character set. Other Oracle-supported character sets that do not appear on this list can continue to be used in Oracle Database 10g Release 2, but may be desupported in a future release. Starting with the next major functional release after Oracle Database 10g Release 2, the choice for the database character set will be limited to this list of recommended character sets for new system deployment. Customers will still be able to migrate their existing databases in the next major functional release after Oracle Database 10g Release 2 even if the character set is not on the recommended list. However, Oracle suggests that customers migrate to a recommended character set as soon as possible. At the top of the list of character sets Oracle recommends for all new system deployment is the Unicode character set AL32UTF8.

Choosing Unicode as a Database Character Set

Oracle Corporation recommends using Unicode for all new system deployments. Migrating legacy systems eventually to Unicode is also recommended. Deploying your systems today in Unicode offers many advantages in usability, compatibility, and extensibility. Oracle Database’s comprehensive support enables you to deploy high-performing systems faster and more easily while utilizing the advantages of Unicode. Even if you do not need to support multilingual data today or have any requirement for Unicode, it is still likely to be the best choice for a new system in the long run and will ultimately save you time and money as well as give you competitive advantages. See Chapter 6, "Supporting Multilingual Databases with Unicode" for more information about Unicode.

Choosing a National Character Set

A national character set is an alternate character set that enables you to store Unicode character data in a database that does not have a Unicode database character set. Other reasons for choosing a national character set are:

• The properties of a different character encoding scheme may be more desirable for extensive character processing operations.

• Programming in the national character set is easier.

SQL NCHAR, NVARCHAR2, and NCLOB datatypes have been redefined to support Unicode data only. You can use either the UTF8 or the AL 16UTF16 character set. The default is AL16UTF16.

Character Sets

http://download-west.oracle.com/docs/cd/B19306_01/server.102/b14225/applocaledata.htm#i635016

Oracle-supported character sets are listed in the following sections according to three broad categories.

• Recommended Database Character Sets

• Other Character Sets

• Client-Only Character Sets

In addition, common character set subset/superset combinations are listed. Some character sets can only be used with certain data types. For example, the AL16UTF16 character set can only be used as an NCHAR character set, and not as a database character set.

Can You use AL16UTF16 as NLS_CHARACTERSET?

No, AL16UTF16 can only be used as NLS_NCHAR_CHARACTERSET in 9i and above. Trying to create a database with  a AL16UTF16 NLS_CHARACTERSET will fail.

(Source of this answer is REPETTAS WORDPRESS BLOG )

1) AL24UTFFSS:

This character set was the first Unicode character set supported by Oracle. The AL24UTFFSS encoding scheme was based on the Unicode 1.1 standard, which is now obsolete. This unicode character set was used between oracle version 7.2 to 8.1.

2) UTF-8:

UTF8 was the UTF-8 encoded character set in Oracle8 and 8i. It followed the
Unicode 2.1 standard between Oracle 8.0 and 8.1.6, and was upgraded to Unicode
version 3.0 for oracle versions 8.1.7, 9i, 10g and 11g. If supplementary characters are inserted into in a UTF8 database encoded with Unicode version 3.0, then the actual data will be treated as 2 separate undefined characters, occupying 6 bytes in storage. So for fully support of supplementary characters use AL32UTF8 character set instead of UTF8.

3) UTFE:

UTFE has the same properties as UTF8 on ASCII based platforms. As of UTF8 it is used in different oracle versions.

4) AL32UTF8:

This is the UTF-8 encoded character set introduced in Oracle9i.
In Oracle 9.2 AL32UTF8 implemented unicode 3.1,
in 10.1 it implemented the Unicode 3.2 standard,
in Oracle 10.2 it supports the Unicode 4.01 standard and
in Oracle 11.1 it supports the Unicode 5.0.
AL32UTF8 was introduced to provide support for the newly defined supplementary characters. All supplementary characters are stored as 4 bytes in AL32UTF8. As while designed UTF-8 there was no concept of supplementary characters therefore UTF8 has a maximum of 3 bytes per character.

5) AL16UTF16: This is the first UTF-16 encoded character set in Oracle. It was introduced in Oracle9i as the default national character set (NLS_NCHAR_CHARACTERSET). It also provides support for the newly defined supplementary characters. All supplementary characters are stored as 4 bytes.
As with AL32UTF8, the plan is to keep enhancing AL16UTF16 as
necessary to support future version of the Unicode standard.
AL16UTF16 cannot be used as a database character set (NLS_CHARACTERSET), it is only used as the national character set (NLS_NCHAR_CHARACTERSET).
Like, AL32UTF8 In Oracle 9.0 AL16UTF16 implemented unicode 3.0,
in Oracle 9.2 it implemented unicode 3.1,
in 10.1 it implemented the Unicode 3.2 standard,
in Oracle 10.2 it supports the Unicode 4.01 standard and
in Oracle 11.1 it supports the Unicode 5.0.

View some important considerations before setting up Oracle databases to work with WebSphere Portal.

For information about creating databases, refer to the Oracle product documentation. For information on the recommended database architecture and the databases you will need to create, see the Planning for Oracle topic. Be sure that all databases to be used with WebSphere Portal are created as UNICODE character set databases.

If you are using remote Oracle databases, you must also copy the ojdbc14.jar file from the remote Oracle server to the WebSphere Portal machine. The typical location is the oracle_home/jdbc/lib directory. Record the copy location on your local machine for future reference.

When creating Oracle databases for use with WebSphere Portal, you should consider the following information:

• The Oracle databases must be created manually before configuring WebSphere Portal.
• All databases must be created using UNICODE Database and National character sets such as UTF8, AL32UTF8, or AL16UTF16.
• It is recommended that all databases to be used with WebSphere Portal are configured in Dedicated Server Mode.
• Determine if your Oracle server will be remote or local to the WebSphere Portal installation.
• If using an earlier version of Oracle (9i), ensure that Oracle JVM is also installed.
• After installing the database software for WebSphere Portal, you will need to set the buffer pools allocated to the Oracle database in order for WebSphere Portal to communicate with the Java Content Repository database. Use the following recommended values as a guide. Refer to Oracle product documentation for information on how to set the buffer pools. Recommended initial buffer pool sizes:
  

  db_block_size = 8192
  db_cache_size = 300M
  db_files = 1024
  log_buffer = 65536
  open_cursors = 1500
  pga_aggregate_target = 200M
  pre_page_sga = true
  processes = 300
  shared_pool_size = 200M