Tuesday, March 20, 2012

ORA-02002: error while writing to audit trail datafile not extending

Problem Description
The audit table AUD$ has been moved to a dedicated tablespace using package DBMS_AUDIT_MGMT (or otherwise), and the datafiles associated with this tablespace are set to autoextend. Database recyclebin parameter has been set to ON, audit_trail parameter has been set to db,extended. However when AUD$ grows and fills up the tablepace the datafile(s) do not autoextend and the following error is reported:

ORA-02002: error while writing to audit trail datafile not extending.
Typically this error appears within a larger errorstack showing recursive operations:
Errors in file /u01/app/oracle/diag/rdbms/bdafisdc/bdafisdc1/trace/bdafisdc1_j000_13690.trc:
ORA-12012: error on auto execute of job 11
ORA-02002: error while writing to audit trail
ORA-00604: error occurred at recursive SQL level 5
ORA-08243: recursive audit operation attempted
ORA-08243: recursive audit operation attempted
ORA-06512: at "FMS_USER.SPREGISTERJOBS", line 9
ORA-06512: at "FMS_USER.SPREGISTERJOBS", line 92
ORA-06512: at line 1
Errors in file /u01/app/oracle/diag/rdbms/bdafisdc/bdafisdc1/trace/bdafisdc1_j000_13690.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-02002: error while writing to audit trail
ORA-00604: error occurred at recursive SQL level 4
While connecting to the database using sqlplus the following errors occur:
ERROR:
ORA-00604: error occurred at recursive SQL level 3
ORA-08243: recursive audit operation attempted
ORA-08243: recursive audit operation attempted
ORA-02002: error while writing to audit trail
ORA-00604: error occurred at recursive SQL level 3
ORA-08243: recursive audit operation attempted
ORA-08243: recursive audit operation attempted

Cause of the Problem
This issue was reported to development in Bug 11742155 - ORA-2002 and ORA-08243 when the audit tablespace is full.

The errors are caused by the fact that we are purging the recyclebin while writing the audit record. This automatic purge is triggered by the fact that there is no free space in the tablespace. Oracle Development concluded that there is no way to avoid this automatic purge operation.

Solution of the Problem
1.) Make sure that at any time there is sufficient free space in the tablespace of the AUD$ table. Tablespace utilization can be monitored with Enterprise Manager or by running the following query:

select sum(bytes)/1024/1024 from dba_free_space where tablespace_name ='NEW_TABLESPACE';

2.) Move the AUD$ table into a dictionary-managed tablespace. In this case the errors are not seen but consider carefully the potential drawbacks of using dictionary managed tablespaces.

3.) Allocate new extents for AUD$ manually. When doing this the datafile(s) of the tablespace are extended without issues:

alter table sys.aud$ allocate extent;

4) Increase the size of the database manually.

alter database datafile 6 resize 30G;
To know more information about resizing datafile have a look at http://arjudba.blogspot.com/2008/08/how-to-resize-datafile.html.

Sunday, March 18, 2012

ORA-00600: internal error code, arguments: [kews_event_end - event not active]

Problem Description

Errors in file /u01/app/oracle/diag/rdbms/bdafisdc/bdafisdc1/trace/bdafisdc1_ora_11674.trc (incident=261545):
ORA-00600: internal error code, arguments: [kews_event_end - event not active], [], [], [], [], [], [], [], [], [], [], []
ORA-02002: error while writing to audit trail
ORA-00604: error occurred at recursive SQL level 2
ORA-01003: no statement parsed
ORA-06512: at "LBACSYS.LBAC_SYSDBA", line 111
ORA-12432: LBAC error: zllicns:OCIStmtExecute
ORA-06512: at "LBACSYS.SA_SYSDBA", line 44
ORA-06512: at line 2
Incident details in: /u01/app/oracle/diag/rdbms/bdafisdc/bdafisdc1/incident/incdir_261545/bdafisdc1_ora_11674_i261545.trc
Fri Mar 16 15:40:44 2012

From trace file we see following:

========= Dump for incident 261545 (ORA 600 [kews_event_end - event not active]) ========

*** 2012-03-16 15:40:21.458
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
----- Current SQL Statement for this session (sql_id=bmc7wkqsjhapk) -----
BEGIN
SA_SYSDBA.CREATE_POLICY (
policy_name => 'ACCESS_PPM_DATA'
column_name => 'OLS_COLUMN',
default_options => 'READ_CONTROL,LABEL_DEFAULT,HIDE');
END;

Call Stack
=========
skdstdst <- ksedst1 <- ksedst <- dbkedDefDump <- ksedmp <- ksfdmp <- dbgexPhaseII <- dbgexProcessError <- dbgeExecuteForError <- dbgePostErrorKGE <- 1615 <- dbkePostKGE_kgsf <- kgeadse <- kgerinv_internal <- kgerinv <- kgeasnmierr <- kews_pls_jvm_event_ <- end_i <- kkxexe <- opiexe <- kpoal8 <- opiodr <- ttcpip <- opitsk <- opiino <- opiodr <- opidrv <- sou2o <- opimai_real <- ssthrdmain

Cause of the Problem
The problem has been identified as bug 10224424.


Solution of the Problem

1) Apply the patch 10224424 , if available for your version

or

2) Apply the patch set 11.2.0.3 or higher where the issue is already fixed.