Thursday, May 7, 2009

How to cleanup orphaned datapump jobs from DBA_DATAPUMP_JOBS

In many cases you sometimes stop data pump job or in case of an abnormal end of the Data Pump job (the orphaned job) or using undocumented parameter KEEP_MASTER=Y, the master table remain in the database.

Though this topic is related to cleanup orphaned datapump jobs. But it is good to know several things before doing cleanup jobs.

1) You can check the orphaned data pump from the state column of the view dba_datapump_jobs and DBA_DATAPUMP_JOBS is based on gv$datapump_job, obj$, com$, and user$. Orphaned Data Pump jobs do not have an impact on new Data Pump jobs. If a new Data Pump job is started, a new entry will be created, which has no relation to the old Data Pump jobs.

2) For a new data pump job without any job name it is used a system generated name. From the dba_datapump_jobs it is checked for existing data pump jobs and then obtain a unique new system generated jobname.

3) Data pump jobs are different from DBMS_JOBS and they are maintained differently. Jobs created with DBMS_JOBS use there own processes. Data Pump jobs use a master process and worker process(es).

4) If you drop the master table while doing the data pump export or data pump import operation then the scenario is discussed below.

In case of export if you drop data pump export operation then export process will abort.

In case of import if you drop data pump import operation then import process will abort while it leads an incomplete import.

If the data pump job is completed and master table exist (a common if you do export operation with KEEP_MASTER=y) then it is safe to drop the master table.

Step by step cleanup orphaned datapump jobs is discussed below.

Step 01: Check the orphaned datapump jobs.

sqlplus / as sysdba
SET lines 140
COL owner_name FORMAT a10;
COL job_name FORMAT a20
COL state FORMAT a12
COL operation LIKE owner_name
COL job_mode LIKE owner_name
SELECT owner_name, job_name, operation, job_mode,
state, attached_sessions
FROM dba_datapump_jobs;

OWNER_NAME JOB_NAME OPERATION JOB_MODE STATE ATTACHED_SESSIONS
---------- -------------------- ---------- ---------- ------------ -----------------
ARJU SYS_EXPORT_SCHEMA_01 EXPORT SCHEMA NOT RUNNING 0
ARJU SYS_EXPORT_SCHEMA_02 EXPORT SCHEMA NOT RUNNING 0

Step 02: Check the state field. For orphaned jobs the state will be NOT RUNNING. So from the output we can say both are orphaned jobs.

Step 03: Drop the master table.
DROP TABLE ARJU.SYS_EXPORT_SCHEMA_01;
DROP TABLE ARJU.SYS_EXPORT_SCHEMA_02;

Step 04: Check for existing data pump jobs by query issued in step 01. If objects are in recyclebin bin then purge the objects from the recyclebin.

SQL> SELECT owner_name, job_name, operation, job_mode,
state, attached_sessions
FROM dba_datapump_jobs;

OWNER_NAME JOB_NAME OPERATION JOB_MODE STATE ATTACHED_SESSIONS
---------- -------------------- ---------- ---------- ------------ -----------------
ARJU BIN$xMNQdACzQ6yl22kj EXPORT SCHEMA NOT RUNNING 0
9U0B8A==$0
ARJU BIN$BmUy4r5MSX+ojxFk EXPORT SCHEMA NOT RUNNING 0
sw8ocg==$0

SQL> PURGE TABLE ARJU.SYS_EXPORT_SCHEMA_01;

Table purged.

SQL> PURGE TABLE ARJU.SYS_EXPORT_SCHEMA_02;

Table purged.

Check if there is any orphaned jobs again.
SQL> SELECT owner_name, job_name, operation, job_mode,
state, attached_sessions
FROM dba_datapump_jobs;

no rows selected

Step 05: In this stage you did not get any orphaned jobs if the jobs have a master table. If there are still jobs listed in dba_datapump_jobs do cleanup process like below.

SET serveroutput on
SET lines 100
DECLARE
job1 NUMBER;
BEGIN
job1 := DBMS_DATAPUMP.ATTACH('SYS_EXPORT_SCHEMA_01','ARJU');
DBMS_DATAPUMP.STOP_JOB (job1);
END;
/

DECLARE
job2 NUMBER;
BEGIN
job2 := DBMS_DATAPUMP.ATTACH('SYS_EXPORT_SCHEMA_02','ARJU');
DBMS_DATAPUMP.STOP_JOB (job2);
END;
/

Related Documents

Wednesday, May 6, 2009

Understand the Estimate parameter in data pump export

With the ESTIMATE_ONLY parameter as in discussed on http://arjudba.blogspot.com/2009/02/estimate-dumpfile-size-before-taking.html we can estimate the space in bytes per tables that would be consumed without actually performing data pump export operation or in other word without generating dump file.

With help of ESTIMATE parameter of data pump export you can specify the method that Export will use to estimate how much disk space each table in the export job will consume (in bytes) before performing actual data pump export operation.

The ESTIMATE parameter can take two parameters. Either BLOCKS (default) or STATISTICS.

The meaning of these two parameter values are specified below.

BLOCKS: The estimate is calculated by multiplying the number of database blocks used by the target objects with the appropriate block sizes.

STATISTICS: The estimate is calculated using statistics for each table. So to be accurate you must analyze table recently.

Note that the outcome specified by ESTIMATE=BLOCKS is far away from the size of the actual dumpfile. In fact, ESTIMATE=BLOCKS method generates more inaccurate result from dump file size when,

a) The table was created with a much bigger initial extent size than was needed for the actual table data.

b) Many rows have been deleted from the table, or a very small percentage of each block is used.


The outcome generated by ESTIMATE=STATISTICS is most accurate to dump file size if recently table is analyzed.

Below is an example shown both in case of ESTIMATE=STATISTICS and ESTIMATE=BLOCKS. In both cases data pump export dump file is generated after estimation of dump file.

E:\>expdp schemas=arju userid=arju/a dumpfile=arju_11_01_blocks.dmp directory=test estimate=blocks
Export: Release 10.2.0.1.0 - Production on Thursday, 07 May, 2009 11:51:12

Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
Starting "ARJU"."SYS_EXPORT_SCHEMA_03":  schemas=arju userid=arju/********
dumpfile=arju_11_01_blocks.dmp directory=test estimate=blocks
Estimate in progress using BLOCKS method...
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
.  estimated "ARJU"."SYS_EXPORT_SCHEMA_02"                 320 KB
.  estimated "ARJU"."SYS_EXPORT_SCHEMA_01"                 128 KB
.  estimated "ARJU"."AUTHOR"                                64 KB
.  estimated "ARJU"."BOOKAUTHOR"                            64 KB
.  estimated "ARJU"."BOOKS"                                 64 KB
.  estimated "ARJU"."BOOK_CUSTOMER"                         64 KB
.  estimated "ARJU"."BOOK_ORDER"                            64 KB
.  estimated "ARJU"."ORDERITEMS"                            64 KB
.  estimated "ARJU"."PROMOTION"                             64 KB
.  estimated "ARJU"."PUBLISHER"                             64 KB
.  estimated "ARJU"."T"                                     64 KB
Total estimation using BLOCKS method: 1024 KB
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/VIEW/VIEW
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
. . exported "ARJU"."SYS_EXPORT_SCHEMA_02"               214.3 KB    1125 rows
. . exported "ARJU"."SYS_EXPORT_SCHEMA_01"               31.55 KB      12 rows
. . exported "ARJU"."AUTHOR"                             5.835 KB      14 rows
. . exported "ARJU"."BOOKAUTHOR"                         5.609 KB      20 rows
. . exported "ARJU"."BOOKS"                              7.781 KB      14 rows
. . exported "ARJU"."BOOK_CUSTOMER"                      8.234 KB      21 rows
. . exported "ARJU"."BOOK_ORDER"                         8.398 KB      21 rows
. . exported "ARJU"."ORDERITEMS"                         6.742 KB      32 rows
. . exported "ARJU"."PROMOTION"                          5.710 KB       4 rows
. . exported "ARJU"."PUBLISHER"                          6.265 KB       8 rows
. . exported "ARJU"."T"                                  4.914 KB       1 rows
Master table "ARJU"."SYS_EXPORT_SCHEMA_03" successfully loaded/unloaded
******************************************************************************
Dump file set for ARJU.SYS_EXPORT_SCHEMA_03 is:
E:\ORACLE\TEST\ARJU_11_01_blocks.DMP
Job "ARJU"."SYS_EXPORT_SCHEMA_03" successfully completed at 11:51:39

E:\>expdp schemas=arju userid=arju/a dumpfile=arju_11_01_statistics.dmp directory=test estimate=statistics
Export: Release 10.2.0.1.0 - Production on Thursday, 07 May, 2009 11:52:12

Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
Starting "ARJU"."SYS_EXPORT_SCHEMA_03":  schemas=arju userid=arju/******** 
dumpfile=arju_11_01_statistics.dmp directory=test estimate=statistics
Estimate in progress using STATISTICS method...
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
.  estimated "ARJU"."SYS_EXPORT_SCHEMA_02"               187.0 KB
.  estimated "ARJU"."SYS_EXPORT_SCHEMA_01"               36.67 KB
.  estimated "ARJU"."BOOK_ORDER"                         8.832 KB
.  estimated "ARJU"."BOOK_CUSTOMER"                      8.708 KB
.  estimated "ARJU"."BOOKS"                              8.156 KB
.  estimated "ARJU"."ORDERITEMS"                         6.808 KB
.  estimated "ARJU"."PUBLISHER"                          6.542 KB
.  estimated "ARJU"."AUTHOR"                             6.054 KB
.  estimated "ARJU"."PROMOTION"                          5.882 KB
.  estimated "ARJU"."BOOKAUTHOR"                         5.746 KB
.  estimated "ARJU"."T"                                  5.061 KB
Total estimation using STATISTICS method: 285.5 KB
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/VIEW/VIEW
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
. . exported "ARJU"."SYS_EXPORT_SCHEMA_02"               214.3 KB    1125 rows
. . exported "ARJU"."SYS_EXPORT_SCHEMA_01"               31.55 KB      12 rows
. . exported "ARJU"."BOOK_ORDER"                         8.398 KB      21 rows
. . exported "ARJU"."BOOK_CUSTOMER"                      8.234 KB      21 rows
. . exported "ARJU"."BOOKS"                              7.781 KB      14 rows
. . exported "ARJU"."ORDERITEMS"                         6.742 KB      32 rows
. . exported "ARJU"."PUBLISHER"                          6.265 KB       8 rows
. . exported "ARJU"."AUTHOR"                             5.835 KB      14 rows
. . exported "ARJU"."PROMOTION"                          5.710 KB       4 rows
. . exported "ARJU"."BOOKAUTHOR"                         5.609 KB      20 rows
. . exported "ARJU"."T"                                  4.914 KB       1 rows
Master table "ARJU"."SYS_EXPORT_SCHEMA_03" successfully loaded/unloaded
******************************************************************************
Dump file set for ARJU.SYS_EXPORT_SCHEMA_03 is:
E:\ORACLE\TEST\ARJU_11_01_STATISTICS.DMP
Job "ARJU"."SYS_EXPORT_SCHEMA_03" successfully completed at 11:52:25

Using ESTIMATE=BLOCKS, before data pump export size is shown as 1024 KB and using ESTIMATE=STATISTICS, before data pump export size is shown as 285.5 KB and my actual dump file size was 472KB which is far away from estimation using ESTIMATE=BLOCKS as difference is 1024-472=552. In later case difference is 186.5.

Note that if a table involves LOBs, the dump file size may vary as ESTIMATE does not take LOB size into consideration.

Related Documents

ORA-39000, ORA-39143 dump file may be an original export dump file

Problem Description
E:\>impdp directory=test dumpfile=testexp_07_03_09.dmp userid=arju/a

Import: Release 10.2.0.1.0 - Production on Thursday, 07 May, 2009 10:07:00

Copyright (c) 2003, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORA-39001: invalid argument value
ORA-39000: bad dump file specification
ORA-39143: dump file "E:\oracle\Test\testexp_07_03_09.dmp" may be an original export dump file

Cause of the problem
The above problem happened whenever you try to use the Import Data Pump client (impdp) to import a dumpfile that was created with the original Export client (exp).

Though not related but similar error occured.
Like whenever you try to import from an empty file,
E:\>impdp directory=test dumpfile=testexp_07_03_09.dmp userid=arju/a

Import: Release 10.2.0.1.0 - Production on Thursday, 07 May, 2009 10:16:52

Copyright (c) 2003, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORA-39001: invalid argument value
ORA-39000: bad dump file specification
ORA-31619: invalid dump file "E:\oracle\Test\testexp_07_03_09.dmp"
ORA-27091: unable to queue I/O
ORA-27070: async read/write failed
OSD-04006: ReadFile() failure, unable to read from file
O/S-Error: (OS 38) Reached the end of the file.

Whenever you try to import from invalid dump file or corrupted dump file then you may get following error,

E:\>impdp directory=test dumpfile=testexp_07_03_09.dmp userid=arju/a

Import: Release 10.2.0.1.0 - Production on Thursday, 07 May, 2009 10:17:23

Copyright (c) 2003, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORA-39001: invalid argument value
ORA-39000: bad dump file specification
ORA-31619: invalid dump file "E:\oracle\Test\testexp_07_03_09.dmp"

Solution of the Problem
It is obvious that,
-Data pump export client(impdp) can't read file created by the original export client (exp).

and,

-Data exported by data pump client(expdp) can't be readable by original import client(imp).

So to import dumpfile that was created with the original Export client (exp) you must have to use imp.

Like,
imp userid=arju/a file=e:\oracle\testexp_07_03_09.dmp

Otherwise you can export the dumpfile using expdp and then try impdp while importing.

Related Documents
http://arjudba.blogspot.com/2009/02/how-to-do-data-pump-import-from-higher.html
http://arjudba.blogspot.com/2008/04/data-pump-exportimport-how-we-can-do-it.html
http://arjudba.blogspot.com/2009/02/ude-00018-data-pump-client-is.html
http://arjudba.blogspot.com/2008/04/exportimport-datapump-parameter-query.html

Sunday, May 3, 2009

Expdp fails with ORA-39001,ORA-39169,ORA-39006,ORA-39022

Problem Description
Connecting to local 10.2.01 database whenever I want to take data pump export into remote 11g database machine using NETWORK_LINK parameter data pump export fails with ORA-39001, ORA-39169 as below.
[oracle@localhost bin]$ ./expdp system/a NETWORK_LINK=maestro.net schemas=maximsg VERSION=10.2

Export: Release 10.2.0.1.0 - Production on Wednesday, 04 March, 2009 5:27:39

Copyright (c) 2003, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORA-39001: invalid argument value
ORA-39169: Local version of 10.2.0.1.0 cannot work with remote version of 11.1.0.6.0.

Similarly, connecting to local 10.1.0.* database whenever I want to take data pump export into remote 11g database machine using NETWORK_LINK parameter data pump export fails with
ORA-39006: internal error
ORA-39022: Database version 11.1.0.6.0 is not supported.

Doing expdp/impdp connected to a local 10.2.0.3 database fails with,

ORA-39127: unexpected error from call to export_string
:=SYS.DBMS_CUBE_EXP.schema_info_exp('ARJU',1,1,'11.01.00.00.00',newblock)
ORA-37118: The OLAP API library was not preloaded.
ORA-06512: at "SYS.DBMS_CUBE_EXP", line 205
ORA-06512: at "SYS.DBMS_CUBE_EXP", line 280
ORA-06512: at line 1
ORA-06512: at "SYS.DBMS_METADATA", line 5980
ORA-39125: Worker unexpected fatal error in KUPW$WORKER.UNLOAD_METADATA
while calling DBMS_METADATA.FETCH_XML_CLOB [TABLE_STATISTICS]
ORA-06502: PL/SQL: numeric or value error: invalid LOB locator specified:
ORA-22275
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 105
ORA-06512: at "SYS.KUPW$WORKER", line 6234

Doing expdp/impdp connecting to a local 10.2.0.4.0 database
...
ORA-39126: Worker unexpected fatal error in KUPW$WORKER.MAIN [INSERT
INTO sys.ku$_list_filter_temp@tiger SELECT process_order, duplicate,
object_name, base_process_order FROM "SYSTEM"."SYS_EXPORT_SCHEMA_01"
WHERE process_order = :1]
ORA-00947: not enough values
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: at "SYS.KUPW$WORKER", line 6345

Cause of the Problem
The error occurred due to several oracle bugs. Whenever it is attempted to export data over a database link initiated from lower database compatibility level to a higher compatibility level it fails due to bug. That is if data pump export operation is started on a local 10.x database with NETWORK_LINK parameter and then the database link connects to a remote 11.x database then the Data Pump job may fail due to several defects.

As you see above, error messages displayed vary from version to version.

The errors ORA-39006 and ORA-39022 displayed if a Data Pump job with NETWORK_LINK is started on a local 10.1.x database and the database link connects to a remote 10.2.x or 11.x source database.

The errors ORA-39001 and ORA-39169 displayed if a Data Pump job with NETWORK_LINK is started on a local 10.2.0.1 or 10.2.0.2 database and the database link connects to a remote 11.x source database.

The errors ORA-39127 and ORA-37118 displayed if a Data Pump job with NETWORK_LINK is started on a local 10.2.0.3 database and the database link connects to a remote 11.x source database.

The errors ORA-39126 and ORA-00947 displayed if a Data Pump job with NETWORK_LINK is started on a local 10.2.0.4 database and the database link connects to a remote 11.x source database.

Note that compatibility settings of remote database is not an issue here.


Solution of the Problem
Solution 01:
- Do data pump export operation without NETWORK_LINK parameter.
- Transfer the data pump export file on the the other server.
Solution 02:
With usage of NETWORK_LINK parameter,
- Make sure that local connecting database is a 10.2.0.3.0 or higher release database.
- Explicitly use VERSION=10.2 while doing expdp operation.

Related Documents