DATASTAGE ERROR CODES

DATASTAGE ERROR CODES

API error codes in numerical order

CODE	ERROR TOKEN	DESCRIPTION
0	DSJE_NOERROR	No InfoSphere DataStage API error has occurred.
-1	DSJE_BADHANDLE	Invalid JobHandle.
-2	DSJE_BADSTATE	Job is not in the right state (compiled, not running).
-3	DSJE_BADPARAM	ParamName is not a parameter name in the job.
-4	DSJE_BADVALUE	Invalid MaxNumber value.
-5	DSJE_BADTYPE	Information or event type was unrecognized.
-6	DSJE_WRONGJOB	Job for this JobHandle was not started from a call to DSRunJob by the current process.
-7	DSJE_BADSTAGE	StageName does not refer to a known stage in the job.
-8	DSJE_NOTINSTAGE	Internal engine error.
-9	DSJE_BADLINK	LinkName does not refer to a known link for the stage in question.
-10	DSJE_JOBLOCKED	The job is locked by another process.
-11	DSJE_JOBDELETED	The job has been deleted.
-12	DSJE_BADNAME	Invalid project name.
-13	DSJE_BADTIME	Invalid StartTime or EndTime value.
-14	DSJE_TIMEOUT	The job appears not to have started after waiting for a reasonable length of time. (About 30 minutes.)
-15	DSJE_DECRYPTERR	Failed to decrypt encrypted values.
-16	DSJE_NOACCESS	Cannot get values, default values or design default values for any job except the current job.
-99	DSJE_REPERROR	General engine error.
-100	DSJE_NOTADMINUSER	User is not an administrator.
-101	DSJE_ISADMINFAILED	Failed to determine whether user is an administrator.
-102	DSJE_READPROJPROPERTY	Failed to read property.
-103	DSJE_WRITEPROJPROPERTY	Property not supported.
-104	DSJE_BADPROPERTY	Unknown property name.
-105	DSJE_PROPNOTSUPPORTED	Unsupported property.
-106	DSJE_BADPROPVALUE	Invalid value for this property.
-107	DSJE_OSHVISIBLEFLAG	Failed to get value for OSHVisible.
-108	DSJE_BADENVVARNAME	Invalid environment variable name.
-109	DSJE_BADENVVARTYPE	Invalid environment variable type.
-110	DSJE_BADENVVARPROMPT	No prompt supplied.
-111	DSJE_READENVVARDEFNS	Failed to read environment variable definitions.
-112	DSJE_READENVVARVALUES	Failed to read environment variable values.
-113	DSJE_WRITEENVVARDEFNS	Failed to write environment variable definitions.
-114	DSJE_WRITEENVVARVALUES	Failed to write environment variable values.
-115	DSJE_DUPENVVARNAME	Environment variable being added already exists.
-116	DSJE_BADENVVAR	Environment variable does not exist.
-117	DSJE_NOTUSERDEFINED	Environment variable is not user-defined and therefore cannot be deleted.
-118	DSJE_BADBOOLEANVALUE	Invalid value given for a boolean environment variable.
-119	DSJE_BADNUMERICVALUE	Invalid value given for an integer environment variable.
-120	DSJE_BADLISTVALUE	Invalid value given for a list environment variable.
-121	DSJE_PXNOTINSTALLED	Environment variable is specific to parallel jobs which are not available.
-122	DSJE_ISPARALLELLICENCED	Failed to determine if parallel jobs are available.
-123	DSJE_ENCODEFAILED	Failed to encode an encrypted value.
-124	DSJE_DELPROJFAILED	Failed to delete project definition.
-125	DSJE_DELPROJFILESFAILED	Failed to delete project files.
-126	DSJE_LISTSCHEDULEFAILED	Failed to get list of scheduled jobs for project.
-127	DSJE_CLEARSCHEDULEFAILED	Failed to clear scheduled jobs for project.
-128	DSJE_BADPROJNAME	Invalid project name supplied.
-129	DSJE_GETDEFAULTPATHFAILED	Failed to determine default project directory.
-130	DSJE_BADPROJLOCATION	Invalid path name supplied.
-131	DSJE_INVALIDPROJECTLOCATION	Invalid path name supplied.
-132	DSJE_OPENFAILED	Failed to open UV.ACCOUNT file.
-133	DSJE_READUFAILED	Failed to lock project create lock record.
-134	DSJE_ADDPROJECTBLOCKED	Another user is adding a project.
-135	DSJE_ADDPROJECTFAILED	Failed to add project.
-136	DSJE_LICENSEPROJECTFAILED	Failed to license project.
-137	DSJE_RELEASEFAILED	Failed to release project create lock record.
-138	DSJE_DELETEPROJECTBLOCKED	Project locked by another user.
-139	DSJE_NOTAPROJECT	Failed to log to project.
-140	DSJE_ACCOUNTPATHFAILED	Failed to get account path.
-141	DSJE_LOGTOFAILED	Failed to log to UV account.
-201	DSJE_UNKNOWN_JOBNAME	The supplied job name cannot be found in the project.
-1001	DSJE_NOMORE	All events matching the filter criteria have been returned.
-1002	DSJE_BADPROJECT	ProjectName is not a known InfoSphere DataStage project.
-1003	DSJE_NO_DATASTAGE	InfoSphere DataStage is not installed on the system.
-1004	DSJE_OPENFAIL	The attempt to open the job failed – perhaps it has not been compiled.
-1005	DSJE_NO_MEMORY	Failed to allocate dynamic memory.
-1006	DSJE_SERVER_ERROR	An unexpected or unknown error occurred in the engine.
-1007	DSJE_NOT_AVAILABLE	The requested information was not found.
-1008	DSJE_BAD_VERSION	The engine does not support this version of the InfoSphere DataStage API.
-1009	DSJE_INCOMPATIBLE_SERVER	The engine version is incompatible with this version of the InfoSphere DataStage API.

API Communication Layer Error Codes

ERROR NUMBER	DESCRIPTION
39121	The InfoSphere DataStage license has expired.
39134	The InfoSphere DataStage user limit has been reached.
80011	Incorrect system name or invalid user name or password provided.
80019	Password has expired.

DATASTAGE COMMON ERRORS/WARNINGS AND SOLUTIONS

1.     While running ./NodeAgents.sh start command… getting the following error: “LoggingAgent.sh process stopped unexpectedly”

SOL:   needs to kill LoggingAgentSocketImpl

              Ps –ef |  grep  LoggingAgentSocketImpl   (OR)

              PS –ef |               grep Agent  (to check the process id of the above)

2.     Warning: A sequential operator cannot preserve the partitioning of input data set on input port 0

SOL:    Clear the preserve partition flag before Sequential file stages.

3.     Warning: A user defined sort operator does not satisfy the requirements.

SOL:   Check the order of sorting columns and make sure use the same order when use join stage after sort to joing two inputs.

4.     Conversion error calling conversion routine timestamp_from_string data may have been lost. xfmJournals,1: Conversion error calling conversion routine decimal_from_string data may have been lost

SOL:    check for the correct date format or decimal format and also null values in the date or decimal fields before passing to datastage StringToDate, DateToString,DecimalToString or StringToDecimal functions.

5.     To display all the jobs in command line

SOL:   

cd /opt/ibm/InformationServer/Server/DSEngine/bin
./dsjob -ljobs <project_name>

6.     “Error trying to query dsadm[]. There might be an issue in database server”

SOL:   Check XMETA connectivity.

db2 connect to xmeta (A connection to or activation of database “xmeta” cannot be made because of  BACKUP pending)

7.      “DSR_ADMIN: Unable to find the new project location”

SOL:   Template.ini file might be missing in /opt/ibm/InformationServer/Server.

           Copy the file from another severs.

8.      “Designer LOCKS UP while trying to open any stage”

SOL:   Double click on the stage that locks up datastage

           Press ALT+SPACE

           Windows menu will popup and select Restore

           It will show your properties window now

           Click on “X” to close this window.

           Now, double click again and try whether properties window appears.

9.      “Error Setting up internal communications (fifo RT_SCTEMP/job_name.fifo)

SOL:   Remove the locks and try to run (OR)

          Restart DSEngine and try to run (OR)

Go to /opt/ibm/InformationServer/server/Projects/proj_name/

            ls RT_SCT* then

            rm –f  RT_SCTEMP

            then try to restart it.

10.      While attempting to compile job,  “failed to invoke GenRunTime using Phantom process helper”

RC:     /tmp space might be full

           Job status is incorrect

           Format problems with projects uvodbc.config file

SOL:      a)        clean up /tmp directory

              b)        DS Director à JOB à clear status file

              c)         confirm uvodbc.config has the following entry/format:

                       [ODBC SOURCES]

                       <local uv>

                       DBMSTYPE = UNIVERSE

                       Network  = TCP/IP

                       Service =  uvserver

                       Host = 127.0.0.1

Surrogate_Key_Stage

SURROGATE KEY IN DATASTAGE

Surrogate Key is a unique identification key. It is alternative to natural key .

And in natural key, it may have alphanumeric composite key but the surrogate is

always single numeric key.

Surrogate key is used to generate key columns, for which characteristics can be

specified. The surrogate key generates sequential incremental and unique integers for a

provided start point. It can have a single input and a single output link.



WHAT IS THE IMPORTANCE OF OF SURROGATE KEY

Surrogate Key is a Primary Key for a dimensional table. ( Surrogate key is alternate to Primary Key) The most importance of using Surrogate key is not affected by the changes going on with a database.

And in Surrogate Key Duplicates are allowed, where it cant be happened in the Primary Key .

By using Surrogate key we can continue the sequence for any jobs. If any job was aborted at the n records loaded.. By using surrogate key you can continue the sequence from n+1.



Surrogate Key Generator:

The Surrogate Key Generator stage is a processing stage that generates surrogate key columns and maintains the key source.

A surrogate key is a unique primary key that is not derived from the data that it represents, therefore changes to the data will not change the primary key. In a star schema database, surrogate keys are used to join a fact table to a dimension table.

surrogate key generator stage uses:

• Create or delete the key source before other jobs run
• Update a state file with a range of key values
• Generate surrogate key columns and pass them to the next stage in the job
• View the contents of the state file

Generated keys are 64 bit integers and the key source can be stat file or database sequence.

Creating the key source

Drag the surrogate key stage from palette to parallel job canvas with no input and output links.

Double click on the surrogate key stage and click on properties tab.

Properties:

Key Source Action = create

Source Type : FlatFile or Database sequence(in this case we are using FlatFile)

When you run the job it will create an empty file.

If you want to the check the content change the View Stat File = YES and check the job log for details.

skey_genstage,0: State file /tmp/skeycutomerdim.stat is empty.

if you try to create the same file again job will abort with the following error.

skey_genstage,0: Unable to create state file /tmp/skeycutomerdim.stat: File exists.

Deleting the key source:

Updating the stat File:

To update the stat file add surrogate key stage to the job with single input link from other stage.

We use this process to update the stat file if it is corrupted or deleted.

1)open the surrogate key stage editor and go to the properties tab.

If the stat file exists we can update otherwise we can create and update it.

We are using SkeyValue parameter to update the stat file using transformer stage.

Generating Surrogate Keys:

Now we have created stat file and will generate keys using the stat key file.

Click on the surrogate keys stage and go to properties add add type a name for the surrogate key column in the Generated Output Column Name property

Go to ouput and define the mapping like below.

Rowgen we are using 10 rows and hence when we run the job we see 10 skey values in the output.

I have updated the stat file with 100 and below is the output.

If you want to generate the key value from begining you can use following property in the surrogate key stage.

1. If the key source is a flat file, specify how keys are generated:
   • To generate keys in sequence from the highest value that was last used, set the Generate Key from Last Highest Value property to Yes. Any gaps in the key range are ignored.
   • To specify a value to initialize the key source, add the File Initial Value property to the Options group, and specify the start value for key generation.
   • To control the block size for key ranges, add the File Block Size property to the Options group, set this property toUser specified, and specify a value for the block size.
2. If there is no input link, add the Number of Records property to the Options group, and specify how many records to generate.