Transformer Stage in DataStage

The Transformer stage is a processing stage. It appears under the processing category in the tool palette. Transformer stages allow you to create transformations to apply to your data. These transformations can be simple or complex and can be applied to individual columns in your data. Transformations are specified using a set of functions.

Transformer stages can have a single input and any number of outputs. It can also have a reject link that takes any rows which have not been written to any of the outputs links by reason of a write failure or expression evaluation failure.

Transformer NULL Handling and Reject Link

When evaluating expressions for output derivations or link constraints, the Transformer will reject (through the reject link indicated by a dashed line) any row that has a NULL value used in the expression.

To create a Transformer reject link in DataStage Designer, right-click on an output link and choose “Convert to Reject”.

The Transformer rejects NULL derivation results because the rules for arithmetic and string handling of NULL values are by definition undefined.

For this reason, always test for null values before using a column in an expression, for example:

If IsNull(link.col) Then… Else…

Conditionally Aborting Jobs

The Transformer can be used to conditionally abort a job when incoming data matches a specific rule. Create a new output link that will handle rows that match the abort rule.

Within the link constraints dialogue box, apply the abort rule to this output link and set the “Abort After Rows” count to the number of rows allowed before the job should aborted.

What are Stage Variable, Derivation  and Constraint ?

Stage Variable - An intermediate processing variable that retains value during read and doesnt pass the value into target column.  It executes locally within the stage.

Derivation - Expression that specifies value to be passed on to the target column. 

Constraint - Conditions that are either true or false that specifies flow of data with a link. 

Order of execution is:

1. Stage Variable 2. Constraint  3. Derivation

What is the difference between Basic transformer and Normal transformer ?

- A Basic transformer compiles in "Basic Language" whereas a Normal Transformer compiles in "C++".

- Basic transformer does not run on multiple nodes wheras a Normal Transformer can run on multiple nodes giving better performance.

- Basic transfomer takes less time to compile than the Normal Transformer.

- A basic transformer should be used in Server Jobs.

- Normal Transformer should be used in Parallel jobs as it will run on multiple nodes here giving better performance.