Talend Data Mapper: Tips and Tutorials Part 3

Welcome back to the final part of our series about the Talend Data Mapper. This week we discuss how to handle looping structures in different ways, creating dynamic mapping paths with conditionals and a little bonus tip for keeping track of how many loops your mapping has actually gone through.

We are also sharing posts and previews on channels like Instagram, TikTok, LinkedIn and YouTube. Feel free to join the communities over there and keep up to date on Talend, SQL, Data Integration, Analytics and much more!

Talend Data Mapper Applied

The data mapper has many functions, utilities and tricks up its sleeve. Getting to know some of these may help you in deciding if the Talend Data Mapper can help you in the future. This is our continuation of helpful tips and step by step tutorials on getting the most out of the Talend Data mapping tool.

Split Loop vs Unrolled from Loop

The Talend Data Mapper is used for hierarchical data structures with many looping elements but has the flexibility to do more. It can internally change structures to accommodate for custom non-looping elements with the same specified structure. You can develop a standardised structure for the output and then adapt it to each specific mapping, without having to save many duplicates of the base model.

Handling looping elements within each mapping accounts for two scenarios:

• Splitting a looping element into two similar looping elements on the same level. For example, when taking a nested loop from the input into an equal level on the output.
• Unrolling a looping element into one original looping element and one non-looping element with the same structure. For example, when setting a default value if the looping element is not present in the incoming data.

Split Loop in Talend Data Mapper

Splitting a loop is often used when flattening a mapping of recursive loops that are in the incoming data. In the standardized structure for the output is a loop element that accounts for the original loop and all nested recursive loops in the input structure. Splitting the loop in the output would make flattening this data much easier. In our example the structure has a main ‘Order’ and then a recursive loop of multiple ‘Orders’ within.

1. Assuming the input and output structures are in place, highlight the looping element in the output structure that you are going to use for the recursive incoming loop.
2. Right-click the highlighted element and select Split Loop. This will create a duplicate of the original loop with the note ‘(Split from loop)’ and it will add a note ‘[2]’ to the original loop, indicating it has been split.
   
3. Now map the first loop from the output structure to the top-level loop from the input, this will be the first loop of the mapping.
4. Next, you map the second loop in the output structure to the nested loop in the input. This will add a new loop expression to handle the nested loop and for the output it will produce a flattened data structure.
   
   

Unrolled Loop in Talend Data Mapper

Unrolling a loop can be used to customize the output structure to accommodate for a default value if a condition is met or the loop is not present in the input data. This can also be used to flatten incoming data structures, or when working with systems that will receive the output and always need certain fields to be present.

1. With the input and output structures in place, highlight the looping element in the output structure that you are going to unroll.
2. Right-click the highlighted element and select Unroll from loop. A duplicate of the looping element is created, but now without a loop being added to it. This is indicated in the Data Mapper by not having the ‘(0:*)’ and having the added note of ‘(Unrolled from loop)’.For the first value we drag on the ‘OrderSource’ element from the input. The second value will be a constant from the functions panel, which we make a string that is ‘WEB’.
3. We will create a default output value that will be written if a condition is met in the input structure: if OrderSource is equal to WEB. In the Unrolled from loop element, select the Emit tab.
4. In the Emit tab we build a conditional statement that returns true when ‘OrderSource’ is equal to ‘WEB’. From the functions panel, drag the ‘Equal’ function to the Emit tab. It will ask for two values to be compared.
5. For the first value we drag on the ‘OrderSource’ element from the input. The second value will be a constant from the functions panel, which we make a string that is ‘WEB’.
6. Next, we can add our values to the elements in the unrolled loop. These can be hardcoded constants, ExternalMappedValues, functions or elements from the input structure.
7. We also map the looping element in the output structure with the elements from the input structure, which will be used when the ‘OrderSource’ is not equal to ‘WEB’.

Conditional Statements

Conditional statements are at the core of controlling the flow of Data Mapping by setting conditionals. In the Talend Data Mapper the conditional statements are functions in the category of Comparison and Logical. Commonly used functions are: IfThen, IfThenElse and Choice.

When building conditional statements with a programming language, you usually work from left to right. Within the Talend Data Mapper, due to the graphical interface, you work from top to bottom. This might take some practice to get into the right order of setting up the conditionals, since you need to use other functions to work out the comparison or logics of the statement.

A simple example would be: If ‘FirstValue’ is equal to ‘SecondValue’, then ‘ThirdValue’. In programming you’d write from left to right:

if ( FirstValue == SecondValue ) { value = ThirdValue; }

In the Talend Data Mapper you will build the conditional from functions:

IfThen >Equal > FirstValue, SecondValue > Then > ThirdValue

Each of these inner statements can also be functions, using input elements or ExternalVariables. These statements can get complex but are very modular and can easily be adjusted and adapted to your needs.

IfThen

The IfThen function evaluates the Condition argument and returns the value of the Then argument if the condition is True, or nothing is returned if the condition is false.

In our example below we use a Condition Statement to check if the value from the input is equal to a value in an ExternalVariable. When the returned result is true, the value of 1 is added to the output element.

IfThenElse

The IfThenElse function evaluates the Condition argument and returns the value of the Then argument if the condition is True. If the condition returns False it returns the value of the Else argument.

In the next example we use a Conditional Statement to evaluate the value from the input against a constant value. If it returns true, a routine is called to do a lookup on the value of the input element. If it returns false, the input element is directly mapped to the output element.

Choice

The choice function is used to provide the value from a list of conditional statements, like a collection of nested IfThenElse functions, but much easier to change and extend. Each Choice function consists of any number of ChoiceValue arguments and each ChoiceValue is a key/value pair where the key is a condition. If none of the conditions are satisfied, one of the ChoiceValue arguments can be set as a default value.