Transformation in Talend ETL by using tMap and condition-based filter

4 years ago

Request Free Demo

Email: sales@bilytica.com

This blog will be focused on Transformation that how to create a job that uses tMap component by adding condition-based filters. For this purpose, we have a scenario like a data file named movies. BI Company in Saudi Arabia offering exclusive services by working on this we need to Transform (filter) movies according to their release years, here we need to create three files one who contains these movies which was released in 90s, and second which was released in 80s and the remaining are all other movies categories Like below e.g.

For this purpose, there are only few steps which we need to create this job. Those steps are following below:

Create a job and add tMap component.
Add a simple filter.
Store the result in a simple file.
Create a second filter.
Create a filter reject output.

We need to create a new job and adding my movies metadata file as a tFileInputDelimited component and to create new filter we also need to add tMap and SAP BO Business Objects services in Saudi Arabia component in a job to create the flow of data which we need to connect with both components and give new name(movies).

Now we need to add a filter which select only those movies which were released in 1990s or later, for this we will open tMap component and create new output by clicking Plus button with in output section and give it name moviesFormThe90s. we need all column data included in output filter, for this we will select all column value and drag it into output column. Now we need a condition-based filter for filtering data, for this click activate expression filter icon and create filter condition for only 90s movies or later on to create condition click release year and drop it into expression field and click ok.

Store the result in a simple file.

Now add a tFileOutputXML to the component job to store the result in a file and add connection between tMap and tFileOutputXML and give the name movies>=1990s and make sure that with in component view specify the path where you want to store output file now run the job.

Similarly repeat same step for second filter and this time condition will be changed because we will filter those movies which were released in the 80s or before so for this we apply this condition releaseYear>=1990s && movies. releaseYear<1990 and click ok. After that same like above add second tFileOutputXML between tMap connection and specify the path to store the input and in the end run the job.

Create a filter Reject Output.

In the last, we will add another filter to catch the reject output repeat same steps give the output name movies before 80s to create a filter reject output clicking the tMap setting icon and select the ellipsis [….] in the value field of catch output reject property and click true this will collect all rows of data which do not match other two condition and click ok after this add an other a tFileOutputXML to the component job to store the result in a file and add connection between tMap and tFileOutputXML and give the name movies<1980s and make sure with in component view specify the path where you want to store output file now run the job.