Skip to main content Skip to complementary content

Sorting the input records

At the end of this link, Talend Studio automatically accomplishes the recollecting step to group and output the execution results to the next component.

Configuring tSortRow

Procedure

  1. Double-click tSortRow to open its Component view.
    Basic settings view.
  2. Under the Criteria table, click the Add button three times to add three rows to the table.
  3. In the Schema column column, select, for each row, the schema column to be used as the sorting criterion. In this example, select ZipCode, City and Address, sequentially.
  4. In the Sort num or alpha? column, select alpha for all the three rows.
  5. In the Order asc or desc column, select asc for all the three rows.
  6. If the schema does not appear, click the Sync columns button to retrieve the schema from the preceding component.
  7. Click Advanced settings to open its view.
    Advanced settings view.
  8. Select Sort on disk. Then the Temp data directory path field and the Create temp data directory if not exist check box appear.
  9. In Temp data directory path, enter the path to, or browse to the folder you want to use to store the temporary data processed by tSortRow. In this approach, tSortRow is enabled to sort considerably more data.
    As the threads will overwrite each other if they are written in the same directory, you need to create the folder for each thread to be processed using its thread ID.
    To use the variable representing the thread IDs, you need to click  Code  to open its view and in that view, find this variable by searching for thread_id. In this example, this variable is  tCollector_1_THREAD_ID.
    Searching for thread_id in the Code view.
    Then you need to enter the path using this variable  This path reads like:
    "E:/Studio/workspace/temp"+((Integer)globalMap.get("tCollector_1_THREAD_ID")).
  10. Ensure that the Create temp data directory if not exists check box is selected.

Configuring the departitioning step

Procedure

  1. Click the link representing the departitioning step to open its Component view and click the Parallelization tab.
    Parallelization view.
    The Departition row option has been automatically selected in the Type area. If you select None, you are actually disabling parallelization for the data flow to be handled over this link. Note that depending on the link you are configuring, a Repartition row option may become available in the Type area to repartition a data flow already departitioned.
    In this Parallelization view, you need to define the following properties:
    • Buffer Size: the number of rows to be processed before the memory is freed.
    • Merge sort partitions: this allows you to implement the Mergesort algorithm to ensure the consistency of data.
  2. If required, change the values in the Buffer Size field to adapt the memory capacity. In this example, we leave the default value.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here