Skip to main content Skip to complementary content

Comparing the number of airline accidents and filtering these airlines

A pipeline with a test source, a Numbers processor, a Filter processor, and a Test destination.

Before you begin

  • You have previously created a connection to the system storing your source data.

    Here, a Test connection.

  • You have previously added the dataset holding your source data.

    Download and extract the file: numbers-airlines.zip. It contains a dataset with data about airlines with the number of incidents and accidents as well as fatalities, etc.

  • You also have created the connection and the related dataset that will hold the processed data.

    Here, a Test dataset.

Procedure

  1. Click Add pipeline on the Pipelines page. Your new pipeline opens.
  2. Give the pipeline a meaningful name.

    Example

    Compare number of air crashes and filter airlines
  3. Click ADD SOURCE to open the panel allowing you to select your source data, here data about airlines and air crashes.

    Example

    Preview of a data sample about airline and air crashes.
  4. Select your dataset and click Select in order to add it to the pipeline.
    Rename it if needed.
  5. Click Plus and add a Number processor to the pipeline. The configuration panel opens.
  6. Give a meaningful name to the processor.

    Example

    compare number of fatal accidents
  7. In the Configuration area:
    1. Select Compare numbers in the Function name list.
    2. Select .fatal_accidents_85_99 in the Fields to process list as you want to compare this field value (number of fatal accidents) with your custom value.
    3. Enable the Create new column option and name it comparison.
    4. Select greater or equals than in the Compare mode list, select Value in the Use with list and enter 2 in the Value field. This way you will be able to compare the number of fatal accidents per airlines and see which ones have had at least two fatal accidents.
  8. Click Save to save your configuration.

    You can preview your data before and after the comparison.

    Values have been compared and a new field allows you to display which airlines have had at least two fatal accidents (true) and which have had less than two fatal accidents (false).

    Preview of the Numbers processor after applying a comparison rule on the number of accidents.
  9. Click Plus and add a Filter processor to the pipeline. The configuration panel opens.
  10. Give a meaningful name to the processor.

    Example

    airlines with at least 2 fatal accidents
  11. In the Filters area:
    1. Select .comparison in the Input list, as you want to filter airlines based on this value.
    2. Select None in the Optionally select a function to apply list, as you do not want to apply a function while filtering records.
    3. Select == in the Operator list and type in true in the Value list as you want to filter on airlines who met the requirement of "2 or more fatal accidents".
  12. Click Save to save your configuration.

    Look at the preview of the processor to compare your data before and after the operation.

    Preview of the Filter processor after applying a filter on 'true' values.
  13. Click ADD DESTINATION and select the dataset that will hold your processed data.
    Rename it if needed.
  14. On the top toolbar of Talend Cloud Pipeline Designer, click the Run button to open the panel allowing you to select your run profile.
  15. Select your run profile in the list (for more information, see Run profiles), then click Run to run your pipeline.

Results

Your pipeline is being executed, the data is compared and filtered according to the conditions you have stated and you can see that 19 airlines in this dataset have had at least two fatal accidents. The output is sent to the target system you have indicated.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here