Skip to main content Skip to complementary content

Stats & Logs

When executing a Job, Talend Studio allows you to collect key runtime execution information, statistics, logs and volumetrics, for tracking and analysis through the tStatCatcher Statistics option or through using a log component.

Examples include monitoring how often a Job is executed, how long it takes to run, how many records are processed, whether it completes successfully, what is the reason it fails, etc.

The statistics, logs and volumetrics data can be written out to the Java console, delimited files and database tables, or any combination of the three. If the data is stored in delimited files or database tables, it is written to three separate files or tables based on the configuration as described in the following procedure.

Procedure

  1. Click Project settings on the toolbar of the Talend Studio main window or click File > Edit Project Properties from the menu bar to open the Project Settings dialog box.
  2. In the tree view of the dialog box, expand the Job Settings node and click Stats & Logs to open the corresponding view.
    Stats & Logs configuration in the Project Settings dialog box.
    Note that the Stats & Logs preferences can be set for all Jobs in a project via Project Settings if the preferences for Stats & Logs will not change depending upon the context of execution. It can also be set separately for individual Jobs via the Stats & Logs view on the Job tab. For more information, see Automating the use of statistics & logs.
  3. Specify the type of the information you want to capture by selecting the Use statistics (tStatCatcher), Use logs (tLogCatcher), and/or Use volumetrics (tFlowMeterCatcher) check boxes where relevant.
    • When the Use statistics (tStatCatcher) check box is selected, the key information such as the Job start time, end time, duration, and success or failure status is captured. If the Catch components statistics (tStatCatcher Statistics) check box is selected, or if the tStatCatcher Statistics check box on the Advanced settings view of a specific component is selected, the key information for the component is also captured.
    • When the Use logs (tLogCatcher) check box is selected, messages are harvested from Java Exception, the tDie component, and/or the tWarn component according to the state of the Catch runtime errors, Catch user errors, and Catch user warnings check boxes.
    • When the Use volumetrics (tFlowMeterCatcher) check box is selected, the row counts from all tFlowMeter components in a Job is captured.
  4. Select the On Console check box if you want to display data on the console.
  5. Select the On Files check box if you want to store data in delimited files.
    The relevant fields are enabled or disabled according to your settings. Specify the name of the delimited files in the Stats File Name, Log File Name, and/or Meter file name fields according to the type of the information you select.
  6. Select the On Database check box if you want to store data in database tables.
    The relevant fields are enabled or disabled according to your settings. Specify the value of the database connection parameters in the corresponding fields, and the name of the database tables in the Stats Table, Logs Table, and/or Meter Table fields according to the type of the information you select.
    Information noteTip: The stats, logs, and/or meter tables are created automatically by Talend if they do not exist. You can also create those tables using tCreateTable based on the schema of tStatCatcher, tLogCatcher, and tFlowMeterCatcher, with customized column length to avoid data truncation issues.
    Information noteWarning: Stats or logs is not supported with JDBC generic component for Teradata. To store statistics and log data in a Teradata database, select Teradata from the Db Type drop-down list and provide related parameters for connecting the Teradata database as prompted.

In this section

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here