Creating files or database tables
Talend Activity Monitoring Console is an application that allows you to monitor Job executions. The Job executions are monitored using three files or database tables that relate to the following data:
-
collection of logs,
-
component statistics,
-
data flow volumes.
To store this data, you need to create three files or database tables respectively using the schema of the tLogCatcher, tStatCatcher, tFlowMeterCatcher components (present in the Palette of your Talend Studio).
The files can be empty, and their structure will be generated automatically if you use files as datasources to store the activity monitoring information.
To create the database tables to be used as data sources for the monitoring console, do the following:
-
Create a database that you name amc, for example.
-
Launch Talend Studio.
-
Create a Job that contains three tCreateTable components.
-
Define the connection parameters to the amc in the three tCreateTable components.
-
Define the data structure by taking the schema provided in the components: tLogCatcher, tStatCatcher and tFlowMeterCatcher.
Note that after creating the tStatCatcher, tLogCatcher and tFlowMeterCatcher tables, you have to make sure the moment data type is set to datetime instead of date.
-- MySQL dump 10.13 Distrib 5.5.24, for Win32 (x86)
--
-- Host: 127.0.0.1 Database: amc
-- ------------------------------------------------------
-- Server version 5.5.24-log
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Current Database: `amc`
--
CREATE DATABASE /*!32312 IF NOT EXISTS*/ `amc` /*!40100 DEFAULT CHARACTER SET latin1 */;
USE `amc`;
--
-- Table structure for table `flowmeter`
--
DROP TABLE IF EXISTS `flowmeter`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `flowmeter` (
`moment` datetime DEFAULT NULL,
`pid` varchar(20) DEFAULT NULL,
`father_pid` varchar(20) DEFAULT NULL,
`root_pid` varchar(20) DEFAULT NULL,
`system_pid` bigint(8) DEFAULT NULL,
`project` varchar(50) DEFAULT NULL,
`job` varchar(255) DEFAULT NULL,
`job_repository_id` varchar(255) DEFAULT NULL,
`job_version` varchar(255) DEFAULT NULL,
`context` varchar(50) DEFAULT NULL,
`origin` varchar(255) DEFAULT NULL,
`label` varchar(255) DEFAULT NULL,
`count` int(3) DEFAULT NULL,
`reference` int(3) DEFAULT NULL,
`thresholds` varchar(255) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `flowmeter`
--
LOCK TABLES `flowmeter` WRITE;
/*!40000 ALTER TABLE `flowmeter` DISABLE KEYS */;
/*!40000 ALTER TABLE `flowmeter` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `logcatcher`
--
DROP TABLE IF EXISTS `logcatcher`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `logcatcher` (
`moment` datetime DEFAULT NULL,
`pid` varchar(20) DEFAULT NULL,
`root_pid` varchar(20) DEFAULT NULL,
`father_pid` varchar(20) DEFAULT NULL,
`project` varchar(50) DEFAULT NULL,
`job` varchar(255) DEFAULT NULL,
`context` varchar(50) DEFAULT NULL,
`priority` int(3) DEFAULT NULL,
`type` varchar(255) DEFAULT NULL,
`origin` varchar(255) DEFAULT NULL,
`message` varchar(255) DEFAULT NULL,
`code` int(3) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `logcatcher`
--
LOCK TABLES `logcatcher` WRITE;
/*!40000 ALTER TABLE `logcatcher` DISABLE KEYS */;
/*!40000 ALTER TABLE `logcatcher` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `statcatcher`
--
DROP TABLE IF EXISTS `statcatcher`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `statcatcher` (
`moment` datetime DEFAULT NULL,
`pid` varchar(20) DEFAULT NULL,
`father_pid` varchar(20) DEFAULT NULL,
`root_pid` varchar(20) DEFAULT NULL,
`system_pid` bigint(8) DEFAULT NULL,
`project` varchar(50) DEFAULT NULL,
`job` varchar(255) DEFAULT NULL,
`job_repository_id` varchar(255) DEFAULT NULL,
`job_version` varchar(255) DEFAULT NULL,
`context` varchar(50) DEFAULT NULL,
`origin` varchar(255) DEFAULT NULL,
`message_type` varchar(255) DEFAULT NULL,
`message` varchar(255) DEFAULT NULL,
`duration` bigint(8) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `statcatcher`
--
LOCK TABLES `statcatcher` WRITE;
/*!40000 ALTER TABLE `statcatcher` DISABLE KEYS */;
/*!40000 ALTER TABLE `statcatcher` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;Now that the Talend Activity Monitoring Console database has been created, you can monitor Job executions either from Talend Studio or from the Monitoring page of Talend Administration Center.
To do so, you simply need to tell Talend Studio and Talend Administration Center about the connection parameters to these tables. For more information, refer to Configuring datasources for the monitoring console.