Event Monitoring deployment options
The Event Monitoring can be deployed using the following options.
-
Talend Runtime (EM Agent) - Local
This is the most easy-to-use setup as it only requires the Talend Runtime with the tesb:start-em-default command to be started.
With the local deployment option the Event Monitoring data will be just stored in the {KARAF_DATA}/eventmonitoring/em.log file (including up to 10 rolling files names em.log.1 to em.log.9)
The log data can be either consumed locally or captured by a third party log collection product from this file.
-
Talend Runtime (EM Agent) + Talend Log
Server
The combination of the Talend Runtime which produce the Event Monitoring logs with the Talend LogServer is the standard deployment option that is recommended to use.
It also requires the Talend Runtime with the tesb:start-em-default command to be started. The Talend LogServer Filebeat with a preconfigured /etc/filebeat_em.yml captures the events from the local em.log file and sends them to the Talend LogServer (Elastic Search). When the events reach the ElasticSearch Server they can be searched and visualized within Talend Administration Center from the Logging page (based on Kibana).
- Talend Runtime (EM Agent) + JMS Message Broker + Talend Runtime (EM Proxy) + Talend Log
Server
With this deployment option, a more limited setup needs to be done on each Talend Runtime which produces the Events.
The events will be sent to a central JMS Broker (Apache ActiveMQ as included in Talend ESB) and then another central Talend Runtime which acts as a EM Proxy will read the events from the JMS Broker and store them in the em.log as described above. From here you can either use the Talend LogServer Filebeat and the related configuration, or use a third Party log management tool to get the events from this central instance. In addition to the limited installation of Filebeat on the EM Proxy only, the JMS Broker in between also provides a scalable and robust buffering of log events while they get transferred to the backend.