Talend Big Data architecture
Each of the operating principles can be isolated in different functional blocks. The following diagram describes the different types of blocks and their interoperability:
Building and administrating
The CLIENTS block includes one or more Talend Studio APIs and Web browsers that can be on the same or on different machines.
From the Talend Studio API, end-users can carry out technical processes regardless of data volume and process complexity.
Talend Studio allows users to work on any project for which they have authorization. For more information, see Creating a project.
From a Web browser, end-users connect to the remotely based Talend Administration Center through a secured HTTP protocol. The end-user category in this description may include developers, project managers, administrators, and any other person involved in building data flows.
Each of these end-users will use either Talend Studio or Talend Administration Center or both of them depending on the company policy.
Additionally, from the Web Browser you access the Talend Data Preparation Web application. This is where you import your data, from local files or other sources, and cleanse or enrich it by creating new preparations on this data. You can also access the Talend Data Stewardship Web application. This is where campaign owners and data stewards manage campaigns and tasks.
The TALEND SERVERS and DATABASES blocks and the Git gray circle include a web-based Talend Administration Center (application server) connected to two shared repositories: one based on a Git server and one based on a database server (Admin).
Talend Administration Center also enables to configure the tasks that handle Job executions and triggers. It also looks after the Job generation and deployment to the execution servers. For more information, see Getting started with Talend Administration Center.
Talend Administration Center also includes the servers used by the Talend Web applications, namely Talend Data Preparation and Talend Data Stewardship. The Talend Identity and Access Management server is used to enable Single Sign-On between those applications.
Deploying and executing
The Artifact Repository gray circle represents the artifact repository that stores all the Software Updates available for download.
- One or more Talend Runtime (execution container) deployed inside your information system. Talend Runtime deploys and executes the technical processes according to the set up defined in the Talend Administration Center Web application. Those processes are Jobs built from Talend Studio and centralized on the Git server.
- One or more Talend JobServer deployed inside your information system that run technical processes
(Jobs) according to scheduled time, date, or event set in the Talend Administration Center Web application.
The end-user can transfer technical processes to a remote execution server directly from Talend Studio (distant run).
Information noteImportant:You must install the Talend JobServer files ("Agent"), delivered by Talend, on each of the execution servers to become operational.
Monitoring
The Monitoring circle represents the monitoring: Talend Activity Monitoring Console.
Talend Activity Monitoring Console allows end-users to monitor the execution of technical processes. It provides detailed monitoring capabilities that can be used to consolidate log information collected, understand the interaction between underlying data flows, prevent faults that could be unexpectedly generated and support system management decisions. For more information on Talend Activity Monitoring Console, see Talend Activity Monitoring Console User Guide