Logical Architecture
For more information about the Talend Cloud Logical / Functional Architecture, see:
Talend Cloud: Talend Cloud product architecture
Talend Cloud Pipeline Designer: Talend Cloud Pipeline Designer architecture
Talend Cloud API Services Platform: Talend Cloud API Services Platform product architecture
Talend Cloud Data Preparation: Talend Cloud Data Preparation architecture
Talend Cloud Data Stewardship: Talend Cloud Data Stewardship architecture
Talend Cloud Data Catalog: Talend Cloud Data Catalog architecture
Talend Cloud Data Inventory: Talend Cloud Data Inventory architecture
| Component name | Description | Link to the documentation |
|---|---|---|
| Talend Studio | Talend Studio is a rich client comprising of comprehensive set of tools and functions that allows users to create projects using the graphical User Interface and hundreds of pre-built components and connectors, and design Jobs or Routes with a drag-and-drop interface and native code generation. | What is Talend Studio? |
| Talend Management Console (TMC) | TMC is an administrative web interface that allows you to create projects, users, and user roles, manage and run tasks and plans from Jobs, Data Services, and Routes published from Talend Studio and run them directly in the cloud or on Remote Engines, ensuring the security of your data. | Talend Management Console User Guide |
| Talend Remote Engine | A remote engine is an on-premises execution environment that can be installed behind your firewall or security processes within your corporate infrastructure, giving the Talend Studio access to local resources, such as files, databases, or other applications. The deployment of Routes and Data Services is supported on Talend Remote Engine from the Fall '18 release. OSGi type deployments require that Talend Runtime 7.1.1 or a later version is installed and running on the same machine as the Remote Engine. Microservices do not require any other application for Route and REST data service deployments, except the Data Service Runner module inside the Remote Engine. | Configuring Talend Remote Engine |
| Talend Remote Engine Gen2 | Distributed as a docker container, the Remote Engine Gen2 is used by Talend Cloud Data Inventory, Talend Cloud Data Preparation and Talend Pipeline Designer. It performs data discovery and data processing next to your data sources. You have control over your execution environment and resources as you are able to create and configure the engine in your own environment (Virtual Private Cloud or on premises). The engine also ensures optimal performance and security by increasing the data locality instead of moving large data to computation. You can execute pipelines with different resource allocations on a Remote Engine Gen2 by using run profiles that contain the right resource allocation and set of configurations for executing pipelines according to your license type. | Talend Remote Engine Gen2 Quick Start Guide |
| Cloud Engine for Design | The Cloud Engine for Design is a built-in runner that allows you to easily design pipelines without
having to set up any processing engines. You can execute pipelines with
different resource allocations on a Cloud Engine for Design by using run profiles. The following resource limitations are valid for
pipelines running on Cloud Engine for Design:
|
Execution engines in Talend Cloud |
| Talend Cloud Pipeline Designer | Talend Cloud Pipeline Designer provides a graphical web-based user interface in the cloud to create complex end-to-end pipelines to process data at rest or in motion instantly through the schema-on-read framework. Pipelines consist of a process (similar to a Talend Job) that keeps on listening to incoming data, and a pipe where data comes from a source, the dataset, and is sent to a destination. | Talend Cloud Pipeline Designer User Guide |
| Talend Cloud API Designer/Tester | Talend Cloud API Designer allows you to design, document and share API definitions. By combining it with Talend Cloud API Tester, Talend Studio and Talend Management Console, you can design, document, test, implement and deploy your APIs. APIs enable distributed software to communicate over a network by exchanging messages. REST is a popular style of API where client and server exchange http messages such as GET, POST and DELETE. | Introduction to Talend Cloud API Designer |
| Talend Cloud Data Preparation (TDP) | Talend Cloud Data Preparation is a self-service application that helps in Data Discovery, Profiling, Cleaning, Enrichment and Standardizing Datasets for Data Analysis and other Data Driven tasks done by business users. | What is Talend Cloud Data Preparation? |
| Talend Cloud Data Stewardship (TDS) | Talend Cloud Data Stewardship enables the implementation of well-defined data governance processes covering several activities including monitoring, reconciliation, refining, de-duplication, cleansing and aggregation to help deliver quality data to applications and end users. The main units of work in Talend Cloud Data Stewardship are Campaigns (Arbitration, Merging & Resolution) and Tasks which pass through different states according to workflow defined in the campaign. | Talend Cloud Data Stewardship introduction |
| Talend Cloud Data Inventory | Talend Cloud Data Inventory maintains an inventory of data assets managed within Talend Cloud in the form of Datasets. Datasets are added to this inventory whenever they are collected or created using Talend Cloud Pipeline Designer or Talend Cloud Data Preparation. These datasets are automatically profiled, associated with a trust score based on their underlying data quality, crowd-sourced ratings and popularity, and documented with metadata. They can then be shared so that other users in the Talend Cloud platform can easily find them through faceted search, while authorized users can add their own endorsements and custom metadata attributes. | What is Talend Cloud Data Inventory? |
| Talend Dictionary Service | Talend Dictionary Service allows campaign owners to manage the semantic types used by the Talend Data Governance Applications. | What is Talend Dictionary Service? |
| Talend SAP RFC Server | This is a standalone server that acts as a central gateway between Talend Studio and an SAP Server for SAP IDocs or SAP BW Data Source objects processing. It receives these objects from the SAP server and makes them available for processing in Talend Jobs using various Talend Studio components. | Configuring SAP to operate with the Talend SAP RFC Server |
| Metadata Repository | Git repositories are used to store project metadata. | Implementation with Talend |
| Artifact Repository | Stores the third party libraries required to create and execute Talend Artifacts. | Implementation with Talend |
| Talend CI Builder | CI Builder is a maven plugin for build / DevOps tools that allows the automation of the build and test of Talend artifacts. | Storing the CI Builder plugin in Nexus |