Known limitations of Dynamic Engine
The Dynamic Engine solution
still has the following known limitations:
- Dynamic Engine supports only data integration and big data Job tasks. Routes and data service Jobs are not supported.
- Dynamic Engine in the Azure United States - West region (us-west) is still in Beta.
- Each Kubernetes cluster can only have one Dynamic Engine by design. A Dynamic Engine provides the core infrastructure services for task and plan executions.
- There is no mechanism to prevent deploying more than one Dynamic Engine in a Kubernetes cluster. Doing this may cause unexpected connectivity and performance issues for Talend Management Console, Dynamic Engines, and Dynamic Engine environments.
- You must use the manifest files generated by Talend Management Console to deploy Dynamic Engines and Dynamic Engine environments. Custom manifest files are not supported.
- The Kubernetes cluster in which Dynamic Engine is deployed must have access to public internet.
- The following task execution features for data integration Jobs are not supported:
- Remote debugging of a Data Integration Job on Dynamic Engines
- User impersonation
- Custom signature check
- Use cloud storage components such as the S3 or the Azure components to transfer
the data for processing.
These components retrieve data from external storage into the container running the tasks, and push output data to external storage after execution. This is necessary because the downloaded data in the container gets destroyed along with the container after execution completes.
Local file system components like tFileInputDelimited and tFileOutputDelimited can only access the data temporarily downloaded during task execution, and cannot be used for persistent data access. - The tSystem component is not supported.