How Jobs are handled via Talend Administration Center
Talend Jobs are scheduled from Talend Administration Center, which provides a browser management interface, and are parameterized with Context variables.
But often you may prefer to have programmatic control of Jobs via an API. This post covers how to expose Talend Jobs via the Talend Administration Center API. It provides sample Jobs, some useful browser utilities, and an example of wrapping the API in a RESTful service layer using Data Services.
The Job Conductor page of Talend Administration Center is easy to use and powerful: its allows you to schedule Jobs with simple, more complex Cron triggers, or file triggers. You also have the possibility to launch Jobs manually from the browser. With exception of the file trigger, the Job is still running at a pre-determined time or with explicit human intervention. Using file triggers as the mechanism for inter-process communication may require access privileges which are not allowed in a secure environment, which is why it may be preferable to invoke a Talend Job via a real API.
Jobs can also be parametrized with Context variables. Context variables can be overridden by system administrators on the Job Conductor page to provide additional flexibility. But when the Job is running, it always runs with the same set of pre-configured Context variables, whether they are the default values or the overridden values, they cannot be changed without human intervention. It is preferable to be able to pass parameters via an API.
One option is to build Jobs as self-contained .zip files. The generated archives will include launching scripts and all necessary .jar files. However, the resulting Jobs are running in isolation and lack the monitoring, management, and control parts provided by Talend Administration Center. No centralized logging is provided, and there is no concept of Job Servers or a Job grid. Instead, these responsibilities fall on the developer. As individual solutions proliferate, the management of the broader system becomes more difficult and the maintenance tail becomes more unwieldy.
So while exported Jobs provide flexibility, they sacrifice manageability. The Talend Administration Center API provides a very simple and powerful alternative.