Webhooks in Talend Management Console
Use webhooks to trigger Job tasks or plans based on external application calls.
- Do not include sensitive credentials in the webhook trigger request, because it does not need any authentication.
- Keep the webhook URL secure, as it can be used by anyone to trigger your tasks.
- If authentication is required, consider using an API instead of a webhook trigger.
You cannot add a webhook if you already have other trigger types. To add a webhook, delete existing triggers or duplicate the task or plan if both types of triggers are needed.
Webhook-specific parameters
To use a webhook, the source Job must define the resource_webhook_payload context parameter. For plans, at least one task in the first step must include a source Job with this parameter.
Payloads are processed as arrays, with a maximum size of 5 KB for Big Data Jobs. Supported payload types include JSON, XML, and plain text. For further information, see Webhook payload.
When generating a webhook URL in Talend Management Console, configure the following parameters:
-
Window threshold: Defines the number of service calls to group
within a time window.
For example, if the window size is 2, during the time set in the Window threshold parameter, two consecutive payloads are sent to the webhook endpoint:
{"param":"1"} {"param":"2"}
However, the task or plan execution does not receive them as two separate payloads, it receives them as an array:[ {"data":"{"param":"1"}","type":"application/x-www-form-urlencoded"}, {"data":"{"param":"2"}","type":"application/x-www-form-urlencoded"} ]
- Window duration(in seconds): Specifies the time frame (in seconds) for grouping service calls.
The first event triggers the execution.
job.ctx.params.passthrough=true
A successful webhook call returns an HTTP 204 (No Content) response.
- Salesforce: Calls with Salesforce-specific payloads (for example, containing the soap.sforce.com references) respond with 200 OK instead of 204 No Content, even when the response body is empty.
- Dropbox: When the DropboxWebhooks/1.0 user-agent header is included in
the request, the webhook responds with 200 OK and includes the value of
the challenge parameter in the response body. For example, for a call
similar to this
one:The response body looks like:
GET https://webhooks.eu.cloud.talend.com/mywebhook/11fe60a514414a2a8b938b29084c4a29?challenge=parameter User-Agent: DropboxWebhooks/1.0
Response: parameter Content-Type: text/plain;charset=UTF-8 Content-Length: 9
Webhook trigger workflow
-
Set up the webhook context parameter.
You define a specific context parameter, resource_webhook_payload, in the Job to process incoming webhook data.
-
Generate a webhook URL.
You generate a webhook URL in Talend Management Console and configure Window threshold (number of calls) and Window duration (time frame in seconds).
-
Share the webhook URL.
You provide the generated webhook URL to external systems or applications that use it to send payloads.
-
External system calls the webhook URL.
Payloads are sent to the webhook endpoint by external systems.
-
Talend Management Console processes the payloads for using them in your Job.
Incoming payloads are parsed and converted into an array that conforms to the schema expected by the resource_webhook_payload variable.
-
The task or plan execution is triggered.
The webhook triggers the execution according to the configured parameters.
-
Webhook URL returns a response to the external system.
The webhook endpoint sends an HTTP 204 No Content or 200 OK response.