Declarative pipelines
Last updated: 2026-11-20Declarative pipelines let you define, configure, and deploy Qlik Talend Data Integration pipeline projects entirely in code using YAML files. You edit the files in any integrated development environment or version control system, then deploy them to create or update projects.
When to use declarative pipelines
Use declarative pipelines to promote:
-
Collaboration
-
Continuous Integration / Continuous Delivery
-
Repeatable deployment to different environments
-
API-driven automation
Here are some typical use cases:
-
Define pipelines in code for continuous integration and development automation.
-
Deploy identical pipeline configurations across multiple environments (development, verification, production).
-
Reuse and share pipeline definitions across projects or teams.
-
Automate large-scale project creation without manual work.
How it works
A pipeline project is represented as a set of YAML files organized in a folder structure that mirrors the project's tasks and datasets. YAML files are easier to read, see diffs, review, reuse, and create automation.
You can export an existing project as YAML, edit the files, and import them back to apply your changes. If the project is connected to version control, the YAML files are directly available in your GitHub repository.
For more information about the YAML file structure, see Retrieving and editing a pipeline configuration.
Typical workflow
Before you start, you must set up your development environment and version control. For more information, see Getting started with declarative pipelines.
A typical declarative pipeline workflow looks like this:
-
Retrieve the pipeline configuration
You can export an existing project as YAML, retrieve it from version control, or start from a template.
-
Edit the pipeline configuration
Modify the YAML files in your development environment. You can add tasks, change source selections, and update settings.
-
Validate the pipeline configuration
Check the files for syntax and semantic errors before deploying the pipeline.
-
Deploy the pipeline configuration
Import the files to create or update the project in Qlik Talend Data Integration.
You can also commit the files to version control and perform apply from version control in the project in Qlik Talend Data Integration.
Relationship to version control
When a project is connected to version control, the YAML files are stored directly in your GitHub repository. You can edit the YAML files in GitHub or in any IDE, and then apply the changes back to the project.
For information about connecting a project to version control, see Manage your pipeline projects with version control.
Using AI assistants to configure pipelines
Use AI assistants (such as GitHub Copilot or Anthropic Claude) to streamline YAML configuration creation and editing. To ensure accuracy and adherence to project standards, follow these best practices:
-
Select a high-capability model: Use a premium, large-context model (for example, Anthropic Claude Sonnet 5.5 or higher) rather than a standard or "auto" model for complex YAML generation and schema adherence. Advanced models correctly interpret complex pipeline schemas and produce more accurate configurations.
-
Install the declarative pipelines skills: Install the declarative pipelines skills in your AI coding assistant of choice: Claude Code, GitHub Copilot, or Kiro.
-
Claude Code
You can install the skills from the VS Code plugins screen or from a terminal.
Option 1: VS Code plugins screen
-
In the Claude prompt box, type
/pluginsand select Manage plugins. -
On the Marketplaces tab, enter qlik-oss/agentic-skills in the GitHub repo box and click Add.
-
On the Plugins tab, search for qlik-data-pipelines-skills, click Install, and select a scope.
Option 2: Terminal
Open a terminal, either in VS Code or separately, and run the following commands:
claude plugin marketplace add qlik-oss/agentic-skills claude plugin install qlik-data-pipelines-skills@qlik-cloud-skillsAlternatively, run
claudeto start an interactive session, and then enter the following commands:/plugin marketplace add qlik-oss/agentic-skills /plugin install qlik-data-pipelines-skills@qlik-cloud-skillsWhen prompted, select a scope.
-
-
GitHub Copilot
You can install the skills from the GitHub Copilot prompt box or from a terminal.
Option 1: Copilot prompt box
In the GitHub Copilot prompt box, enter the following commands:
/plugin marketplace add qlik-oss/agentic-skills /plugin install qlik-data-pipelines-skills@qlik-cloud-skillsOption 2: Terminal
Open a terminal, either in VS Code or separately, and run the following commands:
copilot plugin marketplace add qlik-oss/agentic-skills copilot plugin install qlik-data-pipelines-skills@qlik-cloud-skillsAlternatively, run
copilotto start an interactive session, and then enter the following commands:/plugin marketplace add qlik-oss/agentic-skills /plugin install qlik-data-pipelines-skills@qlik-cloud-skillsWhen prompted, select a scope.
-
Kiro
-
Select the Powers icon (the Kiro ghost).
-
Click Add custom power and select Import power from GitHub.
-
Enter the following URL: https://github.com/qlik-oss/agentic-skills/tree/main/data-pipelines.
-
-
Using AI assistants with Qlik MCP tools
When you deploy a Qlik MCP server in your tenant, AI assistants can use dedicated MCP tools to help manage pipeline projects programmatically.
Prerequisites
-
Qlik MCP server is deployed in your tenant
-
You have the Qlik MCP: Allowed permission assigned through a custom role
-
You have permissions to create or edit pipeline projects
-
An AI client with MCP support (such as VS Code with GitHub Copilot) is configured to use the Qlik MCP server
Available MCP capabilities for data pipelines
AI assistants can use the following capabilities to configure and manage pipeline projects:
-
Search for spaces: Get a list of available spaces.
-
Search for connections: Get a list of available data connections.
-
Search for projects: Get a list of existing pipeline projects.
-
Search for connection objects: Discover available tables and views from a data source.
-
Get pipeline project details: Retrieve project binding values.
Related resources
-
To deploy the MCP server, see Qlik MCP server
-
For detailed MCP tool documentation, see Qlik MCP tools
-
For permission requirements, see Qlik MCP server access and permissions