GitLab

GitLab is an all-in-one platform designed for software development and DevOps.

GitLab integration replicates data using the GitLab REST API.

See the Schema section for a list of objects available for replication.

A high-level look at this connector, including useful links, and supported features.

Feature/Capability	Support details
Supported Qlik Talend Data Integration projects	Replication projects only. Data pipeline projects are not supported.
Target update methods	Replication tasks: Apply changes Store changes Landing data in a data lake tasks: Change data capture (CDC)
Managing metadata	Manual metadata generation is not required.
Schema evolution	Only the Change column data type operation is supported.
Replication of LOB columns (NCLOB, CLOB, and BLOB)	Not supported.
Scheduled CDC	Required. This is how the target is kept up-to-date with changes to the source. For replication tasks, see Scheduling tasks For lake landing tasks, see Scheduling CDC for lake landing tasks
Notifications	Partially supported Setting notifications for changes in operation
Monitoring	CDC-only, as full load is not relevant for this connector. Monitoring an individual data task
Automatic denesting of JSON column payloads	Not supported. JSON column payloads in source datasets are not denested automatically on the target.

Preparing for authentication

To access your data, you need to authenticate the connection with your account credentials.

Make sure that the account you use has read access to the tables you want to fetch.

Creating the connection

For more information, see Connecting to SaaS applications.

Fill in the required connection properties.
Provide a name for the connection in Connection name.
Select Open connection metadata to define metadata for the connection when it has been created.
Click Create.

Connection settings
Setting	Description
Data gateway	Select a Data Movement gateway if required by your use case. Information note This field is not available with the Qlik Talend Cloud Starter subscription, as it does not support Data Movement gateway. If you have another subscription tier and do not want to use Data Movement gateway, select None. For information on the benefits of Data Movement gateway and use cases that require it, see Qlik Data Gateway - Data Movement.
Start Date	Enter the date, in the format `MM/DD/YYYY`, from which the data must be replicated from your source to your target.
API URL	API URL.
Private Token	Private token.
Projects to track	Enter the projects you want to track separated by spaces.
Groups to track	Enter the groups you want to track separated by spaces.

Schema

Schemas and naming conventions can change from version to version. It is recommended to verify your version before continuing.

The schema and info displayed below is for version 1.

Depending on your destination, table and column names may not appear as they are outlined below.

branches

The branches table contains high-level info about repository branches in your projects.

Note: To replicate branch data, you must set this table and the projects table to replicate. Data for this table will only be replicated when the associated project (in the projects table) is also updated.

Replication Method

CDC

Primary Keys

project_id

name

Useful links

GitLab API method

Join branches with	on
commits	branches.project_id = commits.project_id branches.commit_id = commits.id
issues	branches.project_id = issues.project_id
milestones	branches.project_id = milestones.project_id
projects	branches.project_id = projects.project_id

commit_id

STRING

developers_can_merge

BOOLEAN

developers_can_push

BOOLEAN

merged

BOOLEAN

name

STRING

project_id

INTEGER

protected

BOOLEAN

commits

The commits table contains info about repository commits in a project.

Note: To replicate commit data, you must set this table and the projects table to replicate. Data for this table will only be replicated when the associated project (in the projects table) is also updated.

Replication Method

CDC

Primary Key

Useful links

GitLab API method

Join commits with	on
branches	commits.project_id = branches.project_id commits.id = branches.commit_id
issues	commits.project_id = issues.project_id
milestones	commits.project_id = milestones.project_id
projects	commits.project_id = projects.project_id

allow_failure

BOOLEAN

author_email

STRING

author_name

STRING

committer_email

STRING

committer_name

STRING

created_at

DATE-TIME

STRING

message

STRING

project_id

INTEGER

short_id

STRING

title

STRING

groups

The groups table contains info about the groups in your GitLab account.

Replication Method

Full Table

Primary Key

Useful links

GitLab API method

avatar_url

STRING

description

STRING

full_name

STRING

full_path

STRING

INTEGER

lfs_enabled

BOOLEAN

name

STRING

path

STRING

projects

ARRAY

This is a nested data structure:

id: INTEGER

request_access_enabled

BOOLEAN

visibility_level

INTEGER

web_url

STRING

issues

The issues table contains info about issues contained within projects.

Replication Method	CDC
Primary Key	id
Replication Key	updated_at
Useful links	GitLab API method

Join issues with	on
branches	issues.project_id = branches.project_id
commits	issues.project_id = commits.project_id
milestones	issues.project_id = milestones.project_id issues.milestone_id = milestones.id
projects	issues.project_id = projects.project_id

assignee_id

INTEGER

author_id

INTEGER

confidential

BOOLEAN

created_at

DATE-TIME

description

STRING

due_date

STRING

INTEGER

iid

INTEGER

labels

ARRAY

This is a nested data structure.

milestone_id

INTEGER

project_id

INTEGER

state

STRING

subscribed

BOOLEAN

title

STRING

updated_at

DATE-TIME

user_notes_count

INTEGER

web_url

STRING

milestones

The milestones table contains info about project milestones.

Note: To replicate milestone data, you must set this table and the projects table to replicate. Data for this table will only be replicated when the associated project (in the projects table) is also updated.

Replication Method	CDC
Primary Key	id
Replication Key	updated_at
Useful links	GitLab API method

Join milestones with	on
branches	milestones.project_id = branches.project_id
commits	milestones.project_id = commits.project_id
issues	milestones.project_id = issues.project_id milestones.id = issues.milestone_id
projects	milestones.project_id = projects.project_id

created_at

DATE-TIME

description

STRING

due_date

STRING

group_id

INTEGER

iid

INTEGER

project_id

INTEGER

start_date

STRING

state

STRING

title

STRING

updated_at

DATE-TIME

projects

The projects table contains info about specific projects.

Replication Method	CDC
Primary Key	id
Replication Key	last_activity_at
Useful links	GitLab API method

Join projects with	on
branches	projects.project_id = branches.project_id
commits	projects.project_id = commits.project_id
issues	projects.project_id = issues.project_id
milestones	projects.project_id = milestones.project_id
users	projects.creator_id = users.id

approvals_before_merge

INTEGER

archived

BOOLEAN

avatar_url

STRING

builds_enabled

BOOLEAN

container_registry_enabled

BOOLEAN

created_at

DATE-TIME

creator_id

INTEGER

default_branch

STRING

description

STRING

forks_count

INTEGER

http_url_to_repo

STRING

INTEGER

issues_enabled

BOOLEAN

last_activity_at

DATE-TIME

lfs_enabled

BOOLEAN

merge_requests_enabled

BOOLEAN

name

STRING

name_with_namespace

STRING

namespace

OBJECT

This is a nested data structure:

id: INTEGER
kind: STRING
name: STRING
path: STRING

only_allow_merge_if_all_discussions_are_resolved

BOOLEAN

only_allow_merge_if_build_succeeds

BOOLEAN

open_issues_count

INTEGER

owner_id

INTEGER

path

STRING

path_with_namespace

STRING

permissions

OBJECT

This is a nested data structure:

group_access: OBJECT, which is also a nested data structure.
project_access: OBJECT, which is also a nested data structure.

public

BOOLEAN

public_builds

BOOLEAN

request_access_enabled

BOOLEAN

shared_runners_enabled

BOOLEAN

shared_with_groups

ARRAY

This is a nested data structure:

group_access_level: INTEGER
group_id: INTEGER
group_name: STRING

snippets_enabled

BOOLEAN

ssh_url_to_repo

STRING

star_count

INTEGER

tag_list

ARRAY

This is a nested data structure.

visibility_level

INTEGER

web_url

STRING

wiki_enabled

BOOLEAN

users

The users table contains info about the users in your GitLab account.

Replication Method

Full Table

Primary Key

Useful links

GitLab API method

Join users with	on
projects	users.id = projects.creator_id

avatar_url

STRING

INTEGER

name

STRING

state

STRING

username

STRING

web_url

STRING

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here