CircleCI 

CircleCI is a cloud-based continuous integration and continuous delivery (CI/CD) platform that automates software builds, tests, and deployments triggered by changes in source repositories.

CircleCI integration replicates data using the CircleCI API v2.

Preparing for authentication

To access your data, you need to authenticate the connection with your account credentials.

To set up your CircleCI account, you need:

• A CircleCI account with read access to the projects and organizations whose data you want to replicate.
• A CircleCI Personal API Token. Project API tokens are not supported by the CircleCI v2 API and cannot be used with this connector.
• One or more project slugs in the form vcs-type/org-name/repo-name (for example, gh/acme-org/web-app for a GitHub-hosted repository).

To create your API token:

1. Log into your CircleCI account.
2. Click your avatar in the lower-left corner and select User Settings.
3. Select Personal API Tokens.
4. Click Create New Token.
5. Enter a descriptive name, for example QlikDataIntegration, and click Add API Token.

6. Copy and paste your API token into a secured file.

To identify your project slugs:

• vcs-type — gh for repositories hosted on GitHub, or bb for repositories hosted on Bitbucket.
• org-name — the GitHub organization (or username) or Bitbucket workspace that owns the repository.
• repo-name — the repository name.

You can read the slug directly from a CircleCI pipeline URL. For example, the URL https://app.circleci.com/pipelines/github/acme-org/web-app/123 corresponds to the slug gh/acme-org/web-app.

To replicate from multiple projects in one connection, enter all slugs in the Project Slugs field separated by spaces, for example: gh/my-org/repo-one gh/my-org/repo-two.

Creating the connection

For more information, see Connecting to SaaS applications.

1. Fill in the required connection properties.

2. Provide a name for the connection in Connection name.

3. Select Open connection metadata to define metadata for the connection when it has been created.

4. Click Create.

Limitations and considerations

• Personal API Token only. CircleCI Project API tokens are not supported by the v2 API. The token must belong to a user who has read access to the projects and organizations being replicated. If the token owner loses access (for example, leaves the organization), syncs that depended on that access will return empty results or fail.
• pipelines, workflows, and jobs are scoped to Project Slugs. Only the projects listed in the Project Slugs field are replicated for these three streams. Adding a project requires updating the field and re-saving the connection.
• Organization-scoped streams replicate everything the token can see. The collaborations, context, groups, project, pipeline_definition, schedule, trigger, and deploy streams replicate data for every organization and project visible to the Personal API Token. To limit what these streams return, restrict the access of the underlying CircleCI user.
• workflows is replicated per pipeline. The connector lists pipelines for each configured project, then issues one workflow request per pipeline. Projects with many historical pipelines produce many API calls for the workflows stream on the initial sync; subsequent syncs only fetch newer pipelines.
• jobs is replicated per workflow. Each workflow produces one or more job requests. Projects with many workflows per pipeline can produce a high API-call volume for the jobs stream.
• jobs uses Full Table replication even though pipelines and workflows are incremental. A workflow's job list is fully refreshed on every sync because the CircleCI jobs API does not expose a replication key. As pipeline and workflow bookmarks advance, the set of workflows re-fetched shrinks, but each retained workflow is re-fetched in full.
• Rate-limit responses (HTTP 429) include a Retry-After header; the connector waits for the specified duration before retrying, up to 6 attempts within a 60-second budget. Transient server errors (5xx) are retried with exponential backoff, up to 5 attempts.
• HTTP 404 Not Found responses for individual resources (for example, a deleted pipeline) are treated as empty responses and do not stop the sync. Authentication errors (401) and validation errors (400) are surfaced immediately and fail the sync.
• Start Date is honored only on the initial sync of each stream. Once a stream has a bookmark in state, subsequent syncs resume from that bookmark; changing Start Date in the connection settings does not retroactively affect already-replicated data.
• Interrupted syncs resume per-project. The connector records a currently_syncing marker for the project or pipeline in progress when a sync is interrupted, so the next sync resumes from that point.

Schema

Depending on your destination, table and column names may not appear as they are outlined below.