Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

Centralizing BigQuery metadata

You can use the BigQuery metadata wizard in Talend Studio to create a reusable Google BigQuery connection and retrieve table schemas for your Jobs.

Procedure

  1. In the Repository tree view, expand Metadata, right-click the Google BigQuery node, and select the option to create a new BigQuery connection.
  2. In the BigQuery metadata wizard, enter a name and a description for the connection.
  3. Click Next.
  4. Provide the connection settings required for your Google BigQuery environment:
    Property Description
    Authentication mode Select the mode to be used to authenticate to your project:
    • Service account: authenticate using a Google account that is associated with your Google Cloud Platform project. When selecting this mode, the credentials must be provided in the Service account credentials file field.
    • Workload Identity Federation: authenticate through a Google Workload Identity Pool, allowing integration with identity providers such as AWS, Azure, or OIDC, without needing a service account key. When selecting this mode, the identity federation configuration must be provided in the Client Library Configuration File field.
    • Application Default Credentials: authenticate using the Application Default Credentials. When selecting this mode, no additional parameters need to be defined as credentials are automatically found based on the application environment.
    • OAuth2 - Authorization Code grant: authenticate using OAuth credentials. When selecting this mode, the OAuth credentials must be provided in the Client ID, Client Secret, and Authorization Code fields.
    • OAuth Access Token: authenticate using an OAuth access token. When selecting this mode, the token must be provided in the OAuth Access Token field.

    For more information about Google Cloud authentication, see Google Cloud authentication documentation.
    Project ID Paste the ID of the project hosting the Google BigQuery service you need to use.

    The ID of your project can be found in the URL of the Google API Console, or by hovering over the name of the project in the BigQuery Browser Tool.
    Service account credentials file Enter the path to the credentials file created for the service account to be used. This file must be stored in the machine in which your Talend Job is actually launched and executed.

    This property is only available when you authenticate using Service account.
    Use a custom endpoint Select this check box to use a private endpoint rather than the default one.

    When selected, enter the URL by respecting the following format: "https://bigquery.googleapis.com".

    For more information, see Access Google APIs through endpoints in the Google Cloud documentation.

    This property is only available when you authenticate using Service account.
    Client ID and Client Secret Paste the client ID and client secret for the OAuth client configured to access Google BigQuery and Cloud Storage.

    This property is only available when you authenticate using OAuth2 - Authorization Code grant.
    Redirect URI Enter the redirect URI configured for your OAuth client in Google Cloud. This value must match the authorized redirect URI exactly.

    This property is only available when you authenticate using OAuth2 - Authorization Code grant.
    Authorization Code Paste the authorization code provided by Google for the access you are configuring.

    To obtain the authorization code for metadata configuration, leave this field blank and click Check Connection. Open the URL displayed by the Studio, authorize the request, and then copy and paste the returned authorization code in this field.
    OAuth Access Token Enter an access token.

    The lifetime of the token is one hour. The component does not perform the token refresh operation but will fetch the new access token to operate beyond the one-hour limit.

    This property is only available when you authenticate using OAuth Access Token.
  5. Click Check Connection to verify your settings, and then click Next.
  6. Select the tables whose schemas you want to retrieve, and then click Finish.
    The new BigQuery connection appears under the BigQuery node in the Repository tree view.

Results

You can drag and drop the BigQuery connection or one of its schemas from the Repository view to the design workspace to reuse connection and schema information. For more information, see Using centralized metadata in a Job.

To update the connection metadata, right-click the connection in the Repository tree view and select the edit option for BigQuery metadata.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here