Skip to main content Skip to complementary content

Creating a Snowflake table and populating it with generated data

Example of a pipeline created from the instructions below.

Procedure

  1. Click Connections > Add connection.
  2. In the panel that opens, select the type of connection you want to create.
    Here, select Snowflake.
  3. Select your engine in the Engine list.
  4. Fill in the Snowflake connection properties (authentication type) as described in Snowflake properties, check the connection and click Next.
  5. Enter a description (optional) and a display name (required) for the Data generator connection, and click Validate.
  6. Click Add dataset to create the corresponding Snowflake dataset.
  7. In the Add a new dataset panel, name your dataset.
  8. Fill in the required properties to create a Snowflake table on the fly. Enter a table name that does not exist in the database yet:
    Configuration of a new Snowflake dataset.
  9. Click Validate to save your dataset.
  10. Click Connections > Add connection.
  11. Create a Data generator connection.
  12. Click Add dataset to create the corresponding dataset.
  13. Name your dataset, and fill in the required properties to generate 100 rows with three fields (ID, NAME, and AGE):
    Configuration of a new dataset.
    1. Rows: Enter 100 as you want to generate 100 records.
    2. Add a field:
      Name: ID; Type: Incremental number; Min: 1; Increment: 1
    3. Add another field:
      Name: NAME; Type: First name
    4. Add another field:
      Name: AGE; Type: Random number range; Min: 18; Max: 54
  14. Click Validate to save your dataset.
  15. Click Add pipeline on the Pipelines page. Your new pipeline opens.
  16. Give the pipeline a meaningful name.

    Example

    Create Snowflake table and populate it
  17. Click ADD SOURCE and select your source dataset, the Data generator dataset, in the panel that opens.
  18. Click the ADD DESTINATION item and select the destination dataset, the Snowflake dataset, in the panel that opens.
  19. In the Main tab of the Snowflake Configuration panel:
    1. Select Insert in the Action list, as you want to insert the generated data in the table.
    2. Enable the Create table if not exists option, as you want to create the table on the fly.
    3. Save your changes.
  20. On the top toolbar of Talend Cloud Pipeline Designer, click the Run button to open the panel allowing you to select your run profile.
  21. Select your run profile in the list (for more information, see Run profiles), then click Run to run your pipeline.

Results

Your pipeline is being executed, the Snowflake table is created on the fly, and it is populated with the 100 generated records.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here