tKuduOutput

Select this check box and in the Component List drop-down list, select the desired connection component to reuse the connection details you already defined.

Then enter the locations and the listening ports of the master nodes of the Kudu service to be used.

This component supports only the Apache Kudu service installed on Cloudera.

For compatibility information between Apache Kudu and Cloudera, see the related Cloudera documentation: Compatibility Matrix for Apache Kudu.

• A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. When you create a Spark Job, avoid the reserved word line when naming the fields.

  • Built-In: You create and store the schema locally for this component only.

  • Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs.

• Information noteNote: The schema of a Kudu table must declare a primary key comprised of one or more columns. These columns must be non-nullable, and may not be of type boolean, float or double.

  Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion.

    If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the Repository Content window.

Action on table

Select an operation to be performed on the table defined.

• None: No operation is carried out.

• Drop and create table: The table is removed and created again.

• Create table: The table does not exist and gets created.

• Create table if does not exist: The table is created if it does not exist.

• Drop table if exist and create: The table is removed if it already exists and created again.

Action on data

Select an action to be performed on data of the table defined.

• Insert: Add new entries to the table. If duplicates are found, the Job stops.

• Update: Make changes to existing entries.

• Upsert: Update the record with the given reference. If the record does not exist, a new record would be inserted.

• Delete: Remove entries corresponding to the input flow.

For more information about Kudu tablets and Kudu replication policies, see Distribution and Fault Tolerance.

1. Click the [+] button to add the columns based on which rows are hash-partitioned, for example, add a host column and a metric column.

   These columns must exist in your data and in the schema you have defined in tKuduOutput.

2. In Number of buckets, enter, without double quotation marks, the number of the buckets to be used to store the partitions. These buckets are created on the fly.

At runtime, rows are distributed by hash value in one of those buckets. If you leave this Hash partitions table empty, hash partitioning is not applied during the creation of the table.

For more information about hash partitioning in Kudu, see Hash partitioning.

1. Click the [+] button to add the primary key columns based on which rows are partitioned to contiguous segments, for example, add a time column.

   These columns must exist in your data and in the schema you have defined in tKuduOutput.

2. In N. Partition, enter, without double quotation marks, a number to be used as the ID number of the partition to be created. For example, enter 1 to create Partition 1.
3. In Lower boundary and Upper boundary, enter the boundaries between which rows are partitioned. The upper boundary is excluded and only the data between the boundaries (including the lower boundary) is written to the Kudu table.

   For example, if you are partitioning your data based on a time column of Date type and the format of your time data is yyyy-mm-dd, you can set the lower boundary to be 2016-01-01 by entering, without double quotation marks, TalendDate.parseDate("yyyy-MM-dd", "2016-01-01") and do the same to set the upper boundary to be 2018-01-01, then only rows with dates between the two boundaries enter this partition.

At runtime, rows of these columns are distributed using the values of the columns you have added to this Range partitions table. If you leave this table empty, range partitioning is not applied during the creation of the table.

For more information about range partitioning in Kudu, see Range partitioning.

Select the check box to stop the execution of the Job when an error occurs.

This component is used as an end component and requires an input link.

This connection is effective on a per-Job basis.