Skip to main content Skip to complementary content

tIcebergCatalog properties for Apache Spark Batch

These properties are used to configure tIcebergCatalog running in the Spark Batch Job framework.

The Spark Batch tIcebergCatalog component belongs to the Big Data family.

The component in this framework is available in all Talend products.

This component is available when you have installed the 8.0.1-R2024-04 Studio Monthly or a later one provided by Talend. For more information, check with your administrator.

Basic settings

Register new catalog Select this parameter to register a new catalog.
Replace default spark_catalog with Iceberg's SparkSessionCatalog Select this parameter to replace the default session Spark Catalog. For more information, see Replacing the session catalog from Iceberg documentation.
Name Enter the name of the catalog.
Type Select the type of catalog to be created from the drop-down list:
  • Hadoop: creates an Iceberg catalog loading tables from a Hadoop warehouse.
  • Hive: creates an Iceberg catalog loading tables from a Hive metastore.
  • Rest: creates an Iceberg catalog loading table from a REST metastore.
  • Custom: creates an Iceberg catalog loading tables from a custom metastore.
Warehouse path Enter the root path to the warehouse in which the catalog is stored.

This parameter is only available when you select Hadoop from the Type drop-down list.

Uri Enter the metastore URI string.

This property is only available when you select Hive or Rest from the Type drop-down list.

Catalog implementation Enter the custom catalog implementation to be used.

This property is only available when you select Custom from the Type drop-down list.

Catalog properties Enter any extra properties to be used for the catalog. For more information, see Configuration from the Iceberg documentation.

This property is only available when you select the Register new catalog parameter.

Global Variables

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After variable and it returns a string.

Usage

Usage rule This component is generally used with other Iceberg components.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!