tIcebergCatalog properties for Apache Spark Batch
These properties are used to configure tIcebergCatalog running in the Spark Batch Job framework.
The Spark Batch tIcebergCatalog component belongs to the Big Data family.
The component in this framework is available in all Talend products.
This component is available when you have installed the 8.0.1-R2024-04 Studio Monthly or a later one provided by Talend. For more information, check with your administrator.
Basic settings
| Register new catalog | Select this parameter to register a new catalog. |
| Replace default spark_catalog with Iceberg's SparkSessionCatalog | Select this parameter to replace the default session Spark Catalog. For more information, see Replacing the session catalog from Iceberg documentation. |
| Name | Enter the name of the catalog. |
| Type | Select the type of catalog to be created from the drop-down list:
|
| Warehouse path | Enter the root path to the warehouse in which the catalog is stored. This parameter is only available when you select Hadoop from the Type drop-down list. |
| Uri | Enter the metastore URI string. This property is only available when you select Hive or Rest from the Type drop-down list. |
| Catalog implementation | Enter the custom catalog implementation to be used. This property is only available when you select Custom from the Type drop-down list. |
| Catalog properties | Enter any extra properties to be used for the catalog. For more information,
see Configuration from the Iceberg
documentation. This property is only available when you select the Register new catalog parameter. |
Global Variables
|
ERROR_MESSAGE |
The error message generated by the component when an error occurs. This is an After variable and it returns a string. |
Usage
| Usage rule | This component is generally used with other Iceberg components. |