Delta Lake properties
Delta Lake connection
Select Delta Lake in the list and configure the connection.
Configuration
Select your engine from the list and set the main and advanced settings.
| Property | Configuration |
|---|---|
| If Define JDBC URL is disabled | If this option is disabled, enter each parameter that identifies the
database to be used in the corresponding fields. Click Load default values in order to help you pre-fill the fields with the default values associated with this type of database. Information noteNote: Use the host name of the target system
instead of 'localhost' in the URL as the Remote Engine Gen2 needs to be able to communicate
with the target system.
|
| If Define JDBC URL is enabled | If this option is enabled, enter the JDBC URL that identifies the
database to be used. The expected format is the following: jdbc:spark://<host>[:<port>]/<database_name> Information noteNote: Use the host name of the target system
instead of 'localhost' in the URL as the Remote Engine Gen2 needs to be able to communicate
with the target system.
|
| User name | Enter the username used to connect to the database. |
| Password | Enter the password used to connect to the database. |
| Property | Configuration |
|---|---|
| Force protocol | If Define JDBC URL is disabled, you can enable this option to define the JDBC driver protocol. |
| Connection timeout | Sets the maximum number of seconds that a user will wait for a connection to be available. If this time is exceeded and the connection is still unavailable, an exception is thrown. |
| Connection validation timeout | Sets the maximum waiting time in seconds for a connection to be considered as alive. |
After configuring the connection, give it a display name (mandatory) and a description (optional).
Delta Lake dataset
| Property | Configuration | |
|---|---|---|
| Dataset name | Enter a display name for the dataset. This name will be used as a unique identifier of the dataset in all Talend Cloud apps. | |
| Connection | Select your connection in the list. If you are creating a dataset based on an existing connection, this field is read-only. | |
| Type | Select the type of dataset you want to create:
|
|
| Property | Configuration |
|---|---|
| Query | Enter the SQL query to access the data of your choice located in your Amazon Aurora table. |
| Table name | Select or enter the unique name of your Amazon Aurora table. |
| Table streams | Select or enter the unique name of your Amazon Aurora table, as well as
the name of the table stream to indicate the type of changes tracked in the
table. For more information on table streams and CDC, read the Snowflake documentation. |
| Property | Configuration |
|---|---|
| Fetch size | Specifies the amount of data sent during one single communication step with the database. In the Fetch size field displayed, you need to enter the size in KB. |
Additional JDBC parameters might be displayed depending on whether the connector is used as a source or destination dataset, read the JDBC parameters section to know more about these parameters.