Component-specific settings for tFileOutputParquet
The following table describes the Job script functions and parameters that you can define in the setSettings {} function of the component.
Function/parameter | Description | Mandatory? |
---|---|---|
DEFINE_STORAGE_ CONFIGURATION |
Set this parameter to true to provide the configuration information for the connection to the target file system such as HDFS. If set this parameter to false, the target file system is the local system. Note that the configuration component to be used must be present in the same Job. For example, if a tHDFSConfiguration component is present in the Job, you can select it to write the result in the specified HDFS system. By default, this parameter is set to true. Information noteNote:
This parameter works only in a Spark Batch or Spark Streaming Job. |
Yes for a Spark Batch/Streaming Job |
FILENAME |
Type in the path pointing to the data to be used in the file system. |
Yes |
FILE_ACTION |
Type in an operation for writing data. Acceptable values:
|
No |
COMPRESSION |
Type in an output compression option.
|
No |