Skip to main content

Component-specific settings for tFileOutputParquet

The following table describes the Job script functions and parameters that you can define in the setSettings {} function of the component.

Function/parameter Description Mandatory?

DEFINE_STORAGE_ CONFIGURATION

Set this parameter to true to provide the configuration information for the connection to the target file system such as HDFS.

If set this parameter to false, the target file system is the local system.

Note that the configuration component to be used must be present in the same Job.

For example, if a tHDFSConfiguration component is present in the Job, you can select it to write the result in the specified HDFS system.

By default, this parameter is set to true.

Information noteNote:

This parameter works only in a Spark Batch or Spark Streaming Job.

Yes for a Spark Batch/Streaming Job

FILENAME

Type in the path pointing to the data to be used in the file system.

Yes

FILE_ACTION

Type in an operation for writing data. Acceptable values:

  • CREATE: Create a file and write data in it. This is the default action.
  • OVERWRITE: Overwrites the file existing in the directory specified using the FILENAME parameter.

No

COMPRESSION

Type in an output compression option.

  • UNCOMPRESSED: Do not compress the output. This is the default option.
  • GZIP: Enable Gzip compression on the output.
  • SNAPPY: Enable Snappy compression on the output.

No

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!