Reading data from a HDFS connection on Spark

Using predefined HDFS metadata, you can read data from a HDFS filesystem on Spark.

1. In the Designer, add an input component.

   Example

   Add a tFileInputDelimited component.
2. Double-click the component.
   Your component is configured with the tHDFSConfiguration component information, under Storage.

   

3. Click the […] button next to Edit schema.
   
4. Click the plus button to add a data column.

   Example

   1. CustomerID
   2. FirstName
   3. LastName
5. Select the column Types.

   Example

   For CustomerID, select the Integer Type.
6. Click OK.
7. In the File Name field, enter the file path and name of your choice.