# tKMeansStrModel properties for Apache Spark Streaming

These properties are used to configure tKMeansStrModel running in the Spark Streaming Job framework.

The Spark Streaming tKMeansStrModel component belongs to the Machine Learning family.

This component is available in Talend Real Time Big Data Platform and Talend Data Fabric.

## Basic settings

| Save on disk | Select this check box to store the clustering model in an HDFS<br>directory you put in the Path<br>field.<br><br>In this case, you need to enter the time interval (in minutes) at the<br>end of which the model is saved.<br><br>If you clear this check box, your model will be stored in<br>memory. |
| --- | --- |
| Path | Select this check box to store the model in a given file system. Otherwise, the model is<br>stored in memory. The button for browsing does not work with the Spark Local mode; if you are using the Spark Yarn<br>or the Spark Standalone mode, ensure that you have properly<br>configured the connection in a configuration component in the same Job, such as tHDFSConfiguration.<br><br>In the Path field, enter the HDFS<br>directory to be used.<br><br>This field is available when you select the check boxes used to save a model to or read a model from a file system. |
| Load a precomputed model from<br>disk | Select this check box to use an existing K-Means model stored in the<br>directory you have specified in the Path field. This is the common case when using tKMeansStrModel. In this situation, the following<br>behaviors can be expected:<br><br><ul> <li> <p>If you select the <span>Reuse the model                                    transformation associated with the model</span> check box,                                    <span>tKMeansStrModel</span> reuses, along                                    with this model to be used, the feature pre-processing                                    algorithms that were previously implemented during the creation                                    of this model. This reuse allows <span>tKMeansStrModel</span> to directly transform new incoming                                    data into K-Means compliant feature vectors and process these                                    vectors, without having to wait for another implementation of                                    the same algorithms.</p> <p>However, with this option activated, you need to check the                                    schema of the data that was transformed by these feature                                    pre-processing algorithms and ensure that the new input data to                                        <span>tKMeansStrModel</span> uses the                                    same schema. </p> <p>You can simply see this schema in the Job which initially                                    implemented these feature pre-processing algorithms.</p> </li> <li> <p>If you clear the <span>Reuse the model                                    transformation associated with the model</span> check box,                                    you need to place one or several <span>tModelEncoder</span> components in front of <span>tKMeansStrModel</span> to transform the incoming                                    data to feature vectors required by K-Means. Then select the                                    column that provides these feature vectors from the <span>Vector to process</span> drop-down list that is                                    displayed.</p> <p>For further information about <span>tModelEncoder</span>, see <a href="https://help.qlik.com/talend/en-US/components/7.3/machine-learning/tmodelencoder" target="_blank">tModelEncoder</a>.</p> </li> <li> <p>If the model to be loaded does not actually exist, <span>tKMeansStrModel</span> will automatically                                    initialize 2 clusters to create a K-Means model.</p> </li> </ul><br><br>If you clear this Load a precomputed model from<br>disk check box, tKMeansStrModel will create a new K-Means model from<br>scratch. |
| Vector to process | Select the input column used to provide feature vectors. Very often,<br>this column is the output of the feature engineering computations<br>performed by tModelEncoder.<br><br>This list appears when you have cleared either the Load a precomputed model from disk check box<br>or the Reuse the model transformation associated<br>with the model check box. |
| Size of your feature vector | Enter the size of the feature vectors to be processed from the column<br>you have selected from the Vector to<br>process list. |
| Display the vector size | Select this check box to display the feature vectors to be used in the<br>console of the Run view.<br><br>This feature will slow down your Job but is useful when you do not<br>know what value to be entered in the Size of your<br>feature vector field. |
| Number of clusters (K) | Enter the number of clusters into which you want tKMeansModel to cluster data.<br><br>In general, a large number of clusters can decreases errors in<br>predictions but increases the risk of overfitting.<br><br>This field appears when you have cleared the Load a precomputed model from disk check box to create a<br>K-Means model from scratch. |
| Decay factor | Enter the decay rate (ranging between 0 and 1) to be applied to<br>discount the weight of existing points against the new incoming points<br>in the process of evaluating new cluster centers.<br><br>Lower decay rate means more importance to be attached to the new<br>incoming data. When decay rate is *0*,<br>new cluster centers are determined completely by the new points; when<br>decay rate is *1*, the existing points<br>and new incoming points are evaluated equally. |
| Time unit | Select the unit on which the decay rate is applied: point or batch of<br>points. |

## Advanced settings

| Display the centers after the<br>processing | Select this check box to output the vectors of the cluster centers<br>into the console of the Run<br>view.<br><br>This feature is often useful when you need to understand how the<br>cluster centers move in the process of training your K-Means<br>model. |
| --- | --- |

## Usage

| **Usage rule** | This component is used as an end component and requires an input link. |
| --- | --- |
| Model evaluation | The parameters you need to set are free parameters and so their values may be provided by<br>previous experiments, empirical guesses or the like. They do not have any optimal values<br>applicable for all datasets.<br><br>Therefore, you need to train the relationship model you are generating with different sets<br>of parameter values until you can obtain the best evaluation result. But note that you need<br>to write the evaluation code yourself to rank your model with scores. |