# Extraction Logs

Information note

**Get 60 days of Extraction Logs**

Advanced and Premium plans come with of Extraction Logs, allowing you to view an integration’s extraction behavior over time, identify patterns, and quickly resolve errors when they arise.

For more info about Stitch Advanced, contact the [Stitch Sales team](https://www.stitchdata.com/contact/?utm_medium=docs&utm_campaign=extraction-log-retention).

Extraction logs provide detail about the extraction portion of the replication process for a given integration. In this guide, we’ll cover:

* [How long logs are retained](https://help.qlik.com/en-US/stitch/replication/integration-extraction-logs#log-retention)

* [How to read the logs](https://help.qlik.com/en-US/stitch/replication/integration-extraction-logs#log-composition)

* [Some examples of extraction logs](https://help.qlik.com/en-US/stitch/replication/integration-extraction-logs#extraction-log-examples)

* [How to handle errors during extraction](https://help.qlik.com/en-US/stitch/replication/integration-extraction-logs#extraction-errors)

## Log retention

Extraction logs are grouped by day. The number of days’ worth of logs available to you depends on your Stitch plan:

* **Free trial**: 1 day

* **Standard**: 7 days

* **Advanced**: 60 days

* **Premium**: 60 days

### Logs and plan changes

Changing your plan can impact logs currently available to you.

#### Plan downgrades

If you downgrade to a plan that offers fewer days’ logs, you’ll **lose** access to the difference between your current plan and your new plan.

For example: If you downgrade to Standard from the Advanced plan, you’ll lose access to 53 days’ worth of logs.

#### Plan upgrades

Likewise, if you upgrade to a plan that offers more days’ logs, you’ll immediately **gain** access to the difference.

For example: If you upgrade to Advanced from the Standard plan, you’ll gain access to an additional 53 days’ worth of logs.

## Log composition

The graph at the top of the Extractions tab displays every time Stitch connected to the integration by day, based on the integration’s [Replication Frequency](https://help.qlik.com/en-US/stitch/replication/replication-scheduling/replication-frequency).

![Extraction graph with intervals of 30 minutes](https://help.qlik.com/en-US/stitch/images/replication/extraction-graph-rep-frequency.gif)Extraction graph with intervals of 30 minutesExtraction graph with intervals of 30 minutes

In addition to displaying the time an extraction began, the tooltips also include how long the extraction ran for and if any errors arose.

To view the raw logs for a specific extraction, click the **View Logs** link in the tooltip or the bar in the graph. This will open the logs for the job in the **Extraction Logs** section, located below the graph.

### Logs for in progress jobs

When a replication job is currently running, the job will have an **In Progress** status and a **Stop Extraction** button:

![An In Progress replication job and Stop Extraction button, highlighted](https://help.qlik.com/en-US/stitch/images/replication/stop-in-progress-job.png)An In Progress replication job and Stop Extraction button, highlightedAn In Progress replication job and Stop Extraction button, highlighted

You can use the **Stop Extraction** button to stop an in progress job, which is useful when the data source is experiencing issues. Refer to the [Start and stop extraction jobs documentation](https://help.qlik.com/en-US/stitch/replication/extractions/start-stop-extraction-jobs) for more info.

### Extraction log fields

Lines in raw extraction logs are made up of four fields. Before we get into the field details, take a look at this example line:

```
2017-11-17 16:44:41,159Z tap - INFO State update: adding bookmarks.ads.updated_time = "2017-11-06T13:29:23-05:00"
```

In the table below, we’ll break down each of the fields and explain what they mean.

| Order | Name | Example | Description |
| --- | --- | --- | --- |
| 1 | Timestamp | `2017-11-17 16:44:41,159Z` | The date and time the action occurred. |
| 2 | Location | `tap` | Indicates the part of the integration the message comes from. **This is used only by Stitch support to debug issues.**<br><br>Possible values are:<br><br><ul> <li><code>main</code></li> <li><code>tap</code></li> <li><code>target</code></li></ul> |
| 3 | Message type | `INFO` | The message type. In general:<br><br><ul> <li><code>INFO</code> indicates things are working as expected</li> <li><code>DEBUG</code> indicates info of interest when diagnosing issues</li> <li><code>WARNING</code> indicates something unexpected happened, but extraction continued. This status isn’t usually noteworthy unless a <code>CRITICAL</code> error is also returned.</li> <li><code>ERROR</code> indicates the integration was unable to perform some function, but extraction continued. This status isn’t usually noteworthy unless a <code>CRITICAL</code> error is also returned.</li> <li><code>CRITICAL</code> indicates an error occurred that caused the extraction to terminate. See <a href="https://help.qlik.com/en-US/stitch/replication/integration-extraction-logs#extraction-errors">Extraction Errors</a> for more info.</li></ul> |
| 4 | Message body | `State update: adding bookmarks.ads.updated_time = "2017-11-06T13:29:23-05:00"` | A description of the action currently being taken. |

## Extraction log examples

When reading the extraction logs for your integrations, pay particular attention to the content of the message body. The message body will contain information about what’s currently happening in the extraction process and errors, should they arise.

Below are some examples of extraction logs, what they indicate, and how to read them.

### Replication Key values

[Replication Keys](https://help.qlik.com/en-US/stitch/replication/replication-keys) are columns used to identify new and updated data in tables that use [Key-based Incremental Replication](https://help.qlik.com/en-US/stitch/replication/replication-methods/key-based-incremental).

The extraction logs contain information about the current Replication Key value for a given table, as well as the updated value detected during the extraction process.

Information note

**SaaS Integrations & Replication Keys**

Unlike database integrations, Stitch automatically selects the field to use for Key-based Incremental Replication. This can make it difficult to remember which field extraction is based on.

To see the Replication Keys for a given integration, check the **Schema** section of the [integration’s documentation](https://help.qlik.com/en-US/stitch/integrations/saas). Look for fields with a ![](https://help.qlik.com/en-US/stitch/images/replication/replication-key-icon.svg) icon next to their names.

#### Replication Key values and extraction

The last saved maximum Replication Key value for a given table is used to detect new and updated data.

For example: In this log line, Stitch will extract data from the `ads` endpoint that has an `updated_at` timestamp greater than or equal to `2017-11-06T12:48:15-05:00`:

```
2017-11-21 18:16:08,389Z    tap - INFO found current bookmark for ads:  2017-11-06T12:48:15-05:00
```

#### Updated Replication Key values

During the extraction process, Stitch will update the Replication Key values for the tables set to replicate.

Generally, the log line containing the saved Replication Key value for a table will be similar to this:

```
2017-11-21 18:21:15,307Z   main - INFO State update: adding bookmarks.ads.updated_time = "2017-11-06T13:29:23-05:00"
```

Beginning after `bookmarks`, read the following as `table_name.replication_key_field`. In this case, the table is `ads`, the Replication Key is `updated_time`, and the saved Replication Key value for the table is being updated to `2017-11-06T13:29:23-05:00`.

**Note**: This line will not display in the logs for tables that have unchanged Replication Key values. This means if Stitch doesn’t detect any new or updated data for a table, a line like the above will not appear in the logs for that table.

## Extraction errors

If an error occurs that terminates the extraction process, a line with a message type of `CRITICAL` will appear in the log. Generally, this will also display as the last line of the log.

For example: During this extraction of Salesforce data, Stitch detected that there wasn’t sufficient API quota available to continue replication:

```
2017-11-20 07:48:45,410Z    tap - CRITICAL Salesforce has reported 32115/100000 (32.12%) total REST quota used across all Salesforce Applications. Terminating replication to not continue past configured percentage of 30.0% total quota.
```

For the majority of errors, Stitch will parse out and display the messages separately from the raw extraction logs:

![Error message from error that occurred during extraction](https://help.qlik.com/en-US/stitch/images/replication/extraction-log-error.png)Error message from error that occurred during extractionError message from error that occurred during extraction

If an error arises, check out the [extraction error references](https://help.qlik.com/en-US/stitch/troubleshooting/error-notifications#integration-extraction-errors) for help.