Shopify | Qlik Cloud Help
Skip to main content Skip to complementary content

Shopify

Shopify is an e-commerce platform. You can use Shopify as a data source when landing or replicating data with the new Shopify connector in Qlik Talend Cloud.

Shopify integration replicates data using the Shopify GraphQL Admin API.

See the Schema section for a list of objects available for replication.

Preparing for authentication

To access your data, you need to authenticate the connection with your account credentials.

Information noteMake sure that the account you use has read access to the tables you want to fetch.

To set up your Shopify account, you need:

  • Your Shopify store short name (the part before .myshopify.com).
  • A Shopify app in the Shopify Dev Dashboard with a client ID and client secret.
  • The app and target store in the same Shopify organization. The client credentials grant is only available for apps in your organization's Dev Dashboard installed in stores that you own. Apps distributed through the Shopify App Store or created from the store admin cannot be used with this connector.
  • The app installed on the target store (installation is per store).

To create the app and retrieve your credentials:

  1. Log into your Shopify Dev Dashboard.
  2. Create a custom app for your organization.
  3. Configure the app's access scopes to grant read access to the resources you intend to replicate.
  4. Install the app on the store that you want to replicate.
  5. Navigate to Settings, and copy-paste your client ID and client secret into a secured file.

Access scopes to grant

Grant read access for each resource you intend to replicate. Metafields are covered by the scope of the resource they are attached to.

All tables are offered for selection regardless of which scopes have been granted, so the list you see when setting up a task does not tell you what the app can actually read. Scopes are enforced when the task runs. Grant the scopes for the tables you intend to select before running a task.

Grant this scopeTo replicate these tables
read_ordersorders, order_refunds, order_shipping_lines, transactions, abandoned_checkouts, metafields_orders
read_productsproducts, product_variants, collections, metafields_products, metafields_collections
read_customerscustomers, metafields_customers
read_inventoryinventory_items, inventory_levels
read_locationslocations, inventory_levels
read_assigned_fulfillment_orders, read_merchant_managed_fulfillment_orders, read_third_party_fulfillment_ordersfulfillment_orders

The events table returns activity log entries for store resources rather than one resource type of its own. Grant the scopes for the resources whose events you want to see.

Two scopes deserve particular attention:

  • read_all_orders: by default, read_orders only exposes orders created in the last 60 days. To replicate older order history, request read_all_orders from Shopify. This is a restricted scope that Shopify grants on request, not one you can simply select. Without it, a start date earlier than 60 days ago returns no data before the 60-day boundary.
  • read_users: the author field on the events table requires this scope, which Shopify makes available on Shopify Plus plans only. Where it is not granted, events replicates without that one field and the rest of the table is unaffected.

Creating the connection

For more information, see Connecting to SaaS applications.

  1. Fill in the required connection properties.
  2. Provide a name for the connection in Connection name.

  3. Select Open connection metadata to define metadata for the connection when it has been created.

  4. Click Create.

Connection settings
SettingDescription
Data gateway

Select a Data Movement gateway if required by your use case.

Information note

This field is not available with the Qlik Talend Cloud Starter subscription, as it does not support Data Movement gateway. If you have another subscription tier and do not want to use Data Movement gateway, select None.

For information on the benefits of Data Movement gateway and use cases that require it, see Qlik Data Gateway - Data Movement.

Start Date

Enter the date, in the format MM/DD/YYYY, from which the data must be replicated from your source to your target.

Shopify ShopShopify store short name (the part before .myshopify.com).
Client IDClient ID.
Client SecretClient secret.

Limitations and considerations

  • Deletes are not captured. The connector replicates inserts and updates only. A record deleted in Shopify remains in the destination. Use a periodic full reload if you need deletes reflected.
  • Order history older than 60 days needs an extra scope. See read_all_orders in the access scopes section above.
  • Only one replication task per store and app can run at a time. The orders table uses Shopify's bulk operations API, which allows only one bulk query per store per app at a time. If a second task starts against the same store while the first is still running, the connector waits and backs off for up to roughly ten minutes before failing. Schedule tasks so they do not overlap, or use a separate Shopify app per task.
  • Large order backfills can take a long time. A bulk operation is submitted per date window and Shopify processes it asynchronously. Reducing Date Window Size produces more, smaller operations, which usually finishes sooner on a large store.
  • Start Date applies to the initial sync only. Once a table has a stored bookmark, later syncs resume from it. Changing Start Date does not retroactively re-replicate data.
  • Incremental runs can re-deliver boundary records. Each incremental run filters from the previous run's high-water mark inclusive, so records whose timestamp exactly equals it are replicated again. Records are keyed on id, so this resolves to an upsert in the destination rather than a duplicate.
  • Rate limits are handled automatically. Shopify applies per-store API limits and returns HTTP 429 Too Many Requests when they are exceeded. The connector waits and retries. Server errors and timeouts are retried separately, up to 5 attempts, with exponential backoff.
  • Access tokens are refreshed automatically. Shopify access tokens expire after 24 hours. The connector requests a new one as needed. The client ID and client secret do not expire, but if the secret is rotated in the Dev Dashboard the new value must be saved to the connection.
  • Selecting a table the app cannot read fails the task. Because every table is offered regardless of scope, it is possible to select one the app has not been granted access to. The task then fails when it reaches that table. The fulfillment_orders table is handled differently: the connector skips it, replicates the remaining selected tables, and reports that the connection needs to be re-authorized with the fulfillment order scopes.
  • Adding a scope later requires reinstalling the app. If you grant a new scope after installation, apply it to the app in the Dev Dashboard and reinstall the app on the store before rerunning the task.

Schema

Information note

Schemas and naming conventions can change from version to version. It is recommended to verify your version before continuing.

Depending on your destination, table and column names may not appear as they are outlined below.

Every table carries three additional columns identifying the store the record came from: _sdc_shop_id, _sdc_shop_name, and _sdc_shop_myshopify_domain.

Two tables use createdAt as the replication key rather than updatedAt: locations (because the Shopify locations query does not support filtering on updatedAt, so a modified location is not picked up by an incremental run) and transactions (because transactions are read through their parent order, not as a top-level resource).

abandoned_checkouts

The abandoned_checkouts table contains info about abandoned checkouts. Shopify considers a checkout to be abandoned when a customer has entered billing and shipping details, but hasn’t completed the purchase.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

abandoned_checkouts schema on GitHub

Shopify API method

collections

The collections table contains info about collections.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

collections schema on GitHub

Shopify API method

customers

The customers table contains info about the shop’s customers. This includes their contact details, order history, and email marketing preferences.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify documentation

customers schema on GitHub

events

The events table contains info about events in the shop.

Replication Method

CDC

Primary Key

id

Replication Key

createdAt

Useful links

Shopify documentation

events schema on GitHub

fulfillment_orders

The fulfillment_orders table contains datails about fulfillment orders.

Replication Method

CDC

Primary Key

id

Useful links

Shopify documentation

fulfillment_orders schema on GitHub

inventory_items

The inventory_items table contains info about items in a shop.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify documentation

inventory_items schema on GitHub

inventory_levels

The inventory_levels table contains info about quantities of an inventory item for a location.

Replication Method

CDC

Primary Keys

id

Replication Key

updatedAt

Useful links

Shopify documentation

inventory_levels schema on GitHub

locations

The locations table contains info about locations.

Replication Method

CDC

Primary Key

id

Replication Key

createdAt

Useful links

Shopify documentation

locations schema on GitHub

metafields_collections

The metafields_collections table contains info about metafields for collections.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify API method

metafields_collections schema on GitHub

metafields_customers

The metafields_customers table contains info about metafields for customers.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify API method

metafields_customers schema on GitHub

metafields_orders

The metafields_orders table contains info about metafields for orders.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify API method

metafields_orders schema on GitHub

metafields_products

The metafields_products table contains info about metafields for products.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify API method

metafields_products schema on GitHub

order_refunds

The order_refunds table contains info about refunds associated with orders.

Replication Method

CDC

Primary Key

id

Replication Key

createdAt

Useful links

Shopify API method

order_refunds schema on GitHub

order_shipping_lines

The order_shipping_lines table contains the shipping details that the customer chose for their order.

Replication Method

CDC

Primary Key

id

Useful links

Shopify API method

order_shipping_lines schema on GitHub

orders

The orders table contains info about a shop’s completed orders.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify documentation

orders schema on GitHub

product_variants

The product_variants table contains info about variants that can be added to a product to represent one version of a product with several options.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify documentation

product_variants schema on GitHub

products

The products table contains info about a shop’s products.

Replication Method

CDC

Primary Key

id

Replication Key

updatedAt

Useful links

Shopify documentation

products schema on GitHub

transactions

The transactions table contains info about transactions.

Replication Method

CDC

Primary Key

id

Replication Key

createdAt

Useful links

Shopify API method

transactions schema on GitHub

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!