currently we are fetching data from shopify using copy activity with version 1.0 shopify linked service. version 1.0 is going to get deprecated by the 3rd week of Oct 2025. version 1.0 supports query so we are able to provide list of columns we want and give whereclause to limit the record we want. this gives us incremental load. query is not supported in version 2.0, we are not sure how to fetch incremental data. it seems inefficient to fetch all data in stage and then filter it.url for copy data from shopify: https://learn.microsoft.com/en-us/azure/data-factory/connector-shopify?tabs=data-factory#upgrade-the-shopify-connector-from-version-10-to-version-20
1 Answer
I already left you an answer on Microsoft Learn Q&A.
With the Shopify v2.0 ADF connector you can’t push a WHERE clause anymore but you can call Shopify directly via the REST connector or HTTP and pass a watermark like updated_at_min, then paginate until no Link: … rel="next" remains.
https://shopify.dev/docs/api/admin-rest/latest/resources/customer
https://learn.microsoft.com/en-us/azure/data-factory/connector-rest
Don’t try to force incrementals through the connector since Microsoft explicitly removed query in v2 and you must set tableName and fetch the table as it is. You can instead use REST or GraphQL for filtered pulls instead.
https://learn.microsoft.com/en-us/azure/data-factory/connector-shopify
