This SQL query creates a rolled-up view of customer data from Shopify. It combines customer information with their tags, groups customers by email address, and aggregates various attributes. The query performs deduplication by using ROW_NUMBER() to identify the most recent customer record for each email. It also cleans the email addresses by converting them to lowercase. The query integrates customer data with customer tags and aggregates multiple fields, including customer IDs, phone numbers, tags, and various timestamps. It also handles boolean fields by taking the maximum value.
CleaningDeduplicationIntegrationAggregationWITH customers AS (
SELECT
*,
ROW_NUMBER() OVER (PARTITION BY email ORDER BY created_timestamp DESC) AS customer_index
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer
WHERE
NOT email IS NULL /* nonsensical to include any null emails here */
), customer_tags AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer_tag
), rollup_customers AS (
SELECT
LOWER(customers.email) AS email, /* fields to group by */
customers.source_relation,
LISTAGG(DISTINCT CAST(customers.customer_id AS TEXT), ', ') AS customer_ids, /* fields to string agg together */
LISTAGG(DISTINCT CAST(customers.phone AS TEXT), ', ') AS phone_numbers,
LISTAGG(DISTINCT CAST(customer_tags.value AS TEXT), ', ') AS customer_tags,
MIN(customers.created_timestamp) AS first_account_created_at, /* fields to take aggregates of */
MAX(customers.created_timestamp) AS last_account_created_at,
MAX(customers.updated_timestamp) AS last_updated_at,
MAX(customers.marketing_consent_updated_at) AS marketing_consent_updated_at,
MAX(customers._fivetran_synced) AS last_fivetran_synced,
MAX(
CASE WHEN customers.customer_index = 1 THEN customers.is_tax_exempt ELSE NULL END
) AS is_tax_exempt, /* take true if ever given for boolean fields */ /* since this changes every year */
MAX(customers.is_verified_email) AS is_verified_email
/* for all other fields, just take the latest value */
FROM customers
LEFT JOIN customer_tags
ON customers.customer_id = customer_tags.customer_id
AND customers.source_relation = customer_tags.source_relation
GROUP BY
1,
2
)
SELECT
*
FROM rollup_customers
Name | Type | Comment |
---|
This SQL query analyzes abandoned checkouts in a Shopify database. It first filters out deleted checkouts, then aggregates daily data on abandoned checkouts, including counts of unique checkouts, customers, and customer emails. The results are grouped by source relation and date.
FilteringFeaturizationAggregationWITH abandoned_checkout AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout
/* "deleted" abandoned checkouts do not appear to have any data tying them to customers, */ /* discounts, or products (and should therefore not get joined in) but let's filter them out here */
WHERE
NOT COALESCE(is_deleted, FALSE)
), abandoned_checkout_aggregates AS (
SELECT
source_relation,
CAST(DATE_TRUNC('DAY', created_at) AS DATE) AS date_day,
COUNT(DISTINCT checkout_id) AS count_abandoned_checkouts,
COUNT(DISTINCT customer_id) AS count_customers_abandoned_checkout,
COUNT(DISTINCT email) AS count_customer_emails_abandoned_checkout
FROM abandoned_checkout
GROUP BY
1,
2
)
SELECT
*
FROM abandoned_checkout_aggregates
Name | Type | Comment |
---|
This SQL query performs daily aggregations of Shopify order and order line data. It combines information from orders and order lines, calculating various metrics such as order counts, customer counts, sales totals, discounts, refunds, and product quantities. The query also includes time-based aggregations and creates a comprehensive daily summary of order-related activities and financial metrics.
FilteringIntegrationAggregationFeaturizationWITH orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
WHERE
NOT COALESCE(is_deleted, FALSE)
), order_lines AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__order_lines
), order_aggregates AS (
SELECT
source_relation,
CAST(DATE_TRUNC('DAY', created_timestamp) AS DATE) AS date_day,
COUNT(DISTINCT order_id) AS count_orders,
SUM(line_item_count) AS count_line_items,
AVG(line_item_count) AS avg_line_item_count,
COUNT(DISTINCT customer_id) AS count_customers,
COUNT(DISTINCT email) AS count_customer_emails,
SUM(order_adjusted_total) AS order_adjusted_total,
AVG(order_adjusted_total) AS avg_order_value,
SUM(shipping_cost) AS shipping_cost,
SUM(order_adjustment_amount) AS order_adjustment_amount,
SUM(order_adjustment_tax_amount) AS order_adjustment_tax_amount,
SUM(refund_subtotal) AS refund_subtotal,
SUM(refund_total_tax) AS refund_total_tax,
SUM(total_discounts) AS total_discounts,
AVG(total_discounts) AS avg_discount,
SUM(shipping_discount_amount) AS shipping_discount_amount,
AVG(shipping_discount_amount) AS avg_shipping_discount_amount,
SUM(percentage_calc_discount_amount) AS percentage_calc_discount_amount,
AVG(percentage_calc_discount_amount) AS avg_percentage_calc_discount_amount,
SUM(fixed_amount_discount_amount) AS fixed_amount_discount_amount,
AVG(fixed_amount_discount_amount) AS avg_fixed_amount_discount_amount,
SUM(count_discount_codes_applied) AS count_discount_codes_applied,
COUNT(DISTINCT location_id) AS count_locations_ordered_from,
SUM(CASE WHEN count_discount_codes_applied > 0 THEN 1 ELSE 0 END) AS count_orders_with_discounts,
SUM(CASE WHEN refund_subtotal > 0 THEN 1 ELSE 0 END) AS count_orders_with_refunds,
MIN(created_timestamp) AS first_order_timestamp,
MAX(created_timestamp) AS last_order_timestamp
FROM orders
GROUP BY
1,
2
), order_line_aggregates AS (
SELECT
order_lines.source_relation,
CAST(DATE_TRUNC('DAY', orders.created_timestamp) AS DATE) AS date_day,
SUM(order_lines.quantity) AS quantity_sold,
SUM(order_lines.refunded_quantity) AS quantity_refunded,
SUM(order_lines.quantity_net_refunds) AS quantity_net,
SUM(order_lines.quantity) / COUNT(DISTINCT order_lines.order_id) AS avg_quantity_sold,
SUM(order_lines.quantity_net_refunds) / COUNT(DISTINCT order_lines.order_id) AS avg_quantity_net,
COUNT(DISTINCT order_lines.variant_id) AS count_variants_sold,
COUNT(DISTINCT order_lines.product_id) AS count_products_sold,
SUM(
CASE WHEN order_lines.is_gift_card THEN order_lines.quantity_net_refunds ELSE 0 END
) AS quantity_gift_cards_sold,
SUM(
CASE
WHEN order_lines.is_shipping_required
THEN order_lines.quantity_net_refunds
ELSE 0
END
) AS quantity_requiring_shipping
FROM order_lines
LEFT JOIN orders /* just joining with order to get the created_timestamp */
ON order_lines.order_id = orders.order_id
AND order_lines.source_relation = orders.source_relation
GROUP BY
1,
2
), final AS (
SELECT
order_aggregates.*,
order_line_aggregates.quantity_sold,
order_line_aggregates.quantity_refunded,
order_line_aggregates.quantity_net,
order_line_aggregates.count_variants_sold,
order_line_aggregates.count_products_sold,
order_line_aggregates.quantity_gift_cards_sold,
order_line_aggregates.quantity_requiring_shipping,
order_line_aggregates.avg_quantity_sold,
order_line_aggregates.avg_quantity_net
FROM order_aggregates
LEFT JOIN order_line_aggregates
ON order_aggregates.date_day = order_line_aggregates.date_day
AND order_aggregates.source_relation = order_line_aggregates.source_relation
)
SELECT
*
FROM final
Name | Type | Comment |
---|
This SQL query analyzes abandoned checkouts and their associated discounts in a Shopify dataset. It filters out deleted checkouts and empty discount types, joins relevant tables (abandoned checkouts, discount codes, and shipping lines), aggregates data on discounts, prices, and customer counts, and presents a summary of abandoned checkouts grouped by discount code, type, and source relation.
FilteringIntegrationAggregationWITH abandoned_checkout AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout
/* "deleted" abandoned checkouts do not appear to have any data tying them to customers, */ /* discounts, or products (and should therefore not get joined in) but let's filter them out here */
WHERE
NOT COALESCE(is_deleted, FALSE)
), abandoned_checkout_discount_code AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout_discount_code
/* we need the TYPE of discount (shipping, percentage, fixed_amount) to avoid fanning out of joins */ /* so filter out records that have this */
WHERE
COALESCE(type, '') <> ''
), abandoned_checkout_shipping_line AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout_shipping_line
), roll_up_shipping_line AS (
SELECT
checkout_id,
source_relation,
SUM(price) AS price
FROM abandoned_checkout_shipping_line
GROUP BY
1,
2
), abandoned_checkouts_aggregated AS (
SELECT
abandoned_checkout_discount_code.code,
abandoned_checkout_discount_code.type,
abandoned_checkout_discount_code.source_relation,
SUM(abandoned_checkout_discount_code.amount) AS total_abandoned_checkout_discount_amount,
SUM(COALESCE(abandoned_checkout.total_line_items_price, 0)) AS total_abandoned_checkout_line_items_price,
SUM(COALESCE(roll_up_shipping_line.price, 0)) AS total_abandoned_checkout_shipping_price,
COUNT(DISTINCT customer_id) AS count_abandoned_checkout_customers,
COUNT(DISTINCT email) AS count_abandoned_checkout_customer_emails,
COUNT(DISTINCT abandoned_checkout.checkout_id) AS count_abandoned_checkouts
FROM abandoned_checkout_discount_code
LEFT JOIN abandoned_checkout
ON abandoned_checkout_discount_code.checkout_id = abandoned_checkout.checkout_id
AND abandoned_checkout_discount_code.source_relation = abandoned_checkout.source_relation
LEFT JOIN roll_up_shipping_line
ON roll_up_shipping_line.checkout_id = abandoned_checkout_discount_code.checkout_id
AND roll_up_shipping_line.source_relation = abandoned_checkout_discount_code.source_relation
GROUP BY
1,
2,
3
)
SELECT
*
FROM abandoned_checkouts_aggregated
Name | Type | Comment |
---|
This SQL query aggregates order discount data from Shopify. It joins order discount codes with order details, then calculates various metrics such as average and total discount amounts, total order prices, shipping costs, refund amounts, and counts of unique customers and orders for each discount code. The results are grouped by discount code, type, and source relation.
IntegrationAggregationWITH order_discount_code AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_discount_code
), orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
), orders_aggregated AS (
SELECT
order_discount_code.code,
order_discount_code.type,
order_discount_code.source_relation,
AVG(order_discount_code.amount) AS avg_order_discount_amount,
SUM(order_discount_code.amount) AS total_order_discount_amount,
MAX(orders.total_line_items_price) AS total_order_line_items_price, /* summing would multiply the total by the # of discount codes applied to an order */
MAX(orders.shipping_cost) AS total_order_shipping_cost, /* summing would multiply the total by the # of discount codes applied to an order */
MAX(orders.refund_subtotal + orders.refund_total_tax) AS total_order_refund_amount, /* summing would multiply the total by the # of discount codes applied to an order */
COUNT(DISTINCT customer_id) AS count_customers,
COUNT(DISTINCT email) AS count_customer_emails,
COUNT(DISTINCT order_discount_code.order_id) AS count_orders
FROM order_discount_code
JOIN orders
ON order_discount_code.order_id = orders.order_id
AND order_discount_code.source_relation = orders.source_relation
GROUP BY
1,
2,
3
)
SELECT
*
FROM orders_aggregated
Name | Type | Comment |
---|
This SQL query aggregates and analyzes customer order data from Shopify. It combines information from various tables including orders, transactions, customers, and order line aggregates. The query calculates several metrics for each customer email and source relation, such as first and most recent order timestamps, average order value, lifetime total spent, refunded amounts, order counts, and various aggregations related to quantity, tax, discounts, and shipping. The result provides a comprehensive view of customer purchasing behavior and order statistics.
FilteringCleaningIntegrationAggregationFeaturizationWITH orders AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order
), order_aggregates AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders__order_line_aggregates
), transactions AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__transactions
WHERE
LOWER(status) = 'success'
AND NOT LOWER(kind) IN ('authorization', 'void')
AND LOWER(gateway) <> 'gift_card' /* redeeming a giftcard does not introduce new revenue */
), transaction_aggregates AS (
/* this is necessary as customers can pay via multiple payment gateways */
SELECT
order_id,
source_relation,
LOWER(kind) AS kind,
SUM(currency_exchange_calculated_amount) AS currency_exchange_calculated_amount
FROM transactions
GROUP BY
1,
2,
3
), customer_emails AS (
/* in case any orders records don't have the customer email attached yet */
SELECT
customer_id,
source_relation,
email
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer
WHERE
NOT email IS NULL
GROUP BY
1,
2,
3
), aggregated AS (
SELECT
LOWER(customer_emails.email) AS email,
orders.source_relation,
MIN(orders.created_timestamp) AS first_order_timestamp,
MAX(orders.created_timestamp) AS most_recent_order_timestamp,
AVG(transaction_aggregates.currency_exchange_calculated_amount) AS avg_order_value,
SUM(transaction_aggregates.currency_exchange_calculated_amount) AS lifetime_total_spent,
SUM(refunds.currency_exchange_calculated_amount) AS lifetime_total_refunded,
COUNT(DISTINCT orders.order_id) AS lifetime_count_orders,
AVG(order_aggregates.order_total_quantity) AS avg_quantity_per_order,
SUM(order_aggregates.order_total_tax) AS lifetime_total_tax,
AVG(order_aggregates.order_total_tax) AS avg_tax_per_order,
SUM(order_aggregates.order_total_discount) AS lifetime_total_discount,
AVG(order_aggregates.order_total_discount) AS avg_discount_per_order,
SUM(order_aggregates.order_total_shipping) AS lifetime_total_shipping,
AVG(order_aggregates.order_total_shipping) AS avg_shipping_per_order,
SUM(order_aggregates.order_total_shipping_with_discounts) AS lifetime_total_shipping_with_discounts,
AVG(order_aggregates.order_total_shipping_with_discounts) AS avg_shipping_with_discounts_per_order,
SUM(order_aggregates.order_total_shipping_tax) AS lifetime_total_shipping_tax,
AVG(order_aggregates.order_total_shipping_tax) AS avg_shipping_tax_per_order
FROM orders
JOIN customer_emails
ON orders.customer_id = customer_emails.customer_id
AND orders.source_relation = customer_emails.source_relation
LEFT JOIN transaction_aggregates
ON orders.order_id = transaction_aggregates.order_id
AND orders.source_relation = transaction_aggregates.source_relation
AND transaction_aggregates.kind IN ('sale', 'capture')
LEFT JOIN transaction_aggregates AS refunds
ON orders.order_id = refunds.order_id
AND orders.source_relation = refunds.source_relation
AND refunds.kind = 'refund'
LEFT JOIN order_aggregates
ON orders.order_id = order_aggregates.order_id
AND orders.source_relation = order_aggregates.source_relation
GROUP BY
1,
2
)
SELECT
*
FROM aggregated
Name | Type | Comment |
---|
This SQL query aggregates inventory-related data from Shopify orders, focusing on sales, refunds, and fulfillment statistics. It joins various tables (orders, order lines, fulfillments, and refunds) to create a comprehensive view of inventory movements. The query calculates metrics such as total sales, quantity sold, unique orders and customers, first and last order dates, fulfillment status counts, and refund information. The results are grouped by variant ID, location ID, and source relation.
IntegrationAggregationCleaningFeaturizationWITH __dbt__cte__shopify__orders__order_refunds AS (
WITH refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__refund
), order_line_refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line_refund
), refund_join AS (
SELECT
refunds.refund_id,
refunds.created_at,
refunds.order_id,
refunds.user_id,
refunds.source_relation,
order_line_refunds.order_line_refund_id,
order_line_refunds.order_line_id,
order_line_refunds.restock_type,
order_line_refunds.quantity,
order_line_refunds.subtotal,
order_line_refunds.total_tax
FROM refunds
LEFT JOIN order_line_refunds
ON refunds.refund_id = order_line_refunds.refund_id
AND refunds.source_relation = order_line_refunds.source_relation
)
SELECT
*
FROM refund_join
), order_lines AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line
), fulfillment AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__fulfillment
), orders AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order
WHERE
NOT COALESCE(is_deleted, FALSE)
), refunds AS (
SELECT
*
FROM __dbt__cte__shopify__orders__order_refunds
), refunds_aggregated AS (
SELECT
order_line_id,
source_relation,
SUM(quantity) AS quantity,
SUM(COALESCE(subtotal, 0)) AS subtotal
FROM refunds
GROUP BY
1,
2
), joined AS (
SELECT
order_lines.order_line_id,
order_lines.variant_id,
order_lines.source_relation,
fulfillment.location_id, /* location id is stored in fulfillment rather than order */
orders.order_id,
orders.customer_id,
fulfillment.fulfillment_id,
LOWER(orders.email) AS email,
order_lines.pre_tax_price,
order_lines.quantity,
orders.created_timestamp AS order_created_timestamp,
fulfillment.status AS fulfillment_status,
refunds_aggregated.subtotal AS subtotal_sold_refunds,
refunds_aggregated.quantity AS quantity_sold_refunds
FROM order_lines
JOIN orders
ON order_lines.order_id = orders.order_id
AND order_lines.source_relation = orders.source_relation
JOIN fulfillment
ON orders.order_id = fulfillment.order_id
AND orders.source_relation = fulfillment.source_relation
LEFT JOIN refunds_aggregated
ON refunds_aggregated.order_line_id = order_lines.order_line_id
AND refunds_aggregated.source_relation = order_lines.source_relation
), aggregated AS (
SELECT
variant_id,
location_id,
source_relation,
SUM(pre_tax_price) AS subtotal_sold,
SUM(quantity) AS quantity_sold,
COUNT(DISTINCT order_id) AS count_distinct_orders,
COUNT(DISTINCT customer_id) AS count_distinct_customers,
COUNT(DISTINCT email) AS count_distinct_customer_emails,
MIN(order_created_timestamp) AS first_order_timestamp,
MAX(order_created_timestamp) AS last_order_timestamp,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'pending' THEN fulfillment_id END) AS count_fulfillment_pending,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'open' THEN fulfillment_id END) AS count_fulfillment_open,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'success' THEN fulfillment_id END) AS count_fulfillment_success,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'cancelled' THEN fulfillment_id END) AS count_fulfillment_cancelled,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'error' THEN fulfillment_id END) AS count_fulfillment_error,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'failure' THEN fulfillment_id END) AS count_fulfillment_failure,
SUM(COALESCE(subtotal_sold_refunds, 0)) AS subtotal_sold_refunds,
SUM(COALESCE(quantity_sold_refunds, 0)) AS quantity_sold_refunds
FROM joined
GROUP BY
1,
2,
3
)
SELECT
*
FROM aggregated
Name | Type | Comment |
---|
This SQL query calculates shipping-related aggregates for Shopify orders. It joins order shipping line data with order shipping tax line data, then aggregates the shipping price, discounted shipping price, and shipping tax for each order. The result provides a summary of shipping costs and taxes per order.
IntegrationAggregationWITH order_shipping_line AS (
SELECT
order_id,
source_relation,
order_shipping_line_id,
SUM(price) AS shipping_price,
SUM(discounted_price) AS discounted_shipping_price
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_shipping_line
GROUP BY
1,
2,
3
), order_shipping_tax_line AS (
SELECT
order_shipping_line_id,
source_relation,
SUM(price) AS shipping_tax
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_shipping_tax_line
GROUP BY
1,
2
), aggregated AS (
SELECT
order_shipping_line.order_id,
order_shipping_line.source_relation,
SUM(order_shipping_line.shipping_price) AS shipping_price,
SUM(order_shipping_line.discounted_shipping_price) AS discounted_shipping_price,
SUM(order_shipping_tax_line.shipping_tax) AS shipping_tax
FROM order_shipping_line
LEFT JOIN order_shipping_tax_line
ON order_shipping_line.order_shipping_line_id = order_shipping_tax_line.order_shipping_line_id
AND order_shipping_line.source_relation = order_shipping_tax_line.source_relation
GROUP BY
1,
2
)
SELECT
*
FROM aggregated
Name | Type | Comment |
---|
This SQL query aggregates data from Shopify order lines and orders to create product-level metrics. It calculates various sales statistics for each product, including quantity sold, subtotal, refunds, first and most recent order dates, discounts, taxes, and average quantities and amounts per order line. The query joins order lines with orders and groups the results by product ID and source relation.
IntegrationAggregationFeaturizationWITH order_lines AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__order_lines
), orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
), product_aggregated AS (
SELECT
order_lines.product_id,
order_lines.source_relation,
SUM(order_lines.quantity) AS quantity_sold, /* moved over from shopify__products */
SUM(order_lines.pre_tax_price) AS subtotal_sold,
SUM(order_lines.quantity_net_refunds) AS quantity_sold_net_refunds,
SUM(order_lines.subtotal_net_refunds) AS subtotal_sold_net_refunds,
MIN(orders.created_timestamp) AS first_order_timestamp,
MAX(orders.created_timestamp) AS most_recent_order_timestamp,
SUM(order_lines.total_discount) AS product_total_discount, /* new columns */
SUM(order_lines.order_line_tax) AS product_total_tax,
AVG(order_lines.quantity) AS avg_quantity_per_order_line,
AVG(order_lines.total_discount) AS product_avg_discount_per_order_line,
AVG(order_lines.order_line_tax) AS product_avg_tax_per_order_line
FROM order_lines
LEFT JOIN orders
ON order_lines.order_id = orders.order_id
AND order_lines.source_relation = orders.source_relation
GROUP BY
1,
2
)
SELECT
*
FROM product_aggregated
Name | Type | Comment |
---|
This SQL query integrates data from multiple Shopify-related tables to create a comprehensive view of products. It joins product information with aggregated data from collections, tags, variants, and images. The query performs several aggregations, including collecting collections and tags into comma-separated lists, counting variants and images, and determining if a product has an image. The result is a detailed product dataset with additional attributes derived from related tables.
IntegrationAggregationFeaturizationWITH products AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product
), collection_product AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__collection_product
), collection AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__collection
WHERE
NOT COALESCE(is_deleted, FALSE) /* limit to only active collections */
), product_tag AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_tag
), product_variant AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_variant
), product_image AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_image
), collections_aggregated AS (
SELECT
collection_product.product_id,
collection_product.source_relation,
LISTAGG(collection.title, ', ') AS collections
FROM collection_product
JOIN collection
ON collection_product.collection_id = collection.collection_id
AND collection_product.source_relation = collection.source_relation
GROUP BY
1,
2
), tags_aggregated AS (
SELECT
product_id,
source_relation,
LISTAGG(value, ', ') AS tags
FROM product_tag
GROUP BY
1,
2
), variants_aggregated AS (
SELECT
product_id,
source_relation,
COUNT(variant_id) AS count_variants
FROM product_variant
GROUP BY
1,
2
), images_aggregated AS (
SELECT
product_id,
source_relation,
COUNT(*) AS count_images
FROM product_image
GROUP BY
1,
2
), joined AS (
SELECT
products.*,
collections_aggregated.collections,
tags_aggregated.tags,
variants_aggregated.count_variants,
COALESCE(images_aggregated.count_images, 0) > 0 AS has_product_image
FROM products
LEFT JOIN collections_aggregated
ON products.product_id = collections_aggregated.product_id
AND products.source_relation = collections_aggregated.source_relation
LEFT JOIN tags_aggregated
ON products.product_id = tags_aggregated.product_id
AND products.source_relation = tags_aggregated.source_relation
LEFT JOIN variants_aggregated
ON products.product_id = variants_aggregated.product_id
AND products.source_relation = variants_aggregated.source_relation
LEFT JOIN images_aggregated
ON products.product_id = images_aggregated.product_id
AND products.source_relation = images_aggregated.source_relation
)
SELECT
*
FROM joined
Name | Type | Comment |
---|
This SQL query generates a calendar table with daily dates from January 1, 2019, up to the current date. It uses a clever bit manipulation technique to create a sequence of numbers, which are then transformed into dates. The result is a filtered list of dates within the specified range.
FeaturizationFilteringOtherWITH rawdata AS (
WITH p AS (
SELECT
0 AS generated_number
UNION ALL
SELECT
1
), unioned AS (
SELECT
p0.generated_number * POWER(2, 0) + p1.generated_number * POWER(2, 1) + p2.generated_number * POWER(2, 2) + p3.generated_number * POWER(2, 3) + p4.generated_number * POWER(2, 4) + p5.generated_number * POWER(2, 5) + p6.generated_number * POWER(2, 6) + p7.generated_number * POWER(2, 7) + p8.generated_number * POWER(2, 8) + p9.generated_number * POWER(2, 9) + p10.generated_number * POWER(2, 10) + 1 AS generated_number
FROM p AS p0
CROSS JOIN p AS p1
CROSS JOIN p AS p2
CROSS JOIN p AS p3
CROSS JOIN p AS p4
CROSS JOIN p AS p5
CROSS JOIN p AS p6
CROSS JOIN p AS p7
CROSS JOIN p AS p8
CROSS JOIN p AS p9
CROSS JOIN p AS p10
)
SELECT
*
FROM unioned
WHERE
generated_number <= 2041
ORDER BY
generated_number
), all_periods AS (
SELECT
(
DATEADD(day, ROW_NUMBER() OVER (ORDER BY 1) - 1, CAST('2019-01-01' AS DATE))
) AS date_day
FROM rawdata
), filtered AS (
SELECT
*
FROM all_periods
WHERE
date_day <= CURRENT_DATE
)
SELECT
*
FROM filtered
Name | Type | Comment |
---|
This SQL query creates customer cohorts based on their first order date and calculates various metrics for each customer on a monthly basis. It joins customer data with order data, calculates lifetime metrics using window functions, and assigns a unique identifier to each customer cohort. The query provides a comprehensive view of customer behavior over time, including order counts, total prices, and line item counts both for individual months and cumulatively.
IntegrationAggregationFeaturizationOtherWITH calendar AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__calendar
WHERE
CAST(DATE_TRUNC('MONTH', date_day) AS DATE) = date_day
), customers AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__customers
), orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
), customer_calendar AS (
SELECT
CAST(calendar.date_day AS DATE) AS date_month,
customers.customer_id,
customers.first_order_timestamp,
customers.source_relation,
CAST(DATE_TRUNC('MONTH', first_order_timestamp) AS DATE) AS cohort_month
FROM calendar
INNER JOIN customers
ON CAST(DATE_TRUNC('MONTH', first_order_timestamp) AS DATE) <= calendar.date_day
), orders_joined AS (
SELECT
customer_calendar.date_month,
customer_calendar.customer_id,
customer_calendar.first_order_timestamp,
customer_calendar.cohort_month,
customer_calendar.source_relation,
COALESCE(COUNT(DISTINCT orders.order_id), 0) AS order_count_in_month,
COALESCE(SUM(orders.order_adjusted_total), 0) AS total_price_in_month,
COALESCE(SUM(orders.line_item_count), 0) AS line_item_count_in_month
FROM customer_calendar
LEFT JOIN orders
ON customer_calendar.customer_id = orders.customer_id
AND customer_calendar.source_relation = orders.source_relation
AND customer_calendar.date_month = CAST(DATE_TRUNC('MONTH', created_timestamp) AS DATE)
GROUP BY
1,
2,
3,
4,
5
), windows AS (
SELECT
*,
SUM(total_price_in_month) OVER (PARTITION BY customer_id ORDER BY date_month rows BETWEEN UNBOUNDED preceding AND CURRENT ROW) AS total_price_lifetime,
SUM(order_count_in_month) OVER (PARTITION BY customer_id ORDER BY date_month rows BETWEEN UNBOUNDED preceding AND CURRENT ROW) AS order_count_lifetime,
SUM(line_item_count_in_month) OVER (PARTITION BY customer_id ORDER BY date_month rows BETWEEN UNBOUNDED preceding AND CURRENT ROW) AS line_item_count_lifetime,
ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY date_month ASC) AS cohort_month_number
FROM orders_joined
), final AS (
SELECT
*,
MD5(
CAST(COALESCE(CAST(date_month AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(customer_id AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(source_relation AS TEXT), '_dbt_utils_surrogate_key_null_') AS TEXT)
) AS customer_cohort_id
FROM windows
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
cohort_month | None | The month the cohort belongs to, i.e the first month the customer had an order. |
cohort_month_number | None | The 'number' of the `date_month` of the record, i.e. how many months from their start month this cohort occurred |
customer_cohort_id | None | Unique key representing a customer in a given month. Hashed on 'date_month', 'customer_id', and 'source_relation'. |
customer_id | None | The ID of the related customer. |
date_month | None | The calendar month the customer stats relate to. |
first_order_timestamp | None | The timestamp of the customer's first order. |
line_item_count_in_month | None | Number of line items purchased in the `date_month` |
line_item_count_lifetime | None | Number of line items purchased up until and including this `date_month`. |
order_count_in_month | None | Number of orders purchased in the `date_month` |
order_count_lifetime | None | Number of orders purchased up until and including this `date_month`. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
total_price_in_month | None | Total amount (in shop currency) purchased in the `date_month` |
total_price_lifetime | None | Total amount (in shop currency) up until and including this `date_month`. |
This SQL query creates customer email cohorts for Shopify data. It combines calendar, customer, and order data to track customer behavior over time. The query calculates monthly and lifetime metrics for each customer, including order count, total price, and line item count. It also assigns a cohort month and a unique customer cohort ID to each record.
IntegrationAggregationFeaturizationOtherWITH calendar AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__calendar
WHERE
CAST(DATE_TRUNC('MONTH', date_day) AS DATE) = date_day
), customers AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__customer_emails
), orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
), customer_calendar AS (
SELECT
CAST(calendar.date_day AS DATE) AS date_month,
customers.email,
customers.first_order_timestamp,
customers.source_relation,
DATE_TRUNC('MONTH', first_order_timestamp) AS cohort_month
FROM calendar
INNER JOIN customers
ON CAST(DATE_TRUNC('MONTH', first_order_timestamp) AS DATE) <= calendar.date_day
), orders_joined AS (
SELECT
customer_calendar.date_month,
customer_calendar.email,
customer_calendar.first_order_timestamp,
customer_calendar.cohort_month,
customer_calendar.source_relation,
COALESCE(COUNT(DISTINCT orders.order_id), 0) AS order_count_in_month,
COALESCE(SUM(orders.order_adjusted_total), 0) AS total_price_in_month,
COALESCE(SUM(orders.line_item_count), 0) AS line_item_count_in_month
FROM customer_calendar
LEFT JOIN orders
ON customer_calendar.email = orders.email
AND customer_calendar.source_relation = orders.source_relation
AND customer_calendar.date_month = CAST(DATE_TRUNC('MONTH', created_timestamp) AS DATE)
GROUP BY
1,
2,
3,
4,
5
), windows AS (
SELECT
*,
SUM(total_price_in_month) OVER (PARTITION BY email ORDER BY date_month rows BETWEEN UNBOUNDED preceding AND CURRENT ROW) AS total_price_lifetime,
SUM(order_count_in_month) OVER (PARTITION BY email ORDER BY date_month rows BETWEEN UNBOUNDED preceding AND CURRENT ROW) AS order_count_lifetime,
SUM(line_item_count_in_month) OVER (PARTITION BY email ORDER BY date_month rows BETWEEN UNBOUNDED preceding AND CURRENT ROW) AS line_item_count_lifetime,
ROW_NUMBER() OVER (PARTITION BY email ORDER BY date_month ASC) AS cohort_month_number
FROM orders_joined
), final AS (
SELECT
*,
MD5(
CAST(COALESCE(CAST(date_month AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(email AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(source_relation AS TEXT), '_dbt_utils_surrogate_key_null_') AS TEXT)
) AS customer_cohort_id
FROM windows
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
cohort_month | None | The month the cohort belongs to, i.e the first month the customer had an order. |
cohort_month_number | None | The 'number' of the `date_month` of the record, i.e. how many months from their start month this cohort occurred |
customer_cohort_id | None | Unique key representing a customer in a given month. Hashed on 'date_month','email', and 'source_relation'. |
None | The ID of the related customer. | |
date_month | None | The calendar month the customer stats relate to. |
first_order_timestamp | None | The timestamp of the customer's first order. |
line_item_count_in_month | None | Number of line items purchased in the `date_month` |
line_item_count_lifetime | None | Number of line items purchased up until and including this `date_month`. |
order_count_in_month | None | Number of orders purchased in the `date_month` |
order_count_lifetime | None | Number of orders purchased up until and including this `date_month`. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
total_price_in_month | None | Total amount (in shop currency) purchased in the `date_month` |
total_price_lifetime | None | Total amount (in shop currency) up until and including this `date_month`. |
This SQL query aggregates and integrates customer data from various Shopify-related tables. It calculates lifetime metrics for customers, including order totals, average order values, abandoned checkouts, and various financial metrics (spent, refunded, tax, shipping, etc.). The query joins data from orders, transactions, customer emails, and abandoned checkouts to create a comprehensive customer profile.
IntegrationAggregationFeaturizationCleaningWITH __dbt__cte__int_shopify__emails__order_aggregates AS (
WITH orders AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order
), order_aggregates AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders__order_line_aggregates
), transactions AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__transactions
WHERE
LOWER(status) = 'success'
AND NOT LOWER(kind) IN ('authorization', 'void')
AND LOWER(gateway) <> 'gift_card' /* redeeming a giftcard does not introduce new revenue */
), transaction_aggregates AS (
/* this is necessary as customers can pay via multiple payment gateways */
SELECT
order_id,
source_relation,
LOWER(kind) AS kind,
SUM(currency_exchange_calculated_amount) AS currency_exchange_calculated_amount
FROM transactions
GROUP BY
1,
2,
3
), customer_emails AS (
/* in case any orders records don't have the customer email attached yet */
SELECT
customer_id,
source_relation,
email
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer
WHERE
NOT email IS NULL
GROUP BY
1,
2,
3
), aggregated AS (
SELECT
LOWER(customer_emails.email) AS email,
orders.source_relation,
MIN(orders.created_timestamp) AS first_order_timestamp,
MAX(orders.created_timestamp) AS most_recent_order_timestamp,
AVG(transaction_aggregates.currency_exchange_calculated_amount) AS avg_order_value,
SUM(transaction_aggregates.currency_exchange_calculated_amount) AS lifetime_total_spent,
SUM(refunds.currency_exchange_calculated_amount) AS lifetime_total_refunded,
COUNT(DISTINCT orders.order_id) AS lifetime_count_orders,
AVG(order_aggregates.order_total_quantity) AS avg_quantity_per_order,
SUM(order_aggregates.order_total_tax) AS lifetime_total_tax,
AVG(order_aggregates.order_total_tax) AS avg_tax_per_order,
SUM(order_aggregates.order_total_discount) AS lifetime_total_discount,
AVG(order_aggregates.order_total_discount) AS avg_discount_per_order,
SUM(order_aggregates.order_total_shipping) AS lifetime_total_shipping,
AVG(order_aggregates.order_total_shipping) AS avg_shipping_per_order,
SUM(order_aggregates.order_total_shipping_with_discounts) AS lifetime_total_shipping_with_discounts,
AVG(order_aggregates.order_total_shipping_with_discounts) AS avg_shipping_with_discounts_per_order,
SUM(order_aggregates.order_total_shipping_tax) AS lifetime_total_shipping_tax,
AVG(order_aggregates.order_total_shipping_tax) AS avg_shipping_tax_per_order
FROM orders
JOIN customer_emails
ON orders.customer_id = customer_emails.customer_id
AND orders.source_relation = customer_emails.source_relation
LEFT JOIN transaction_aggregates
ON orders.order_id = transaction_aggregates.order_id
AND orders.source_relation = transaction_aggregates.source_relation
AND transaction_aggregates.kind IN ('sale', 'capture')
LEFT JOIN transaction_aggregates AS refunds
ON orders.order_id = refunds.order_id
AND orders.source_relation = refunds.source_relation
AND refunds.kind = 'refund'
LEFT JOIN order_aggregates
ON orders.order_id = order_aggregates.order_id
AND orders.source_relation = order_aggregates.source_relation
GROUP BY
1,
2
)
SELECT
*
FROM aggregated
), customer_emails AS (
SELECT
*
/* No columns were returned. Maybe the relation doesn't exist yet
or all columns were excluded. This star is only output during
dbt compile, and exists to keep SQLFluff happy. */
FROM TEST.PUBLIC_shopify.int_shopify__customer_email_rollup
), orders AS (
SELECT
*
FROM __dbt__cte__int_shopify__emails__order_aggregates
WHERE
NOT email IS NULL
), abandoned AS (
SELECT
LOWER(email) AS email,
source_relation,
COUNT(DISTINCT checkout_id) AS lifetime_abandoned_checkouts
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout
WHERE
NOT email IS NULL
GROUP BY
1,
2
), joined AS (
SELECT
customer_emails.*,
COALESCE(abandoned.lifetime_abandoned_checkouts, 0) AS lifetime_abandoned_checkouts,
orders.first_order_timestamp,
orders.most_recent_order_timestamp,
orders.avg_order_value,
COALESCE(orders.lifetime_total_spent, 0) AS lifetime_total_spent,
COALESCE(orders.lifetime_total_refunded, 0) AS lifetime_total_refunded,
(
COALESCE(orders.lifetime_total_spent, 0) - COALESCE(orders.lifetime_total_refunded, 0)
) AS lifetime_total_net,
COALESCE(orders.lifetime_count_orders, 0) AS lifetime_count_orders,
orders.avg_quantity_per_order,
COALESCE(orders.lifetime_total_tax, 0) AS lifetime_total_tax,
orders.avg_tax_per_order,
COALESCE(orders.lifetime_total_discount, 0) AS lifetime_total_discount,
orders.avg_discount_per_order,
COALESCE(orders.lifetime_total_shipping, 0) AS lifetime_total_shipping,
orders.avg_shipping_per_order,
COALESCE(orders.lifetime_total_shipping_with_discounts, 0) AS lifetime_total_shipping_with_discounts,
orders.avg_shipping_with_discounts_per_order,
COALESCE(orders.lifetime_total_shipping_tax, 0) AS lifetime_total_shipping_tax,
orders.avg_shipping_tax_per_order
FROM customer_emails
LEFT JOIN orders
ON customer_emails.email = orders.email
AND customer_emails.source_relation = orders.source_relation
LEFT JOIN abandoned
ON customer_emails.email = abandoned.email
AND customer_emails.source_relation = abandoned.source_relation
)
SELECT
*
FROM joined
Name | Type | Comment |
---|---|---|
last_fivetran_synced | None | The time when a record was last updated by Fivetran. |
default_address_id | None | The default address for the customer. |
None | The unique email address of the customer. | |
first_name | None | The customer's first name. |
customer_ids | None | Comma-separated list of customer IDs associated with the email. |
last_name | None | The customer's last name. |
lifetime_count_orders | None | The number of orders associated with this customer. |
phone_numbers | None | Comma-separated list of phone numbers associated with this email. |
account_state | None | The state of the customer's account with a shop. |
is_tax_exempt | None | Whether the customer is exempt from paying taxes on their order. If true, then taxes won't be applied to an order at checkout. If false, then taxes will be applied at checkout. |
last_updated_at | None | The date and time when the customer information was last updated. |
is_verified_email | None | Whether the customer has verified their email address. |
first_order_timestamp | None | The timestamp the customer completed their first order. |
most_recent_order_timestamp | None | The timestamp the customer completed their most recent order. |
avg_order_value | None | The average order value for the customer. |
lifetime_total_spent | None | The total amount of money that the customer has spent on orders across their order history (in shop currency). |
lifetime_total_refunded | None | The total amount of money that the customer has been refunded on orders across their order history. |
lifetime_total_net | None | The total amount of money (minus refunds) that the customer has spent across their order history. |
first_account_created_at | None | Timestamp of when the first account associated with this email was created. |
last_account_created_at | None | Timestamp of when the last account associated with this email was created. |
source_relation | None | The schema or database this record came from, if you are unioning multiple connectors. Null if not. |
lifetime_abandoned_checkouts | None | Total number of abandoned checkouts abandoned by the customer. |
customer_tags | None | A string aggregated list of all tags associated with a customer. |
avg_quantity_per_order | None | Average quantity of items per order customer orders. |
lifetime_total_tax | None | Total amount of tax attributed to the customer. |
avg_tax_per_order | None | Average tax per order attributed to the customer. |
lifetime_total_discount | None | Total discounts attributed to the customer. |
avg_discount_per_order | None | Average discount per order attributed to the customer. |
lifetime_total_shipping | None | Total shipping costs attributed to the customer. |
avg_shipping_per_order | None | Average shipping cost per order attributed to the customer. |
lifetime_total_shipping_with_discounts | None | Total shipping costs after discounts attributed to the customer. |
avg_shipping_with_discounts_per_order | None | Average shipping costs after discounts per order attributed to the customer. |
lifetime_total_shipping_tax | None | Total shipping tax attributed to the customer. |
avg_shipping_tax_per_order | None | Average shipping tax per order attributed to the customer. |
marketing_consent_updated_at | None | The date and time when the customer consented to receive marketing material by email. If no date is provided, then the date and time when the consent information was sent is used. |
currency | None | The three-letter code (ISO 4217 format) for the currency that the customer used when they paid for their last order. Defaults to the shop currency. Returns the shop currency for test orders. |
marketing_consent_state | None | Field indicating if the customer has consented to receive marketing material via email. Coalescing of the new `email_marketing_consent_state` field and the deprecated `accepts_marketing` field. Records with the old field will be marked with '(legacy)'. |
marketing_opt_in_level | None | The marketing subscription opt-in level, as described in the M3AAWG Sender Best Common Practices, that the customer gave when they consented to receive marketing material by email. |
note | None | A note about the customer. |
This SQL query integrates customer data from multiple Shopify-related tables to create a comprehensive customer profile. It combines information from orders, transactions, customer tags, and abandoned checkouts. The query calculates various customer metrics such as lifetime spending, order counts, average order value, and abandoned checkout counts. It also includes aggregations of tax, shipping, and discount information across a customer's order history.
IntegrationAggregationFeaturizationCleaningWITH __dbt__cte__shopify__customers__order_aggregates AS (
WITH orders AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order
WHERE
NOT customer_id IS NULL
), order_aggregates AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders__order_line_aggregates
), transactions AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__transactions
WHERE
LOWER(status) = 'success'
AND NOT LOWER(kind) IN ('authorization', 'void')
AND LOWER(gateway) <> 'gift_card' /* redeeming a giftcard does not introduce new revenue */
), transaction_aggregates AS (
/* this is necessary as customers can pay via multiple payment gateways */
SELECT
order_id,
source_relation,
LOWER(kind) AS kind,
SUM(currency_exchange_calculated_amount) AS currency_exchange_calculated_amount
FROM transactions
GROUP BY
1,
2,
3
), customer_tags AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer_tag
), customer_tags_aggregated AS (
SELECT
customer_id,
source_relation,
LISTAGG(DISTINCT CAST(value AS TEXT), ', ') AS customer_tags
FROM customer_tags
GROUP BY
1,
2
), aggregated AS (
SELECT
orders.customer_id,
orders.source_relation,
customer_tags_aggregated.customer_tags,
MIN(orders.created_timestamp) AS first_order_timestamp,
MAX(orders.created_timestamp) AS most_recent_order_timestamp,
AVG(transaction_aggregates.currency_exchange_calculated_amount) AS avg_order_value,
SUM(transaction_aggregates.currency_exchange_calculated_amount) AS lifetime_total_spent,
SUM(refunds.currency_exchange_calculated_amount) AS lifetime_total_refunded,
COUNT(DISTINCT orders.order_id) AS lifetime_count_orders,
AVG(order_aggregates.order_total_quantity) AS avg_quantity_per_order,
SUM(order_aggregates.order_total_tax) AS lifetime_total_tax,
AVG(order_aggregates.order_total_tax) AS avg_tax_per_order,
SUM(order_aggregates.order_total_discount) AS lifetime_total_discount,
AVG(order_aggregates.order_total_discount) AS avg_discount_per_order,
SUM(order_aggregates.order_total_shipping) AS lifetime_total_shipping,
AVG(order_aggregates.order_total_shipping) AS avg_shipping_per_order,
SUM(order_aggregates.order_total_shipping_with_discounts) AS lifetime_total_shipping_with_discounts,
AVG(order_aggregates.order_total_shipping_with_discounts) AS avg_shipping_with_discounts_per_order,
SUM(order_aggregates.order_total_shipping_tax) AS lifetime_total_shipping_tax,
AVG(order_aggregates.order_total_shipping_tax) AS avg_shipping_tax_per_order
FROM orders
LEFT JOIN transaction_aggregates
ON orders.order_id = transaction_aggregates.order_id
AND orders.source_relation = transaction_aggregates.source_relation
AND transaction_aggregates.kind IN ('sale', 'capture')
LEFT JOIN transaction_aggregates AS refunds
ON orders.order_id = refunds.order_id
AND orders.source_relation = refunds.source_relation
AND refunds.kind = 'refund'
LEFT JOIN order_aggregates
ON orders.order_id = order_aggregates.order_id
AND orders.source_relation = order_aggregates.source_relation
LEFT JOIN customer_tags_aggregated
ON orders.customer_id = customer_tags_aggregated.customer_id
AND orders.source_relation = customer_tags_aggregated.source_relation
GROUP BY
1,
2,
3
)
SELECT
*
FROM aggregated
), customers AS (
SELECT
*
/* No columns were returned. Maybe the relation doesn't exist yet
or all columns were excluded. This star is only output during
dbt compile, and exists to keep SQLFluff happy. */
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer
), orders AS (
SELECT
*
FROM __dbt__cte__shopify__customers__order_aggregates
), abandoned AS (
SELECT
customer_id,
source_relation,
COUNT(DISTINCT checkout_id) AS lifetime_abandoned_checkouts
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout
WHERE
NOT customer_id IS NULL
GROUP BY
1,
2
), joined AS (
SELECT
customers.*,
COALESCE(abandoned.lifetime_abandoned_checkouts, 0) AS lifetime_abandoned_checkouts,
orders.first_order_timestamp,
orders.most_recent_order_timestamp,
orders.customer_tags,
orders.avg_order_value,
COALESCE(orders.lifetime_total_spent, 0) AS lifetime_total_spent,
COALESCE(orders.lifetime_total_refunded, 0) AS lifetime_total_refunded,
(
COALESCE(orders.lifetime_total_spent, 0) - COALESCE(orders.lifetime_total_refunded, 0)
) AS lifetime_total_net,
COALESCE(orders.lifetime_count_orders, 0) AS lifetime_count_orders,
orders.avg_quantity_per_order,
COALESCE(orders.lifetime_total_tax, 0) AS lifetime_total_tax,
orders.avg_tax_per_order,
COALESCE(orders.lifetime_total_discount, 0) AS lifetime_total_discount,
orders.avg_discount_per_order,
COALESCE(orders.lifetime_total_shipping, 0) AS lifetime_total_shipping,
orders.avg_shipping_per_order,
COALESCE(orders.lifetime_total_shipping_with_discounts, 0) AS lifetime_total_shipping_with_discounts,
orders.avg_shipping_with_discounts_per_order,
COALESCE(orders.lifetime_total_shipping_tax, 0) AS lifetime_total_shipping_tax,
orders.avg_shipping_tax_per_order
FROM customers
LEFT JOIN orders
ON customers.customer_id = orders.customer_id
AND customers.source_relation = orders.source_relation
LEFT JOIN abandoned
ON customers.customer_id = abandoned.customer_id
AND customers.source_relation = abandoned.source_relation
)
SELECT
*
FROM joined
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
created_timestamp | None | The date and time when the customer was created. |
default_address_id | None | The default address for the customer. |
None | The unique email address of the customer. Attempting to assign the same email address to multiple customers returns an error. | |
first_name | None | The customer's first name. |
customer_id | None | A unique identifier for the customer. |
last_name | None | The customer's last name. |
lifetime_count_orders | None | The number of orders associated with this customer. |
phone | None | The unique phone number (E.164 format) for this customer. Attempting to assign the same phone number to multiple customers returns an error. |
account_state | None | The state of the customer's account with a shop. |
is_tax_exempt | None | Whether the customer is exempt from paying taxes on their order. If true, then taxes won't be applied to an order at checkout. If false, then taxes will be applied at checkout. |
updated_timestamp | None | The date and time when the customer information was last updated. |
is_verified_email | None | Whether the customer has verified their email address. |
first_order_timestamp | None | The timestamp the customer completed their first order. |
most_recent_order_timestamp | None | The timestamp the customer completed their most recent order. |
avg_order_value | None | The average order value for the customer. |
lifetime_total_spent | None | The total amount of money in shop currency that the customer has spent on orders across their order history. |
lifetime_total_refunded | None | The total amount of money that the customer has been refunded on orders across their order history. |
lifetime_total_net | None | The total amount of money (minus refunds) that the customer has spent across their order history. |
source_relation | None | The schema or database this record came from, if you are unioning multiple connectors. Null if not. |
lifetime_abandoned_checkouts | None | Total number of abandoned checkouts abandoned by the customer. |
customer_tags | None | A string aggregated list of all tags associated with a customer. |
avg_quantity_per_order | None | Average quantity of items per order customer orders. |
lifetime_total_tax | None | Total amount of tax attributed to the customer. |
avg_tax_per_order | None | Average tax per order attributed to the customer. |
lifetime_total_discount | None | Total discounts attributed to the customer. |
avg_discount_per_order | None | Average discount per order attributed to the customer. |
lifetime_total_shipping | None | Total shipping costs attributed to the customer. |
avg_shipping_per_order | None | Average shipping cost per order attributed to the customer. |
lifetime_total_shipping_with_discounts | None | Total shipping costs after discounts attributed to the customer. |
avg_shipping_with_discounts_per_order | None | Average shipping costs after discounts per order attributed to the customer. |
lifetime_total_shipping_tax | None | Total shipping tax attributed to the customer. |
avg_shipping_tax_per_order | None | Average shipping tax per order attributed to the customer. |
currency | None | The three-letter code (ISO 4217 format) for the currency that the customer used when they paid for their last order. Defaults to the shop currency. Returns the shop currency for test orders. |
note | None | A note about the customer. |
marketing_consent_updated_at | None | The date and time when the customer consented to receive marketing material by email. If no date is provided, then the date and time when the consent information was sent is used. |
marketing_opt_in_level | None | The marketing subscription opt-in level, as described in the M3AAWG Sender Best Common Practices, that the customer gave when they consented to receive marketing material by email. |
marketing_consent_state | None | Field indicating if the customer has consented to receive marketing material via email. Coalescing of the new `email_marketing_consent_state` field and the deprecated `accepts_marketing` field. Records with the old field will be marked with '(legacy)'. |
This SQL query aggregates customer order data from various Shopify-related tables. It calculates customer-level metrics such as first and most recent order dates, average order value, lifetime total spent, refunded amounts, order counts, and various aggregations related to quantity, tax, discounts, and shipping. The query also includes customer tags and filters out certain transaction types. The result provides a comprehensive overview of each customer's purchasing behavior and order history.
FilteringIntegrationAggregationFeaturizationWITH orders AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order
WHERE
NOT customer_id IS NULL
), order_aggregates AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders__order_line_aggregates
), transactions AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__transactions
WHERE
LOWER(status) = 'success'
AND NOT LOWER(kind) IN ('authorization', 'void')
AND LOWER(gateway) <> 'gift_card' /* redeeming a giftcard does not introduce new revenue */
), transaction_aggregates AS (
/* this is necessary as customers can pay via multiple payment gateways */
SELECT
order_id,
source_relation,
LOWER(kind) AS kind,
SUM(currency_exchange_calculated_amount) AS currency_exchange_calculated_amount
FROM transactions
GROUP BY
1,
2,
3
), customer_tags AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer_tag
), customer_tags_aggregated AS (
SELECT
customer_id,
source_relation,
LISTAGG(DISTINCT CAST(value AS TEXT), ', ') AS customer_tags
FROM customer_tags
GROUP BY
1,
2
), aggregated AS (
SELECT
orders.customer_id,
orders.source_relation,
customer_tags_aggregated.customer_tags,
MIN(orders.created_timestamp) AS first_order_timestamp,
MAX(orders.created_timestamp) AS most_recent_order_timestamp,
AVG(transaction_aggregates.currency_exchange_calculated_amount) AS avg_order_value,
SUM(transaction_aggregates.currency_exchange_calculated_amount) AS lifetime_total_spent,
SUM(refunds.currency_exchange_calculated_amount) AS lifetime_total_refunded,
COUNT(DISTINCT orders.order_id) AS lifetime_count_orders,
AVG(order_aggregates.order_total_quantity) AS avg_quantity_per_order,
SUM(order_aggregates.order_total_tax) AS lifetime_total_tax,
AVG(order_aggregates.order_total_tax) AS avg_tax_per_order,
SUM(order_aggregates.order_total_discount) AS lifetime_total_discount,
AVG(order_aggregates.order_total_discount) AS avg_discount_per_order,
SUM(order_aggregates.order_total_shipping) AS lifetime_total_shipping,
AVG(order_aggregates.order_total_shipping) AS avg_shipping_per_order,
SUM(order_aggregates.order_total_shipping_with_discounts) AS lifetime_total_shipping_with_discounts,
AVG(order_aggregates.order_total_shipping_with_discounts) AS avg_shipping_with_discounts_per_order,
SUM(order_aggregates.order_total_shipping_tax) AS lifetime_total_shipping_tax,
AVG(order_aggregates.order_total_shipping_tax) AS avg_shipping_tax_per_order
FROM orders
LEFT JOIN transaction_aggregates
ON orders.order_id = transaction_aggregates.order_id
AND orders.source_relation = transaction_aggregates.source_relation
AND transaction_aggregates.kind IN ('sale', 'capture')
LEFT JOIN transaction_aggregates AS refunds
ON orders.order_id = refunds.order_id
AND orders.source_relation = refunds.source_relation
AND refunds.kind = 'refund'
LEFT JOIN order_aggregates
ON orders.order_id = order_aggregates.order_id
AND orders.source_relation = order_aggregates.source_relation
LEFT JOIN customer_tags_aggregated
ON orders.customer_id = customer_tags_aggregated.customer_id
AND orders.source_relation = customer_tags_aggregated.source_relation
GROUP BY
1,
2,
3
)
SELECT
*
FROM aggregated
Name | Type | Comment |
---|
This SQL query creates a daily summary of shop performance metrics, combining order data, abandoned checkout information, and shop details. It calculates various aggregates such as order counts, sales figures, discounts, refunds, and product quantities. The query joins multiple tables to provide a comprehensive daily view of each shop's activities, including both successful orders and abandoned checkouts.
FilteringIntegrationAggregationFeaturizationWITH __dbt__cte__int_shopify__daily_orders AS (
WITH orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
WHERE
NOT COALESCE(is_deleted, FALSE)
), order_lines AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__order_lines
), order_aggregates AS (
SELECT
source_relation,
CAST(DATE_TRUNC('DAY', created_timestamp) AS DATE) AS date_day,
COUNT(DISTINCT order_id) AS count_orders,
SUM(line_item_count) AS count_line_items,
AVG(line_item_count) AS avg_line_item_count,
COUNT(DISTINCT customer_id) AS count_customers,
COUNT(DISTINCT email) AS count_customer_emails,
SUM(order_adjusted_total) AS order_adjusted_total,
AVG(order_adjusted_total) AS avg_order_value,
SUM(shipping_cost) AS shipping_cost,
SUM(order_adjustment_amount) AS order_adjustment_amount,
SUM(order_adjustment_tax_amount) AS order_adjustment_tax_amount,
SUM(refund_subtotal) AS refund_subtotal,
SUM(refund_total_tax) AS refund_total_tax,
SUM(total_discounts) AS total_discounts,
AVG(total_discounts) AS avg_discount,
SUM(shipping_discount_amount) AS shipping_discount_amount,
AVG(shipping_discount_amount) AS avg_shipping_discount_amount,
SUM(percentage_calc_discount_amount) AS percentage_calc_discount_amount,
AVG(percentage_calc_discount_amount) AS avg_percentage_calc_discount_amount,
SUM(fixed_amount_discount_amount) AS fixed_amount_discount_amount,
AVG(fixed_amount_discount_amount) AS avg_fixed_amount_discount_amount,
SUM(count_discount_codes_applied) AS count_discount_codes_applied,
COUNT(DISTINCT location_id) AS count_locations_ordered_from,
SUM(CASE WHEN count_discount_codes_applied > 0 THEN 1 ELSE 0 END) AS count_orders_with_discounts,
SUM(CASE WHEN refund_subtotal > 0 THEN 1 ELSE 0 END) AS count_orders_with_refunds,
MIN(created_timestamp) AS first_order_timestamp,
MAX(created_timestamp) AS last_order_timestamp
FROM orders
GROUP BY
1,
2
), order_line_aggregates AS (
SELECT
order_lines.source_relation,
CAST(DATE_TRUNC('DAY', orders.created_timestamp) AS DATE) AS date_day,
SUM(order_lines.quantity) AS quantity_sold,
SUM(order_lines.refunded_quantity) AS quantity_refunded,
SUM(order_lines.quantity_net_refunds) AS quantity_net,
SUM(order_lines.quantity) / COUNT(DISTINCT order_lines.order_id) AS avg_quantity_sold,
SUM(order_lines.quantity_net_refunds) / COUNT(DISTINCT order_lines.order_id) AS avg_quantity_net,
COUNT(DISTINCT order_lines.variant_id) AS count_variants_sold,
COUNT(DISTINCT order_lines.product_id) AS count_products_sold,
SUM(
CASE WHEN order_lines.is_gift_card THEN order_lines.quantity_net_refunds ELSE 0 END
) AS quantity_gift_cards_sold,
SUM(
CASE
WHEN order_lines.is_shipping_required
THEN order_lines.quantity_net_refunds
ELSE 0
END
) AS quantity_requiring_shipping
FROM order_lines
LEFT JOIN orders /* just joining with order to get the created_timestamp */
ON order_lines.order_id = orders.order_id
AND order_lines.source_relation = orders.source_relation
GROUP BY
1,
2
), final AS (
SELECT
order_aggregates.*,
order_line_aggregates.quantity_sold,
order_line_aggregates.quantity_refunded,
order_line_aggregates.quantity_net,
order_line_aggregates.count_variants_sold,
order_line_aggregates.count_products_sold,
order_line_aggregates.quantity_gift_cards_sold,
order_line_aggregates.quantity_requiring_shipping,
order_line_aggregates.avg_quantity_sold,
order_line_aggregates.avg_quantity_net
FROM order_aggregates
LEFT JOIN order_line_aggregates
ON order_aggregates.date_day = order_line_aggregates.date_day
AND order_aggregates.source_relation = order_line_aggregates.source_relation
)
SELECT
*
FROM final
), __dbt__cte__int_shopify__daily_abandoned_checkouts AS (
WITH abandoned_checkout AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout
/* "deleted" abandoned checkouts do not appear to have any data tying them to customers, */ /* discounts, or products (and should therefore not get joined in) but let's filter them out here */
WHERE
NOT COALESCE(is_deleted, FALSE)
), abandoned_checkout_aggregates AS (
SELECT
source_relation,
CAST(DATE_TRUNC('DAY', created_at) AS DATE) AS date_day,
COUNT(DISTINCT checkout_id) AS count_abandoned_checkouts,
COUNT(DISTINCT customer_id) AS count_customers_abandoned_checkout,
COUNT(DISTINCT email) AS count_customer_emails_abandoned_checkout
FROM abandoned_checkout
GROUP BY
1,
2
)
SELECT
*
FROM abandoned_checkout_aggregates
), shop AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__shop
), calendar AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__calendar
WHERE
CAST(DATE_TRUNC('DAY', date_day) AS DATE) = date_day
), daily_orders AS (
SELECT
*
FROM __dbt__cte__int_shopify__daily_orders
), daily_abandoned_checkouts AS (
SELECT
*
FROM __dbt__cte__int_shopify__daily_abandoned_checkouts
), shop_calendar AS (
SELECT
CAST(DATE_TRUNC('DAY', calendar.date_day) AS DATE) AS date_day,
shop.shop_id,
shop.name,
shop.domain,
shop.is_deleted,
shop.currency,
shop.enabled_presentment_currencies,
shop.iana_timezone,
shop.created_at,
shop.source_relation
FROM calendar
JOIN shop
ON CAST(shop.created_at AS DATE) <= calendar.date_day
), final AS (
SELECT
shop_calendar.*,
COALESCE(daily_orders.count_orders, 0) AS count_orders,
COALESCE(daily_orders.count_line_items, 0) AS count_line_items,
daily_orders.avg_line_item_count,
COALESCE(daily_orders.count_customers, 0) AS count_customers,
COALESCE(daily_orders.count_customer_emails, 0) AS count_customer_emails,
COALESCE(daily_orders.order_adjusted_total, 0) AS order_adjusted_total,
daily_orders.avg_order_value,
COALESCE(daily_orders.shipping_cost, 0) AS shipping_cost,
COALESCE(daily_orders.order_adjustment_amount, 0) AS order_adjustment_amount,
COALESCE(daily_orders.order_adjustment_tax_amount, 0) AS order_adjustment_tax_amount,
COALESCE(daily_orders.refund_subtotal, 0) AS refund_subtotal,
COALESCE(daily_orders.refund_total_tax, 0) AS refund_total_tax,
COALESCE(daily_orders.total_discounts, 0) AS total_discounts,
daily_orders.avg_discount,
COALESCE(daily_orders.shipping_discount_amount, 0) AS shipping_discount_amount,
daily_orders.avg_shipping_discount_amount,
COALESCE(daily_orders.percentage_calc_discount_amount, 0) AS percentage_calc_discount_amount,
daily_orders.avg_percentage_calc_discount_amount,
COALESCE(daily_orders.fixed_amount_discount_amount, 0) AS fixed_amount_discount_amount,
daily_orders.avg_fixed_amount_discount_amount,
COALESCE(daily_orders.count_discount_codes_applied, 0) AS count_discount_codes_applied,
COALESCE(daily_orders.count_locations_ordered_from, 0) AS count_locations_ordered_from,
COALESCE(daily_orders.count_orders_with_discounts, 0) AS count_orders_with_discounts,
COALESCE(daily_orders.count_orders_with_refunds, 0) AS count_orders_with_refunds,
daily_orders.first_order_timestamp,
daily_orders.last_order_timestamp,
COALESCE(daily_orders.quantity_sold, 0) AS quantity_sold,
COALESCE(daily_orders.quantity_refunded, 0) AS quantity_refunded,
COALESCE(daily_orders.quantity_net, 0) AS quantity_net,
daily_orders.avg_quantity_sold,
daily_orders.avg_quantity_net,
COALESCE(daily_orders.count_variants_sold, 0) AS count_variants_sold,
COALESCE(daily_orders.count_products_sold, 0) AS count_products_sold,
COALESCE(daily_orders.quantity_gift_cards_sold, 0) AS quantity_gift_cards_sold,
COALESCE(daily_orders.quantity_requiring_shipping, 0) AS quantity_requiring_shipping,
COALESCE(daily_abandoned_checkouts.count_abandoned_checkouts, 0) AS count_abandoned_checkouts,
COALESCE(daily_abandoned_checkouts.count_customers_abandoned_checkout, 0) AS count_customers_abandoned_checkout,
COALESCE(daily_abandoned_checkouts.count_customer_emails_abandoned_checkout, 0) AS count_customer_emails_abandoned_checkout
FROM shop_calendar
LEFT JOIN daily_orders
ON shop_calendar.source_relation = daily_orders.source_relation
AND shop_calendar.date_day = daily_orders.date_day
LEFT JOIN daily_abandoned_checkouts
ON shop_calendar.source_relation = daily_abandoned_checkouts.source_relation
AND shop_calendar.date_day = daily_abandoned_checkouts.date_day
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
date_day | None | Day for which the shop activity is being measured. |
shop_id | None | The ID for the shop. A 64-bit unsigned integer. |
name | None | The name of the shop. |
domain | None | The shop's domain. |
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
currency | None | The three-letter code (ISO 4217 format) for the shop's default currency. |
enabled_presentment_currencies | None | An array of of enabled currencies (ISO 4217 format) that the shop accepts. Merchants can enable currencies from their Shopify Payments settings in the Shopify Admin. |
iana_timezone | None | The name of the timezone assigned by the [IANA](https://www.iana.org/time-zones). |
created_at | None | The date and time (ISO 8601) when the shop was created. |
count_orders | None | Count of the distinct orders placed on this day. |
count_line_items | None | Count of the line items included in orders placed on this day. |
count_customers | None | Count of distinct customers who placed an order on this day. |
count_customer_emails | None | Count of distinct customer email addresses who placed an order on this day. |
order_adjusted_total | None | Order total adjusted for refunds and other adjustments. The calculation used for this field is as follows: total price listed on the original order (including shipping costs and discounts) + adjustments + adjustments tax - total refunds - refunds tax The order_adjusted_total will equate to the total sales - refunds listed within the transactions table for the order (after currency exchange). |
avg_order_value | None | Average adjusted total per order placed on this day (in shop currency). |
shipping_cost | None | The shipping cost of the orders placed on this day (in shop currency). |
order_adjustment_amount | None | Total adjustment amount (in shop currency) applied to the orders placed on this day. |
order_adjustment_tax_amount | None | Total tax applied to adjustments (in shop currency) on the orders placed on this day. |
refund_subtotal | None | Total refund amount applied to the orders placed on this day. |
refund_total_tax | None | Total tax applied to the refund on the orders placed on this day (in shop currency). |
total_discounts | None | The total amount of the discount allocated to this day's orders in the shop's currency. |
shipping_discount_amount | None | The total amount of discount (in shop currency) allocated toward shipping for orders placed on this day. |
percentage_calc_discount_amount | None | The total amount of discount (in shop currency) allocated via a percentage-based discount for orders placed on this day. |
fixed_amount_discount_amount | None | The total amount of discount (in shop currency) allocated via a fixed-amount discount for orders placed on this day. |
count_discount_codes_applied | None | Distinct discount codes applied by customers on orders for this day. |
count_locations_ordered_from | None | Distinct locations with orders placed against them on this day. |
count_orders_with_discounts | None | Count of orders in which a discount was applied. |
count_orders_with_refunds | None | Count of orders in which there was a refund. |
first_order_timestamp | None | Timestamp of the first order of the day for the shop. |
last_order_timestamp | None | Timestamp of the last order of the day for the shop. |
quantity_sold | None | Total quantity sold from the inventory level. Includes refunds. |
quantity_refunded | None | The quantity of goods ORDERED on this day that have been refunded. |
quantity_net | None | Net quantity sold from this shop on this day. Excludes refunds. |
count_variants_sold | None | Distinct product variants sold on this day (includes refunds). |
count_products_sold | None | Distinct products sold on this day (includes refunds). |
quantity_gift_cards_sold | None | Quantity of gift cards sold on this day. |
quantity_requiring_shipping | None | Quantity of goods sold on this day that require shipping. |
count_abandoned_checkouts | None | Count of abandoned checkouts on this day. |
count_customers_abandoned_checkout | None | Count of distinct customers who abandoned checkouts on this day. |
count_customer_emails_abandoned_checkout | None | Count of distinct customer emails who abandoned checkouts on this day. |
count_fulfillment_attempted_delivery | None | Count of distinct fulfillments on this day where the delivery of the shipment was attempted, but unable to be completed. |
count_fulfillment_delivered | None | Count of successful distinct fulfillments successfully delivered on this day. |
count_fulfillment_failure | None | Count of failed distinct fulfillments on this day. Something went wrong when pulling tracking information for the shipment, such as the tracking number was invalid or the shipment was canceled. |
count_fulfillment_in_transit | None | Count of distinct fulfillments in transit on this day. Each shipment is being transported between shipping facilities on the way to its destination. |
count_fulfillment_out_for_delivery | None | Count of distinct fulfillments being delivered to their final destination. |
count_fulfillment_ready_for_pickup | None | Count of distinct fulfillments ready for pickup at a shipping depot. |
count_fulfillment_picked_up | None | Count of distinct fulfillments successfully picked up on this day. |
count_fulfillment_label_printed | None | Count of distinct fulfillments for which a purchased shipping label has been printed on this day. |
count_fulfillment_label_purchased | None | Count of distinct fulfillments for which a shipping label has been purchased (but not printed yet) on this day. |
count_fulfillment_confirmed | None | Count of confirmed distinct fulfillments. This is the default status when no other information is available about a fulfillment. |
count_fulfillment_delayed | None | Count of delayed distinct fulfillments. |
avg_line_item_count | None | Average line item count for orders placed on this day. |
avg_discount | None | Average total discount placed on orders on this day (in shop currency). |
avg_shipping_discount_amount | None | Average discount amount (in shop currency) allocated toward shipping on orders placed on this day. |
avg_percentage_calc_discount_amount | None | Average discount amount (in shop currency) allocated via a percentage-based-discount on orders placed on this day. |
avg_fixed_amount_discount_amount | None | Average discount amount (in shop currency) allocated via a fixed-amount-discount on orders placed on this day. |
avg_quantity_sold | None | Average quantity sold per order on this day. Includes refunds. (in shop currency) |
avg_quantity_net | None | Average net quantity sold per order on this day. Excludes refunds. (in shop currency) |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query integrates data from multiple Shopify-related tables to create a comprehensive view of discounts. It combines information about discount codes, price rules, orders, and abandoned checkouts. The query performs various aggregations to calculate metrics such as total discount amounts, order counts, customer counts, and shipping costs. It also joins these aggregated results with the main discount and price rule data to provide a detailed analysis of discount performance across both completed orders and abandoned checkouts.
IntegrationAggregationFeaturizationWITH __dbt__cte__int_shopify__discounts__order_aggregates AS (
WITH order_discount_code AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_discount_code
), orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
), orders_aggregated AS (
SELECT
order_discount_code.code,
order_discount_code.type,
order_discount_code.source_relation,
AVG(order_discount_code.amount) AS avg_order_discount_amount,
SUM(order_discount_code.amount) AS total_order_discount_amount,
MAX(orders.total_line_items_price) AS total_order_line_items_price, /* summing would multiply the total by the # of discount codes applied to an order */
MAX(orders.shipping_cost) AS total_order_shipping_cost, /* summing would multiply the total by the # of discount codes applied to an order */
MAX(orders.refund_subtotal + orders.refund_total_tax) AS total_order_refund_amount, /* summing would multiply the total by the # of discount codes applied to an order */
COUNT(DISTINCT customer_id) AS count_customers,
COUNT(DISTINCT email) AS count_customer_emails,
COUNT(DISTINCT order_discount_code.order_id) AS count_orders
FROM order_discount_code
JOIN orders
ON order_discount_code.order_id = orders.order_id
AND order_discount_code.source_relation = orders.source_relation
GROUP BY
1,
2,
3
)
SELECT
*
FROM orders_aggregated
), __dbt__cte__int_shopify__discounts__abandoned_checkouts AS (
WITH abandoned_checkout AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout
/* "deleted" abandoned checkouts do not appear to have any data tying them to customers, */ /* discounts, or products (and should therefore not get joined in) but let's filter them out here */
WHERE
NOT COALESCE(is_deleted, FALSE)
), abandoned_checkout_discount_code AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout_discount_code
/* we need the TYPE of discount (shipping, percentage, fixed_amount) to avoid fanning out of joins */ /* so filter out records that have this */
WHERE
COALESCE(type, '') <> ''
), abandoned_checkout_shipping_line AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout_shipping_line
), roll_up_shipping_line AS (
SELECT
checkout_id,
source_relation,
SUM(price) AS price
FROM abandoned_checkout_shipping_line
GROUP BY
1,
2
), abandoned_checkouts_aggregated AS (
SELECT
abandoned_checkout_discount_code.code,
abandoned_checkout_discount_code.type,
abandoned_checkout_discount_code.source_relation,
SUM(abandoned_checkout_discount_code.amount) AS total_abandoned_checkout_discount_amount,
SUM(COALESCE(abandoned_checkout.total_line_items_price, 0)) AS total_abandoned_checkout_line_items_price,
SUM(COALESCE(roll_up_shipping_line.price, 0)) AS total_abandoned_checkout_shipping_price,
COUNT(DISTINCT customer_id) AS count_abandoned_checkout_customers,
COUNT(DISTINCT email) AS count_abandoned_checkout_customer_emails,
COUNT(DISTINCT abandoned_checkout.checkout_id) AS count_abandoned_checkouts
FROM abandoned_checkout_discount_code
LEFT JOIN abandoned_checkout
ON abandoned_checkout_discount_code.checkout_id = abandoned_checkout.checkout_id
AND abandoned_checkout_discount_code.source_relation = abandoned_checkout.source_relation
LEFT JOIN roll_up_shipping_line
ON roll_up_shipping_line.checkout_id = abandoned_checkout_discount_code.checkout_id
AND roll_up_shipping_line.source_relation = abandoned_checkout_discount_code.source_relation
GROUP BY
1,
2,
3
)
SELECT
*
FROM abandoned_checkouts_aggregated
), discount AS (
SELECT
*,
MD5(
CAST(COALESCE(CAST(source_relation AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(discount_code_id AS TEXT), '_dbt_utils_surrogate_key_null_') AS TEXT)
) AS discounts_unique_key
FROM TEST.PUBLIC_stg_shopify.stg_shopify__discount_code
), price_rule AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__price_rule
), orders_aggregated AS (
SELECT
*
FROM __dbt__cte__int_shopify__discounts__order_aggregates
), abandoned_checkouts_aggregated AS (
SELECT
*
FROM __dbt__cte__int_shopify__discounts__abandoned_checkouts
), discount_price_rule_joined AS (
SELECT
discount.*,
price_rule.target_selection,
price_rule.target_type,
price_rule.title,
price_rule.usage_limit,
price_rule.value,
price_rule.value_type,
price_rule.allocation_limit,
price_rule.allocation_method,
price_rule.is_once_per_customer,
price_rule.customer_selection,
price_rule.prereq_min_quantity,
price_rule.prereq_max_shipping_price,
price_rule.prereq_min_subtotal,
price_rule.prereq_min_purchase_quantity_for_entitlement,
price_rule.prereq_buy_x_get_this,
price_rule.prereq_buy_this_get_y,
price_rule.starts_at,
price_rule.ends_at,
price_rule.created_at AS price_rule_created_at,
price_rule.updated_at AS price_rule_updated_at
FROM discount
LEFT JOIN price_rule
ON discount.price_rule_id = price_rule.price_rule_id
AND discount.source_relation = price_rule.source_relation
), aggregates_joined AS (
SELECT
discount_price_rule_joined.*,
COALESCE(orders_aggregated.count_orders, 0) AS count_orders,
COALESCE(abandoned_checkouts_aggregated.count_abandoned_checkouts, 0) AS count_abandoned_checkouts,
orders_aggregated.avg_order_discount_amount,
COALESCE(orders_aggregated.total_order_discount_amount, 0) AS total_order_discount_amount,
COALESCE(abandoned_checkouts_aggregated.total_abandoned_checkout_discount_amount, 0) AS total_abandoned_checkout_discount_amount,
COALESCE(orders_aggregated.total_order_line_items_price, 0) AS total_order_line_items_price,
COALESCE(orders_aggregated.total_order_shipping_cost, 0) AS total_order_shipping_cost,
COALESCE(abandoned_checkouts_aggregated.total_abandoned_checkout_shipping_price, 0) AS total_abandoned_checkout_shipping_price,
COALESCE(orders_aggregated.total_order_refund_amount, 0) AS total_order_refund_amount,
COALESCE(orders_aggregated.count_customers, 0) AS count_customers,
COALESCE(orders_aggregated.count_customer_emails, 0) AS count_customer_emails,
COALESCE(abandoned_checkouts_aggregated.count_abandoned_checkout_customers, 0) AS count_abandoned_checkout_customers,
COALESCE(abandoned_checkouts_aggregated.count_abandoned_checkout_customer_emails, 0) AS count_abandoned_checkout_customer_emails
FROM discount_price_rule_joined
LEFT JOIN orders_aggregated
ON discount_price_rule_joined.code = orders_aggregated.code
AND discount_price_rule_joined.source_relation = orders_aggregated.source_relation
AND /* in case one CODE can apply to both shipping and line items, percentages and fixed_amounts */ (
CASE
WHEN discount_price_rule_joined.target_type = 'shipping_line'
THEN 'shipping' /* when target_type = 'shipping', value_type = 'percentage' */
ELSE discount_price_rule_joined.value_type
END
) = orders_aggregated.type
LEFT JOIN abandoned_checkouts_aggregated
ON discount_price_rule_joined.code = abandoned_checkouts_aggregated.code
AND discount_price_rule_joined.source_relation = abandoned_checkouts_aggregated.source_relation
AND /* in case one CODE can apply to both shipping and line items, percentages and fixed_amounts */ (
CASE
WHEN discount_price_rule_joined.target_type = 'shipping_line'
THEN 'shipping' /* when target_type = 'shipping', value_type = 'percentage' */
ELSE discount_price_rule_joined.value_type
END
) = abandoned_checkouts_aggregated.type
)
SELECT
*
FROM aggregates_joined
Name | Type | Comment |
---|---|---|
discounts_unique_key | None | Unique key representing a discount. Hashed on 'discount_code_id' and 'source_relation'. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
code | None | The case-insensitive discount code that customers use at checkout. Shopify recommends this map onto the associated `price_rule.title`. |
created_at | None | The date and time (ISO 8601 format) when the discount code was created. |
discount_code_id | None | The ID for the discount code. |
price_rule_id | None | The ID for the price rule. |
updated_at | None | The date and time (ISO 8601 format) when the discount code was updated. |
usage_count | None | The number of times that the discount code has been redeemed. |
allocation_limit | None | The number of times the discount can be allocated on the cart - if eligible. For example a Buy 1 hat Get 1 hat for free discount can be applied 3 times on a cart having more than 6 hats, where maximum of 3 hats get discounted - if the allocation_limit is 3. Empty (null) allocation_limit means unlimited number of allocations. |
allocation_method | None | The allocation method of the price rule. Valid values include `each` (the discount is applied to each of the entitled items. For example, for a price rule that takes $15 off, each entitled line item in a checkout will be discounted by $15) and `across` (the calculated discount amount will be applied across the entitled items. For example, for a price rule that takes $15 off, the discount will be applied across all the entitled items). |
price_rule_created_at | None | The date and time (ISO 8601 format) when the price rule was created. |
customer_selection | None | The customer selection for the price rule. Valid values include `all` (the price rule is valid for all customers) and `prerequisite` (the customer must either belong to one of the customer segments specified by customer_segment_prerequisite_ids, or be one of the customers specified by prerequisite_customer_ids). |
ends_at | None | The date and time (ISO 8601 format) when the price rule ends. Must be after starts_at. |
is_once_per_customer | None | Boolean representing whether the generated discount code will be valid only for a single use per customer. This is tracked using customer ID. |
prereq_min_quantity | None | If `customer_selection` is `prerequisite`, the minimum number of items for the price rule to be applicable. The quantity of an entitled cart item must be greater than or equal to this value. |
prereq_max_shipping_price | None | If `customer_selection` is `prerequisite`, the maximum shipping price for the price rule to be applicable. The shipping price must be less than or equal to this value |
prereq_min_subtotal | None | If `customer_selection` is `prerequisite`, the minimum subtotal for the price rule to be applicable. The subtotal of the entitled cart items must be greater than or equal to this value for the discount to apply. |
prereq_min_purchase_quantity_for_entitlement | None | If `customer_selection` is `prerequisite`, the prerequisite purchase for a Buy X Get Y discount. The minimum purchase amount required to be entitled to the discount. |
prereq_buy_x_get_this | None | If `customer_selection` is `prerequisite`, in a Buy/Get ratio for a Buy X Get Y discount, this is the offered 'get' quantity. |
prereq_buy_this_get_y | None | If `customer_selection` is `prerequisite`, in a Buy/Get ratio for a Buy X Get Y discount, this defines the necessary 'buy' quantity. |
starts_at | None | The date and time (ISO 8601 format) when the price rule starts. |
target_selection | None | The target selection method of the price rule. Valid values include `all` (the price rule applies the discount to all line items in the checkout) and `entitled` (the price rule applies the discount to selected entitlements only). |
target_type | None | The target type that the price rule applies to. Valid values include `line_item` (the price rule applies to the cart's line items) and `shipping_line` (the price rule applies to the cart's shipping lines). |
title | None | The title of the price rule. This is used by the Shopify admin search to retrieve discounts. It is also displayed on the Discounts page of the Shopify admin for bulk discounts. Shopify recommends that this map onto the associated `discount_code.code`. |
price_rule_updated_at | None | The date and time (ISO 8601 format) when the price rule was updated. |
usage_limit | None | The maximum number of times the price rule can be used, per discount code. |
value | None | The value of the price rule. If if the value of `target_type` is `shipping_line`, then only -100 is accepted. The value must be negative. |
value_type | None | The value type of the price rule. Valid values include `fixed_amount` (applies a discount of value as a unit of the store's currency. For example, if value is -30 and the store's currency is USD, then $30 USD is deducted when the discount is applied) and `percentage` (applies a percentage discount of value. For example, if value is -30, then 30% will be deducted when the discount is applied). If `target_type` is `shipping_line`, then only `percentage` is accepted. |
total_order_discount_amount | None | Total monetary amount (in shop currency) of discounts taken off of orders. |
total_abandoned_checkout_discount_amount | None | Total monetary amount (in shop currency) of discounts taken off abandoned checkout orders. |
total_order_line_items_price | None | Total monetary amount (in shop currency) of line items for orders that have used this discount. |
total_order_shipping_cost | None | Total shipping costs for orders that used this discount. |
total_abandoned_checkout_shipping_price | None | Total projected shipping costs for abandoned checkouts that applied this discount first. |
total_order_refund_amount | None | Total refunded amount (in shop currency) for orders that used this discount code. |
count_customers | None | Count of distinct customers who placed orders using this discount. |
count_customer_emails | None | Count of distinct customer emails who placed orders using this discount. |
avg_order_discount_amount | None | Average amount (in shop currency) of discount taken off orders. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
count_orders | None | Count of orders in which this discount code was applied. |
count_abandoned_checkouts | None | Count of abandoned checkouts in which this discount code was applied. |
count_abandoned_checkout_customers | None | Distinct count of customers who applied this discount in a checkout that they abandoned. |
count_abandoned_checkout_customer_emails | None | Distinct count of customer emails who applied this discount in a checkout that they abandoned. |
This SQL query integrates data from multiple Shopify-related tables (inventory levels, items, locations, product variants, orders, refunds, and fulfillments) to create a comprehensive view of inventory and sales performance. It combines inventory data with sales and refund information, calculates various aggregates such as total sales, quantity sold, distinct orders and customers, and fulfillment statuses. The query also includes detailed information about products, variants, and locations, providing a holistic view of inventory across different dimensions.
IntegrationAggregationFeaturizationCleaningWITH __dbt__cte__shopify__orders__order_refunds AS (
WITH refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__refund
), order_line_refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line_refund
), refund_join AS (
SELECT
refunds.refund_id,
refunds.created_at,
refunds.order_id,
refunds.user_id,
refunds.source_relation,
order_line_refunds.order_line_refund_id,
order_line_refunds.order_line_id,
order_line_refunds.restock_type,
order_line_refunds.quantity,
order_line_refunds.subtotal,
order_line_refunds.total_tax
FROM refunds
LEFT JOIN order_line_refunds
ON refunds.refund_id = order_line_refunds.refund_id
AND refunds.source_relation = order_line_refunds.source_relation
)
SELECT
*
FROM refund_join
), __dbt__cte__int_shopify__inventory_level__aggregates AS (
WITH order_lines AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line
), fulfillment AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__fulfillment
), orders AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order
WHERE
NOT COALESCE(is_deleted, FALSE)
), refunds AS (
SELECT
*
FROM __dbt__cte__shopify__orders__order_refunds
), refunds_aggregated AS (
SELECT
order_line_id,
source_relation,
SUM(quantity) AS quantity,
SUM(COALESCE(subtotal, 0)) AS subtotal
FROM refunds
GROUP BY
1,
2
), joined AS (
SELECT
order_lines.order_line_id,
order_lines.variant_id,
order_lines.source_relation,
fulfillment.location_id, /* location id is stored in fulfillment rather than order */
orders.order_id,
orders.customer_id,
fulfillment.fulfillment_id,
LOWER(orders.email) AS email,
order_lines.pre_tax_price,
order_lines.quantity,
orders.created_timestamp AS order_created_timestamp,
fulfillment.status AS fulfillment_status,
refunds_aggregated.subtotal AS subtotal_sold_refunds,
refunds_aggregated.quantity AS quantity_sold_refunds
FROM order_lines
JOIN orders
ON order_lines.order_id = orders.order_id
AND order_lines.source_relation = orders.source_relation
JOIN fulfillment
ON orders.order_id = fulfillment.order_id
AND orders.source_relation = fulfillment.source_relation
LEFT JOIN refunds_aggregated
ON refunds_aggregated.order_line_id = order_lines.order_line_id
AND refunds_aggregated.source_relation = order_lines.source_relation
), aggregated AS (
SELECT
variant_id,
location_id,
source_relation,
SUM(pre_tax_price) AS subtotal_sold,
SUM(quantity) AS quantity_sold,
COUNT(DISTINCT order_id) AS count_distinct_orders,
COUNT(DISTINCT customer_id) AS count_distinct_customers,
COUNT(DISTINCT email) AS count_distinct_customer_emails,
MIN(order_created_timestamp) AS first_order_timestamp,
MAX(order_created_timestamp) AS last_order_timestamp,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'pending' THEN fulfillment_id END) AS count_fulfillment_pending,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'open' THEN fulfillment_id END) AS count_fulfillment_open,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'success' THEN fulfillment_id END) AS count_fulfillment_success,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'cancelled' THEN fulfillment_id END) AS count_fulfillment_cancelled,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'error' THEN fulfillment_id END) AS count_fulfillment_error,
COUNT(DISTINCT CASE WHEN fulfillment_status = 'failure' THEN fulfillment_id END) AS count_fulfillment_failure,
SUM(COALESCE(subtotal_sold_refunds, 0)) AS subtotal_sold_refunds,
SUM(COALESCE(quantity_sold_refunds, 0)) AS quantity_sold_refunds
FROM joined
GROUP BY
1,
2,
3
)
SELECT
*
FROM aggregated
), inventory_level AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__inventory_level
), inventory_item AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__inventory_item
), location AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__location
), product_variant AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_variant
), product AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product
), inventory_level_aggregated AS (
SELECT
*
FROM __dbt__cte__int_shopify__inventory_level__aggregates
), joined_info AS (
SELECT
inventory_level.*,
inventory_item.sku,
inventory_item.is_deleted AS is_inventory_item_deleted,
inventory_item.cost,
inventory_item.country_code_of_origin,
inventory_item.province_code_of_origin,
inventory_item.is_shipping_required,
inventory_item.is_inventory_quantity_tracked,
inventory_item.created_at AS inventory_item_created_at,
inventory_item.updated_at AS inventory_item_updated_at,
location.name AS location_name,
location.is_deleted AS is_location_deleted,
location.is_active AS is_location_active,
location.address_1,
location.address_2,
location.city,
location.country,
location.country_code,
location.is_legacy AS is_legacy_location,
location.province,
location.province_code,
location.phone,
location.zip,
location.created_at AS location_created_at,
location.updated_at AS location_updated_at,
product_variant.variant_id,
product_variant.product_id,
product_variant.title AS variant_title,
product_variant.inventory_policy AS variant_inventory_policy,
product_variant.price AS variant_price,
product_variant.image_id AS variant_image_id,
product_variant.fulfillment_service AS variant_fulfillment_service,
product_variant.inventory_management AS variant_inventory_management,
product_variant.is_taxable AS is_variant_taxable,
product_variant.barcode AS variant_barcode,
product_variant.grams AS variant_grams,
product_variant.inventory_quantity AS variant_inventory_quantity,
product_variant.weight AS variant_weight,
product_variant.weight_unit AS variant_weight_unit,
product_variant.option_1 AS variant_option_1,
product_variant.option_2 AS variant_option_2,
product_variant.option_3 AS variant_option_3,
product_variant.tax_code AS variant_tax_code,
product_variant.created_timestamp AS variant_created_at,
product_variant.updated_timestamp AS variant_updated_at
FROM inventory_level
JOIN inventory_item
ON inventory_level.inventory_item_id = inventory_item.inventory_item_id
AND inventory_level.source_relation = inventory_item.source_relation
JOIN location
ON inventory_level.location_id = location.location_id
AND inventory_level.source_relation = location.source_relation
JOIN product_variant
ON inventory_item.inventory_item_id = product_variant.inventory_item_id
AND inventory_item.source_relation = product_variant.source_relation
), joined_aggregates AS (
SELECT
joined_info.*,
COALESCE(inventory_level_aggregated.subtotal_sold, 0) AS subtotal_sold,
COALESCE(inventory_level_aggregated.quantity_sold, 0) AS quantity_sold,
COALESCE(inventory_level_aggregated.count_distinct_orders, 0) AS count_distinct_orders,
COALESCE(inventory_level_aggregated.count_distinct_customers, 0) AS count_distinct_customers,
COALESCE(inventory_level_aggregated.count_distinct_customer_emails, 0) AS count_distinct_customer_emails,
inventory_level_aggregated.first_order_timestamp,
inventory_level_aggregated.last_order_timestamp,
COALESCE(inventory_level_aggregated.subtotal_sold_refunds, 0) AS subtotal_sold_refunds,
COALESCE(inventory_level_aggregated.quantity_sold_refunds, 0) AS quantity_sold_refunds,
COALESCE(count_fulfillment_pending, 0) AS count_fulfillment_pending,
COALESCE(count_fulfillment_open, 0) AS count_fulfillment_open,
COALESCE(count_fulfillment_success, 0) AS count_fulfillment_success,
COALESCE(count_fulfillment_cancelled, 0) AS count_fulfillment_cancelled,
COALESCE(count_fulfillment_error, 0) AS count_fulfillment_error,
COALESCE(count_fulfillment_failure, 0) AS count_fulfillment_failure
FROM joined_info
LEFT JOIN inventory_level_aggregated
ON joined_info.location_id = inventory_level_aggregated.location_id
AND joined_info.variant_id = inventory_level_aggregated.variant_id
AND joined_info.source_relation = inventory_level_aggregated.source_relation
), final AS (
SELECT
*,
subtotal_sold - subtotal_sold_refunds AS net_subtotal_sold,
quantity_sold - quantity_sold_refunds AS net_quantity_sold
FROM joined_aggregates
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
available_quantity | None | The available quantity of an inventory item at the inventory level's associated location. Returns null if the inventory item is not tracked. |
inventory_item_id | None | The ID of the inventory item associated with the inventory level. |
location_id | None | The ID of the location that the inventory level belongs to. |
updated_at | None | The date and time (ISO 8601 format) when the inventory level was last modified. |
is_inventory_item_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
sku | None | The unique SKU (stock keeping unit) of the inventory item. |
cost | None | The unit cost of the inventory item. The shop's default currency is used. |
province_code_of_origin | None | The province code (ISO 3166-2 alpha-2) of where the item came from. The province code is only used if the shipping provider for the inventory item is Canada Post. |
country_code_of_origin | None | The country code (ISO 3166-1 alpha-2) of where the item came from. |
is_shipping_required | None | Boolean representing whether a customer needs to provide a shipping address when placing an order containing the inventory item. |
is_inventory_quantity_tracked | None | Boolean representing whether inventory levels are tracked for the item. If true, then the inventory quantity changes are tracked by Shopify. |
inventory_item_created_at | None | The date and time (ISO 8601 format) when the inventory item was created. |
inventory_item_updated_at | None | The date and time (ISO 8601 format) when the inventory item was last modified. |
is_location_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
is_location_active | None | Boolean representing whether the location is active. If true, then the location can be used to sell products, stock inventory, and fulfill orders. |
address_1 | None | The location's street address. |
address_2 | None | The optional second line of the location's street address. |
city | None | The city the location is in. |
country | None | The country the location is in (two-letter code). |
country_code | None | The two-letter code (ISO 3166-1 alpha-2 format) corresponding to country the location is in. |
location_created_at | None | The date and time (ISO 8601 format) when the location was created. |
is_legacy_location | None | Boolean representing whether this is a fulfillment service location. If true, then the location is a fulfillment service location. If false, then the location was created by the merchant and isn't tied to a fulfillment service. |
location_name | None | The name of the location. |
phone | None | The phone number of the location. This value can contain special characters, such as - or +. |
province | None | The province, state, or district of the location. |
province_code | None | The province, state, or district code (ISO 3166-2 alpha-2 format) of the location. |
location_updated_at | None | The date and time (ISO 8601 format) when the location was last updated. |
zip | None | The zip or postal code. |
variant_barcode | None | The barcode, UPC, or ISBN number for the product. |
variant_created_at | None | The date and time (ISO 8601 format) when the product variant was created. |
variant_fulfillment_service | None | The fulfillment service associated with the product variant. |
variant_grams | None | The weight of the product variant in grams. |
variant_id | None | The unique numeric identifier for the product variant. |
variant_image_id | None | The unique numeric identifier for a product's image. The image must be associated to the same product as the variant. |
variant_inventory_management | None | The fulfillment service that tracks the number of items in stock for the product variant. |
variant_inventory_policy | None | Whether customers are allowed to place an order for the product variant when it's out of stock. |
variant_inventory_quantity | None | An aggregate of inventory across all locations. To adjust inventory at a specific location, use the InventoryLevel resource. |
variant_option_1 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
variant_option_2 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
variant_option_3 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
variant_price | None | The price of the product variant in shop currency. |
product_id | None | The unique numeric identifier for the product. |
is_variant_taxable | None | Whether a tax is charged when the product variant is sold. |
variant_tax_code | None | This parameter applies only to the stores that have the Avalara AvaTax app installed. Specifies the Avalara tax code for the product variant. |
variant_title | None | The title of the product variant. The title field is a concatenation of the option1, option2, and option3 fields. You can only update title indirectly using the option fields. |
variant_updated_at | None | The date and time when the product variant was last modified. Gets returned in ISO 8601 format. |
variant_weight | None | The weight of the product variant in the unit system specified with weight_unit. |
variant_weight_unit | None | The unit of measurement that applies to the product variant's weight. If you don't specify a value for weight_unit, then the shop's default unit of measurement is applied. Valid values: g, kg, oz, and lb. |
subtotal_sold | None | Total amount (monetary, in shop currency) sold from the inventory level. Includes refunds. |
quantity_sold | None | Total quantity sold from the inventory level. Includes refunds. |
count_distinct_orders | None | The number of distinct orders placed from this inventory level. |
count_distinct_customers | None | The number of distinct customers (based on customer_id) that have placed orders against this inventory level. |
count_distinct_customer_emails | None | The number of distinct customer emails that have placed orders against this inventory level. |
first_order_timestamp | None | The timetamp of the first order against this inventory level. |
last_order_timestamp | None | The timestamp of the first order against this inventory level. |
subtotal_sold_refunds | None | The monetary amount (in shop currency) of inventory level goods that have been refunded. |
quantity_sold_refunds | None | The quantity of inventory level goods that have been refunded. |
net_subtotal_sold | None | Net monetary amount sold (in shop currency) from the inventory level. Excludes refunds. |
net_quantity_sold | None | Net quantity sold from this inventory level. Excludes refunds. |
count_fulfillment_pending | None | Count of currently pending distinct fulfillments for this inventory level. "Pending" = Shopify has created the fulfillment and is waiting for the third-party fulfillment service to transition it to 'open' or 'success'. |
count_fulfillment_open | None | Count of currently open distinct fulfillments for this inventory level. "Open" = The fulfillment has been acknowledged by the service and is in processing. |
count_fulfillment_success | None | Count of successful distinct fulfillments for this inventory level. "Success" = The fulfillment was successful. |
count_fulfillment_cancelled | None | Count of cancelled distinct fulfillments for this inventory level. "Cancelled" = The fulfillment was cancelled. |
count_fulfillment_error | None | Count of distinct fulfillments for this inventory level that encountered an error. "Error" = There was an error with the fulfillment request. |
count_fulfillment_failure | None | Count of distinct fulfillments for this inventory level that failed. "Failure" = The fulfillment request failed. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query integrates data from multiple Shopify-related tables to create a comprehensive view of order lines. It joins order line data with refund information, product variant details, and tax information. The query calculates net quantities and subtotals after refunds, and includes various product variant attributes. It also aggregates refund and tax data at the order line level.
IntegrationAggregationFeaturizationWITH __dbt__cte__shopify__orders__order_refunds AS (
WITH refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__refund
), order_line_refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line_refund
), refund_join AS (
SELECT
refunds.refund_id,
refunds.created_at,
refunds.order_id,
refunds.user_id,
refunds.source_relation,
order_line_refunds.order_line_refund_id,
order_line_refunds.order_line_id,
order_line_refunds.restock_type,
order_line_refunds.quantity,
order_line_refunds.subtotal,
order_line_refunds.total_tax
FROM refunds
LEFT JOIN order_line_refunds
ON refunds.refund_id = order_line_refunds.refund_id
AND refunds.source_relation = order_line_refunds.source_relation
)
SELECT
*
FROM refund_join
), order_lines AS (
SELECT
*,
MD5(
CAST(COALESCE(CAST(source_relation AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(order_line_id AS TEXT), '_dbt_utils_surrogate_key_null_') AS TEXT)
) AS order_lines_unique_key
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line
), product_variants AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_variant
), refunds AS (
SELECT
*
FROM __dbt__cte__shopify__orders__order_refunds
), refunds_aggregated AS (
SELECT
order_line_id,
source_relation,
SUM(quantity) AS quantity,
SUM(COALESCE(subtotal, 0)) AS subtotal,
LISTAGG(DISTINCT CAST(refunds.restock_type AS TEXT), ', ') AS restock_types
FROM refunds
GROUP BY
1,
2
), tax_lines AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__tax_line
), tax_lines_aggregated AS (
SELECT
tax_lines.order_line_id,
tax_lines.source_relation,
SUM(tax_lines.price) AS order_line_tax
FROM tax_lines
GROUP BY
1,
2
), joined AS (
SELECT
order_lines.*,
refunds_aggregated.restock_types,
COALESCE(refunds_aggregated.quantity, 0) AS refunded_quantity,
COALESCE(refunds_aggregated.subtotal, 0) AS refunded_subtotal,
order_lines.quantity - COALESCE(refunds_aggregated.quantity, 0) AS quantity_net_refunds,
order_lines.pre_tax_price - COALESCE(refunds_aggregated.subtotal, 0) AS subtotal_net_refunds,
product_variants.created_timestamp AS variant_created_at,
product_variants.updated_timestamp AS variant_updated_at,
product_variants.inventory_item_id,
product_variants.image_id,
product_variants.price AS variant_price,
product_variants.sku AS variant_sku,
product_variants.position AS variant_position,
product_variants.inventory_policy AS variant_inventory_policy,
product_variants.compare_at_price AS variant_compare_at_price,
product_variants.fulfillment_service AS variant_fulfillment_service,
product_variants.is_taxable AS variant_is_taxable,
product_variants.barcode AS variant_barcode,
product_variants.grams AS variant_grams,
product_variants.inventory_quantity AS variant_inventory_quantity,
product_variants.weight AS variant_weight,
product_variants.weight_unit AS variant_weight_unit,
product_variants.option_1 AS variant_option_1,
product_variants.option_2 AS variant_option_2,
product_variants.option_3 AS variant_option_3,
product_variants.tax_code AS variant_tax_code,
tax_lines_aggregated.order_line_tax
FROM order_lines
LEFT JOIN refunds_aggregated
ON refunds_aggregated.order_line_id = order_lines.order_line_id
AND refunds_aggregated.source_relation = order_lines.source_relation
LEFT JOIN product_variants
ON product_variants.variant_id = order_lines.variant_id
AND product_variants.source_relation = order_lines.source_relation
LEFT JOIN tax_lines_aggregated
ON tax_lines_aggregated.order_line_id = order_lines.order_line_id
AND tax_lines_aggregated.source_relation = order_lines.source_relation
)
SELECT
*
FROM joined
Name | Type | Comment |
---|---|---|
order_lines_unique_key | None | Unique key representing an order line. Hashed on 'order_line_id' and 'source_relation'. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
fulfillable_quantity | None | The amount available to fulfill, calculated as follows: quantity - max(refunded_quantity, fulfilled_quantity) - pending_fulfilled_quantity - open_fulfilled_quantity |
fulfillment_status | None | How far along an order is in terms line items fulfilled. |
is_gift_card | None | Whether the item is a gift card. If true, then the item is not taxed or considered for shipping charges. |
grams | None | The weight of the item in grams. |
order_line_id | None | The ID of the line item. |
name | None | The name of the product variant. |
order_id | None | The ID of the related order. |
price | None | The price of the item before discounts have been applied in the shop currency. |
product_id | None | The ID of the product that the line item belongs to. Can be null if the original product associated with the order is deleted at a later date. |
quantity | None | The number of items that were purchased. |
is_shipping_required | None | Whether the item requires shipping. |
sku | None | The item's SKU (stock keeping unit). |
is_taxable | None | Whether the item was taxable. |
title | None | The title of the product. |
total_discount | None | The total amount of the discount allocated to the line item in the shop currency. |
variant_id | None | The ID of the product variant. |
vendor | None | The name of the item's supplier. |
refunded_quantity | None | Quantity of the item that has been refunded. |
quantity_net_refunds | None | Quantity ordered, excluding refunds. |
variant_barcode | None | The barcode, UPC, or ISBN number for the product. |
variant_compare_at_price | None | The original price of the item before an adjustment or a sale in shop currency. |
variant_created_at | None | The date and time (ISO 8601 format) when the product variant was created. |
variant_fulfillment_service | None | The fulfillment service associated with the product variant. |
variant_grams | None | The weight of the product variant in grams. |
inventory_item_id | None | The unique identifier for the inventory item, which is used in the Inventory API to query for inventory information. |
variant_inventory_management | None | The fulfillment service that tracks the number of items in stock for the product variant. |
variant_inventory_policy | None | Whether customers are allowed to place an order for the product variant when it's out of stock. |
variant_inventory_quantity | None | An aggregate of inventory across all locations. To adjust inventory at a specific location, use the InventoryLevel resource. |
variant_option_1 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
variant_option_2 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
variant_option_3 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
variant_position | None | The order of the product variant in the list of product variants. The first position in the list is 1. The position of variants is indicated by the order in which they are listed. |
variant_price | None | The price of the product variant. |
variant_sku | None | A unique identifier for the product variant in the shop. Required in order to connect to a FulfillmentService. |
variant_is_taxable | None | Whether a tax is charged when the product variant is sold. |
variant_tax_code | None | This parameter applies only to the stores that have the Avalara AvaTax app installed. Specifies the Avalara tax code for the product variant. |
variant_title | None | The title of the product variant. The title field is a concatenation of the option1, option2, and option3 fields. You can only update title indirectly using the option fields. |
variant_updated_at | None | The date and time when the product variant was last modified. Gets returned in ISO 8601 format. |
variant_weight | None | The weight of the product variant in the unit system specified with weight_unit. |
variant_weight_unit | None | The unit of measurement that applies to the product variant's weight. If you don't specify a value for weight_unit, then the shop's default unit of measurement is applied. Valid values: g, kg, oz, and lb. |
refunded_subtotal | None | Subtotal amount of the refund applied to the order line in shop currency. |
subtotal_net_refunds | None | Subtotal of the order line with refunds subtracted in shop currency. |
image_id | None | Image id of the product variant associated with the order line. |
source_relation | None | The schema or database this record came from, if you are unioning multiple connectors. Null if not. |
restock_types | None | List of how this refund line item affects inventory levels. |
order_line_tax | None | Total taxes for the line item. |
index | None | The index associated with the order. |
pre_tax_price | None | The total pre tax price of the order. |
tax_code | None | This parameter applies only to the stores that have the Avalara AvaTax app installed. Specifies the Avalara tax code for the product variant. |
pre_tax_price_set | None | The pre tax price of the line item in shop currency and presentment currency. |
price_set | None | The price of the line item in shop and presentment currencies. |
total_discount_set | None | The total amount allocated to the line item in the presentment currency. |
properties | None | Line item properties. |
This SQL query combines and transforms data from multiple Shopify-related tables to create a comprehensive view of orders. It joins order information with related data such as refunds, adjustments, discounts, tags, and fulfillments. The query calculates various aggregates and derived fields, including adjusted totals, discount amounts, and shipping information. It also determines whether each order is from a new or repeat customer based on the order sequence for each customer.
IntegrationAggregationFeaturizationCleaningWITH __dbt__cte__shopify__orders__order_refunds AS (
WITH refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__refund
), order_line_refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line_refund
), refund_join AS (
SELECT
refunds.refund_id,
refunds.created_at,
refunds.order_id,
refunds.user_id,
refunds.source_relation,
order_line_refunds.order_line_refund_id,
order_line_refunds.order_line_id,
order_line_refunds.restock_type,
order_line_refunds.quantity,
order_line_refunds.subtotal,
order_line_refunds.total_tax
FROM refunds
LEFT JOIN order_line_refunds
ON refunds.refund_id = order_line_refunds.refund_id
AND refunds.source_relation = order_line_refunds.source_relation
)
SELECT
*
FROM refund_join
), orders AS (
SELECT
*,
MD5(
CAST(COALESCE(CAST(source_relation AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(order_id AS TEXT), '_dbt_utils_surrogate_key_null_') AS TEXT)
) AS orders_unique_key
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order
), order_lines AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders__order_line_aggregates
), order_adjustments AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_adjustment
), order_adjustments_aggregates AS (
SELECT
order_id,
source_relation,
SUM(amount) AS order_adjustment_amount,
SUM(tax_amount) AS order_adjustment_tax_amount
FROM order_adjustments
GROUP BY
1,
2
), refunds AS (
SELECT
*
FROM __dbt__cte__shopify__orders__order_refunds
), refund_aggregates AS (
SELECT
order_id,
source_relation,
SUM(subtotal) AS refund_subtotal,
SUM(total_tax) AS refund_total_tax
FROM refunds
GROUP BY
1,
2
), order_discount_code AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_discount_code
), discount_aggregates AS (
SELECT
order_id,
source_relation,
SUM(CASE WHEN type = 'shipping' THEN amount ELSE 0 END) AS shipping_discount_amount,
SUM(CASE WHEN type = 'percentage' THEN amount ELSE 0 END) AS percentage_calc_discount_amount,
SUM(CASE WHEN type = 'fixed_amount' THEN amount ELSE 0 END) AS fixed_amount_discount_amount,
COUNT(DISTINCT code) AS count_discount_codes_applied
FROM order_discount_code
GROUP BY
1,
2
), order_tag AS (
SELECT
order_id,
source_relation,
LISTAGG(DISTINCT CAST(value AS TEXT), ', ') AS order_tags
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_tag
GROUP BY
1,
2
), order_url_tag AS (
SELECT
order_id,
source_relation,
LISTAGG(DISTINCT CAST(value AS TEXT), ', ') AS order_url_tags
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_url_tag
GROUP BY
1,
2
), fulfillments AS (
SELECT
order_id,
source_relation,
COUNT(fulfillment_id) AS number_of_fulfillments,
LISTAGG(DISTINCT CAST(service AS TEXT), ', ') AS fulfillment_services,
LISTAGG(DISTINCT CAST(tracking_company AS TEXT), ', ') AS tracking_companies,
LISTAGG(DISTINCT CAST(tracking_number AS TEXT), ', ') AS tracking_numbers
FROM TEST.PUBLIC_stg_shopify.stg_shopify__fulfillment
GROUP BY
1,
2
), joined AS (
SELECT
orders.*,
COALESCE(CAST(PARSE_JSON(total_shipping_price_set)['shop_money']['amount'] AS FLOAT), 0) AS shipping_cost,
order_adjustments_aggregates.order_adjustment_amount,
order_adjustments_aggregates.order_adjustment_tax_amount,
refund_aggregates.refund_subtotal,
refund_aggregates.refund_total_tax,
(
orders.total_price + COALESCE(order_adjustments_aggregates.order_adjustment_amount, 0) + COALESCE(order_adjustments_aggregates.order_adjustment_tax_amount, 0) - COALESCE(refund_aggregates.refund_subtotal, 0) - COALESCE(refund_aggregates.refund_total_tax, 0)
) AS order_adjusted_total,
order_lines.line_item_count,
COALESCE(discount_aggregates.shipping_discount_amount, 0) AS shipping_discount_amount,
COALESCE(discount_aggregates.percentage_calc_discount_amount, 0) AS percentage_calc_discount_amount,
COALESCE(discount_aggregates.fixed_amount_discount_amount, 0) AS fixed_amount_discount_amount,
COALESCE(discount_aggregates.count_discount_codes_applied, 0) AS count_discount_codes_applied,
COALESCE(order_lines.order_total_shipping_tax, 0) AS order_total_shipping_tax,
order_tag.order_tags,
order_url_tag.order_url_tags,
fulfillments.number_of_fulfillments,
fulfillments.fulfillment_services,
fulfillments.tracking_companies,
fulfillments.tracking_numbers
FROM orders
LEFT JOIN order_lines
ON orders.order_id = order_lines.order_id
AND orders.source_relation = order_lines.source_relation
LEFT JOIN refund_aggregates
ON orders.order_id = refund_aggregates.order_id
AND orders.source_relation = refund_aggregates.source_relation
LEFT JOIN order_adjustments_aggregates
ON orders.order_id = order_adjustments_aggregates.order_id
AND orders.source_relation = order_adjustments_aggregates.source_relation
LEFT JOIN discount_aggregates
ON orders.order_id = discount_aggregates.order_id
AND orders.source_relation = discount_aggregates.source_relation
LEFT JOIN order_tag
ON orders.order_id = order_tag.order_id
AND orders.source_relation = order_tag.source_relation
LEFT JOIN order_url_tag
ON orders.order_id = order_url_tag.order_id
AND orders.source_relation = order_url_tag.source_relation
LEFT JOIN fulfillments
ON orders.order_id = fulfillments.order_id
AND orders.source_relation = fulfillments.source_relation
), windows AS (
SELECT
*,
ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY created_timestamp) AS customer_order_seq_number
FROM joined
), new_vs_repeat AS (
SELECT
*,
CASE WHEN customer_order_seq_number = 1 THEN 'new' ELSE 'repeat' END AS new_vs_repeat
FROM windows
)
SELECT
*
FROM new_vs_repeat
Name | Type | Comment |
---|---|---|
orders_unique_key | None | Unique key representing an order. Hashed on 'order_id' and 'source_relation'. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
app_id | None | The ID of the app that created the order. |
billing_address_address_1 | None | The street address of the billing address. |
billing_address_address_2 | None | An optional additional field for the street address of the billing address. |
billing_address_city | None | The city, town, or village of the billing address. |
billing_address_company | None | The company of the person associated with the billing address. |
billing_address_country | None | The name of the country of the billing address. |
billing_address_country_code | None | The two-letter code (ISO 3166-1 format) for the country of the billing address. |
billing_address_first_name | None | The first name of the person associated with the payment method. |
billing_address_last_name | None | The last name of the person associated with the payment method. |
billing_address_latitude | None | The latitude of the billing address. |
billing_address_longitude | None | The longitude of the billing address. |
billing_address_name | None | The full name of the person associated with the payment method. |
billing_address_phone | None | The phone number at the billing address. |
billing_address_province | None | The name of the region (province, state, prefecture, …) of the billing address. |
billing_address_province_code | None | The two-letter abbreviation of the region of the billing address. |
billing_address_zip | None | The postal code (zip, postcode, Eircode, …) of the billing address. |
browser_ip | None | The IP address of the browser used by the customer when they placed the order. |
has_buyer_accepted_marketing | None | Whether the customer consented to receive email updates from the shop. |
cancel_reason | None | The reason why the order was canceled. |
cancelled_timestamp | None | The date and time when the order was canceled. |
cart_token | None | The ID of the cart that's associated with the order. |
closed_timestamp | None | The date and time when the order was closed (archived). |
created_timestamp | None | The autogenerated date and time when the order was created in Shopify. |
currency | None | The three-letter code for the shop currency. |
customer_id | None | The ID of the order's customer. |
None | The customer's email address. | |
financial_status | None | The status of payments associated with the order. Can only be set when the order is created |
fulfillment_status | None | The order's status in terms of fulfilled line items. |
order_id | None | The ID of the order, used for API purposes. This is different from the order_number property, which is the ID used by the shop owner and customer. |
landing_site_base_url | None | The URL for the page where the buyer landed when they entered the shop. |
location_id | None | The ID of the physical location where the order was processed. |
name | None | The order name, generated by combining the order_number property with the order prefix and suffix that are set in the merchant's general settings. |
note | None | An optional note that a shop owner can attach to the order. |
number | None | The order's position in the shop's count of orders. Numbers are sequential and start at 1. |
order_number | None | The order 's position in the shop's count of orders starting at 1001. Order numbers are sequential and start at 1001. |
processed_timestamp | None | The date and time when an order was processed. This value is the date that appears on your orders and that's used in the analytic reports. |
referring_site | None | The website where the customer clicked a link to the shop. |
shipping_address_address_1 | None | The street address of the shipping address. |
shipping_address_address_2 | None | An optional additional field for the street address of the shipping address. |
shipping_address_city | None | The city, town, or village of the shipping address. |
shipping_address_company | None | The company of the person associated with the shipping address. |
shipping_address_country | None | The name of the country of the shipping address. |
shipping_address_country_code | None | The two-letter code (ISO 3166-1 format) for the country of the shipping address. |
shipping_address_first_name | None | The first name of the person associated with the shipping address. |
shipping_address_last_name | None | The last name of the person associated with the shipping address. |
shipping_address_latitude | None | The latitude of the shipping address. |
shipping_address_longitude | None | The longitude of the shipping address. |
shipping_address_name | None | The full name of the person associated with the payment method. |
shipping_address_phone | None | The phone number at the shipping address. |
shipping_address_province | None | The name of the region (province, state, prefecture, …) of the shipping address. |
shipping_address_province_code | None | The two-letter abbreviation of the region of the shipping address. |
shipping_address_zip | None | The postal code (zip, postcode, Eircode, …) of the shipping address. |
source_name | None | Where the order originated. Can be set only during order creation, and is not writeable afterwards. |
source_relation | None | The ID of the order placed on the originating platform. This value doesn't correspond to the Shopify ID that's generated from a completed draft. |
subtotal_price | None | The price of the order in the shop currency after discounts but before shipping, taxes, and tips. |
has_taxes_included | None | Whether taxes are included in the order subtotal. |
is_test_order | None | Whether this is a test order. |
token | None | A unique token for the order. |
total_discounts | None | The total discounts applied to the price of the order in the shop currency. |
total_line_items_price | None | The sum of all line item prices in the shop currency. |
total_price | None | The sum of all line item prices, discounts, shipping, taxes, and tips in the shop currency. Must be positive. |
total_tax | None | The sum of all the taxes applied to the order in th shop currency. Must be positive. |
total_weight | None | The sum of all line item weights in grams. |
updated_timestamp | None | The date and time (ISO 8601 format) when the order was last modified. |
user_id | None | The ID of the user logged into Shopify POS who processed the order, if applicable. |
line_item_count | None | Number of line items included in the order. |
customer_order_seq_number | None | The sequential number of the order as it relates to the customer |
new_vs_repeat | None | Whether the order was a new or repeat order for the customer. |
shipping_cost | None | The shipping cost of the order. |
order_adjustment_amount | None | Total adjustment amount applied to the order in shop currency. |
order_adjustment_tax_amount | None | Total tax applied to the adjustment on the order in shop currency. |
refund_subtotal | None | Total refund amount applied to the order in shop currency. |
refund_total_tax | None | Total tax applied to the refund on the order in shop currency. |
order_adjusted_total | None | Order total adjusted for refunds and other adjustments. The calculation used for this field is as follows: total price listed on the original order (including shipping costs and discounts) + adjustments + adjustments tax - total refunds - refunds tax The order_adjusted_total will equate to the total sales - refunds listed within the transactions table for the order (after currency exchange). |
checkout_token | None | The checkout token applied to the order. |
total_shipping_price_set | None | The total shipping price set to the order. |
order_total_shipping_tax | None | Total shipping tax attributed to the order. |
order_tags | None | List of tags associated with the order. |
order_url_tags | None | List of url tags associated with the order. |
number_of_fulfillments | None | Total fulfillments for the order. |
fulfillment_services | None | List of fulfillment services for the order. |
tracking_companies | None | List of tracking companies for the order. |
tracking_numbers | None | List of tracking numbers for the order. |
total_tip_received | None | The sum of all the tips in the order in the shop currency. |
checkout_id | None | The ID for the checkout. |
client_details_user_agent | None | Details of the browsing client, including software and operating versions. |
customer_locale | None | The two or three-letter language code, optionally followed by a region modifier. Example values - en, en-CA. |
order_status_url | None | The URL pointing to the order status web page, if applicable. |
presentment_currency | None | The three-letter code (ISO 4217 format) of the currency that the customer used at checkout. For the shop's default currency, see `currency`. |
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
total_discounts_set | None | The total discounts applied to the price of the order in shop and presentment currencies. |
total_line_items_price_set | None | The total of all line item prices in shop and presentment currencies. |
total_price_set | None | The total price of the order in shop and presentment currencies. |
total_tax_set | None | The total tax applied to the order in shop and presentment currencies. |
is_confirmed | None | Whether the order is confirmed. |
shipping_discount_amount | None | The total amount of discount (in shop currency) allocated toward shipping. |
percentage_calc_discount_amount | None | The total amount of discount (in shop currency) allocated via a percentage-based discount |
fixed_amount_discount_amount | None | The total amount of discount (in shop currency) allocated via a fixed-amount discount |
count_discount_codes_applied | None | Distinct discount codes applied on the order |
This SQL query aggregates order-related data from various Shopify tables. It calculates metrics such as line item count, total quantity, total tax, total discount, shipping costs, and shipping taxes for each order. The query joins information from order lines, tax lines, and shipping details to create a comprehensive summary of order aggregates.
IntegrationAggregationWITH __dbt__cte__int_shopify__order__shipping_aggregates AS (
WITH order_shipping_line AS (
SELECT
order_id,
source_relation,
order_shipping_line_id,
SUM(price) AS shipping_price,
SUM(discounted_price) AS discounted_shipping_price
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_shipping_line
GROUP BY
1,
2,
3
), order_shipping_tax_line AS (
SELECT
order_shipping_line_id,
source_relation,
SUM(price) AS shipping_tax
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_shipping_tax_line
GROUP BY
1,
2
), aggregated AS (
SELECT
order_shipping_line.order_id,
order_shipping_line.source_relation,
SUM(order_shipping_line.shipping_price) AS shipping_price,
SUM(order_shipping_line.discounted_shipping_price) AS discounted_shipping_price,
SUM(order_shipping_tax_line.shipping_tax) AS shipping_tax
FROM order_shipping_line
LEFT JOIN order_shipping_tax_line
ON order_shipping_line.order_shipping_line_id = order_shipping_tax_line.order_shipping_line_id
AND order_shipping_line.source_relation = order_shipping_tax_line.source_relation
GROUP BY
1,
2
)
SELECT
*
FROM aggregated
), order_line AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line
), tax AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__tax_line
), shipping AS (
SELECT
*
FROM __dbt__cte__int_shopify__order__shipping_aggregates
), tax_aggregates AS (
SELECT
order_line_id,
source_relation,
SUM(price) AS price
FROM tax
GROUP BY
1,
2
), order_line_aggregates AS (
SELECT
order_line.order_id,
order_line.source_relation,
COUNT(*) AS line_item_count,
SUM(order_line.quantity) AS order_total_quantity,
SUM(tax_aggregates.price) AS order_total_tax,
SUM(order_line.total_discount) AS order_total_discount
FROM order_line
LEFT JOIN tax_aggregates
ON tax_aggregates.order_line_id = order_line.order_line_id
AND tax_aggregates.source_relation = order_line.source_relation
GROUP BY
1,
2
), final AS (
SELECT
order_line_aggregates.order_id,
order_line_aggregates.source_relation,
order_line_aggregates.line_item_count,
order_line_aggregates.order_total_quantity,
order_line_aggregates.order_total_tax,
order_line_aggregates.order_total_discount,
shipping.shipping_price AS order_total_shipping,
shipping.discounted_shipping_price AS order_total_shipping_with_discounts,
shipping.shipping_tax AS order_total_shipping_tax
FROM order_line_aggregates
LEFT JOIN shipping
ON shipping.order_id = order_line_aggregates.order_id
AND shipping.source_relation = order_line_aggregates.source_relation
)
SELECT
*
FROM final
Name | Type | Comment |
---|
This SQL query integrates data from two staging tables: 'stg_shopify__refund' and 'stg_shopify__order_line_refund'. It joins these tables based on the refund_id and source_relation columns, combining information about refunds with their corresponding order line details. The query selects all columns from both tables, providing a comprehensive view of refund information along with associated order line refund data.
IntegrationWITH refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__refund
), order_line_refunds AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line_refund
), refund_join AS (
SELECT
refunds.refund_id,
refunds.created_at,
refunds.order_id,
refunds.user_id,
refunds.source_relation,
order_line_refunds.order_line_refund_id,
order_line_refunds.order_line_id,
order_line_refunds.restock_type,
order_line_refunds.quantity,
order_line_refunds.subtotal,
order_line_refunds.total_tax
FROM refunds
LEFT JOIN order_line_refunds
ON refunds.refund_id = order_line_refunds.refund_id
AND refunds.source_relation = order_line_refunds.source_relation
)
SELECT
*
FROM refund_join
Name | Type | Comment |
---|
This SQL query integrates data from multiple Shopify-related tables to create a comprehensive view of products. It joins product information with collections, tags, variants, and images. The query also aggregates order line data to add sales-related metrics to each product. It includes various calculations such as quantity sold, subtotal sold, average quantity per order line, total discounts, and taxes. The result is a detailed product report that combines product attributes with sales performance data.
IntegrationAggregationFeaturizationWITH __dbt__cte__int_shopify__products_with_aggregates AS (
WITH products AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product
), collection_product AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__collection_product
), collection AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__collection
WHERE
NOT COALESCE(is_deleted, FALSE) /* limit to only active collections */
), product_tag AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_tag
), product_variant AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_variant
), product_image AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_image
), collections_aggregated AS (
SELECT
collection_product.product_id,
collection_product.source_relation,
LISTAGG(collection.title, ', ') AS collections
FROM collection_product
JOIN collection
ON collection_product.collection_id = collection.collection_id
AND collection_product.source_relation = collection.source_relation
GROUP BY
1,
2
), tags_aggregated AS (
SELECT
product_id,
source_relation,
LISTAGG(value, ', ') AS tags
FROM product_tag
GROUP BY
1,
2
), variants_aggregated AS (
SELECT
product_id,
source_relation,
COUNT(variant_id) AS count_variants
FROM product_variant
GROUP BY
1,
2
), images_aggregated AS (
SELECT
product_id,
source_relation,
COUNT(*) AS count_images
FROM product_image
GROUP BY
1,
2
), joined AS (
SELECT
products.*,
collections_aggregated.collections,
tags_aggregated.tags,
variants_aggregated.count_variants,
COALESCE(images_aggregated.count_images, 0) > 0 AS has_product_image
FROM products
LEFT JOIN collections_aggregated
ON products.product_id = collections_aggregated.product_id
AND products.source_relation = collections_aggregated.source_relation
LEFT JOIN tags_aggregated
ON products.product_id = tags_aggregated.product_id
AND products.source_relation = tags_aggregated.source_relation
LEFT JOIN variants_aggregated
ON products.product_id = variants_aggregated.product_id
AND products.source_relation = variants_aggregated.source_relation
LEFT JOIN images_aggregated
ON products.product_id = images_aggregated.product_id
AND products.source_relation = images_aggregated.source_relation
)
SELECT
*
FROM joined
), __dbt__cte__int_shopify__product__order_line_aggregates AS (
WITH order_lines AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__order_lines
), orders AS (
SELECT
*
FROM TEST.PUBLIC_shopify.shopify__orders
), product_aggregated AS (
SELECT
order_lines.product_id,
order_lines.source_relation,
SUM(order_lines.quantity) AS quantity_sold, /* moved over from shopify__products */
SUM(order_lines.pre_tax_price) AS subtotal_sold,
SUM(order_lines.quantity_net_refunds) AS quantity_sold_net_refunds,
SUM(order_lines.subtotal_net_refunds) AS subtotal_sold_net_refunds,
MIN(orders.created_timestamp) AS first_order_timestamp,
MAX(orders.created_timestamp) AS most_recent_order_timestamp,
SUM(order_lines.total_discount) AS product_total_discount, /* new columns */
SUM(order_lines.order_line_tax) AS product_total_tax,
AVG(order_lines.quantity) AS avg_quantity_per_order_line,
AVG(order_lines.total_discount) AS product_avg_discount_per_order_line,
AVG(order_lines.order_line_tax) AS product_avg_tax_per_order_line
FROM order_lines
LEFT JOIN orders
ON order_lines.order_id = orders.order_id
AND order_lines.source_relation = orders.source_relation
GROUP BY
1,
2
)
SELECT
*
FROM product_aggregated
), products AS (
SELECT
*
FROM __dbt__cte__int_shopify__products_with_aggregates
), product_order_lines AS (
SELECT
*
FROM __dbt__cte__int_shopify__product__order_line_aggregates
), joined AS (
SELECT
products.*,
COALESCE(product_order_lines.quantity_sold, 0) AS total_quantity_sold,
COALESCE(product_order_lines.subtotal_sold, 0) AS subtotal_sold,
COALESCE(product_order_lines.quantity_sold_net_refunds, 0) AS quantity_sold_net_refunds,
COALESCE(product_order_lines.subtotal_sold_net_refunds, 0) AS subtotal_sold_net_refunds,
product_order_lines.first_order_timestamp,
product_order_lines.most_recent_order_timestamp,
product_order_lines.avg_quantity_per_order_line AS avg_quantity_per_order_line,
COALESCE(product_order_lines.product_total_discount, 0) AS product_total_discount,
product_order_lines.product_avg_discount_per_order_line AS product_avg_discount_per_order_line,
COALESCE(product_order_lines.product_total_tax, 0) AS product_total_tax,
product_order_lines.product_avg_tax_per_order_line AS product_avg_tax_per_order_line
FROM products
LEFT JOIN product_order_lines
ON products.product_id = product_order_lines.product_id
AND products.source_relation = product_order_lines.source_relation
)
SELECT
*
FROM joined
Name | Type | Comment |
---|---|---|
is_deleted | None | Whether the record has been deleted in the source system. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
created_timestamp | None | The date and time when the product was created. |
handle | None | A unique human-friendly string for the product. Automatically generated from the product's title. |
product_id | None | An unsigned 64-bit integer that's used as a unique identifier for the product. Each id is unique across the Shopify system. No two products will have the same id, even if they're from different shops. |
product_type | None | A categorization for the product used for filtering and searching products. |
published_timestamp | None | The date and time (ISO 8601 format) when the product was published. Can be set to null to unpublish the product from the Online Store channel. |
published_scope | None | Whether the product is published to the Point of Sale channel. |
title | None | The name of the product. |
updated_timestamp | None | The date and time when the product was last modified. |
vendor | None | The name of the product's vendor. |
total_quantity_sold | None | Quantity of the product sold. |
subtotal_sold | None | Total amount (in shop currency) of the product sold. |
quantity_sold_net_refunds | None | Quantity of the product sold, excluding refunds. |
subtotal_sold_net_refunds | None | Total amount (in shop currency) of the product sold, excluding refunds. |
first_order_timestamp | None | The timestamp the product was first ordered. |
most_recent_order_timestamp | None | The timestamp the product was most recently ordered. |
source_relation | None | The schema or database this record came from, if you are unioning multiple connectors. Null if not. |
avg_quantity_per_order_line | None | Average quantity per order line with this product. |
product_total_discount | None | Total discounts associated with the product. |
product_avg_discount_per_order_line | None | Average discount per order line with this product. |
product_total_tax | None | Total taxes associated with the product. |
product_avg_tax_per_order_line | None | Average taxes per order line with this product. |
count_variants | None | Count of product variants. |
has_product_image | None | If the product has an image. |
status | None | The status of the product. Valid values: - active: The product is ready to sell and is available to customers on the online store, sales channels, and apps. By default, existing products are set to active. - archived: The product is no longer being sold and isn't available to customers on sales channels and apps. - draft: The product isn't ready to sell and is unavailable to customers on sales channels and apps. By default, duplicated and unarchived products are set to draft. |
collections | None | Comma-separated list of collections associated with the product. |
tags | None | Comma-separated list of tags associated with the product. |
This SQL query integrates data from three Shopify-related tables: transactions, tender transactions, and parent transactions. It creates a unique identifier for each transaction, joins the tables based on transaction IDs and source relations, and calculates exchange rates and currency-adjusted amounts from JSON data in the receipt field. The query enhances the transaction data with payment methods, parent transaction details, and currency exchange information.
IntegrationFeaturizationCleaningWITH transactions AS (
SELECT
*,
MD5(
CAST(COALESCE(CAST(source_relation AS TEXT), '_dbt_utils_surrogate_key_null_') || '-' || COALESCE(CAST(transaction_id AS TEXT), '_dbt_utils_surrogate_key_null_') AS TEXT)
) AS transactions_unique_id
FROM TEST.PUBLIC_stg_shopify.stg_shopify__transaction
), tender_transactions AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__tender_transaction
), joined AS (
SELECT
transactions.*,
tender_transactions.payment_method,
parent_transactions.created_timestamp AS parent_created_timestamp,
parent_transactions.kind AS parent_kind,
parent_transactions.amount AS parent_amount,
parent_transactions.status AS parent_status
FROM transactions
LEFT JOIN tender_transactions
ON transactions.transaction_id = tender_transactions.transaction_id
AND transactions.source_relation = tender_transactions.source_relation
LEFT JOIN transactions AS parent_transactions
ON transactions.parent_id = parent_transactions.transaction_id
AND transactions.source_relation = parent_transactions.source_relation
), exchange_rate AS (
SELECT
*,
COALESCE(
CAST(NULLIF(
PARSE_JSON(receipt)['charges']['data'][0]['balance_transaction']['exchange_rate'],
''
) AS DECIMAL(28, 6)),
1
) AS exchange_rate,
COALESCE(
CAST(NULLIF(
PARSE_JSON(receipt)['charges']['data'][0]['balance_transaction']['exchange_rate'],
''
) AS DECIMAL(28, 6)),
1
) * amount AS currency_exchange_calculated_amount
FROM joined
)
SELECT
*
FROM exchange_rate
Name | Type | Comment |
---|---|---|
transactions_unique_id | None | Unique key representing a transaction. Hashed on 'transaction_id' and 'source_relation'. |
transaction_id | None | The ID for the transaction. |
order_id | None | The ID for the order that the transaction is associated with. |
refund_id | None | The ID associated with a refund in the refund table. |
amount | None | The amount of money included in the transaction in shop currency. |
authorization | None | The authorization code associated with the transaction. |
created_timestamp | None | The date and time when the transaction was created. |
processed_timestamp | None | The date and time when a transaction was processed. |
device_id | None | The ID for the device. |
gateway | None | The name of the gateway the transaction was issued through. |
source_name | None | The origin of the transaction. |
message | None | A string generated by the payment provider with additional information about why the transaction succeeded or failed. |
currency | None | The three-letter code (ISO 4217 format) for the currency used for the payment. |
location_id | None | The ID of the physical location where the transaction was processed. |
parent_id | None | The ID of an associated transaction. |
payment_avs_result_code | None | The response code from the address verification system. |
payment_credit_card_bin | None | The issuer identification number (IIN), formerly known as bank identification number (BIN) of the customer's credit card. |
payment_cvv_result_code | None | The response code from the credit card company indicating whether the customer entered the card security code, or card verification value, correctly. |
payment_credit_card_number | None | The customer's credit card number, with most of the leading digits redacted. |
payment_credit_card_company | None | The name of the company that issued the customer's credit card. |
kind | None | The transaction's type. |
receipt | None | A transaction receipt attached to the transaction by the gateway. |
currency_exchange_id | None | The ID of the adjustment. |
currency_exchange_adjustment | None | The difference between the amounts on the associated transaction and the parent transaction. |
currency_exchange_original_amount | None | The amount of the parent transaction in the shop currency. |
currency_exchange_final_amount | None | The amount of the associated transaction in the shop currency. |
currency_exchange_currency | None | The shop currency. |
error_code | None | A standardized error code, independent of the payment provider. |
status | None | The status of the transaction. |
user_id | None | The ID for the user who was logged into the Shopify POS device when the order was processed, if applicable. |
_fivetran_synced | None | Timestamp of the date the record was synced by Fivetran. |
exchange_rate | None | The exchange rate between the home currency and the currency of sale at the time of the transaction. |
currency_exchange_calculated_amount | None | The total amount of the transaction with the currency exchange rate applied. |
source_relation | None | The schema or database this record came from, if you are unioning multiple connectors. Null if not. |
payment_method | None | Method of payment. |
parent_created_timestamp | None | Created on timestamp of the parent transaction. |
parent_kind | None | Kind of the parent transaction. |
parent_amount | None | Amount of the parent transaction. |
parent_status | None | Status of the parent transaction. |
authorization_expires_at | None | Timestamp when the authorization expires. |
authorization_code | None | The authorization code associated with the transaction. |
This SQL query performs a series of data transformations on the 'stg_shopify__abandoned_checkout_tmp' table. It starts by creating a base CTE that selects all columns from the source table. Then, it defines a 'fields' CTE that casts all columns to specific data types, including several NULL casts. Finally, in the 'final' CTE, it renames some columns, converts timestamps to UTC, and applies some semantic changes (like renaming '_fivetran_deleted' to 'is_deleted'). The query doesn't filter or aggregate data, but primarily focuses on data type casting and column renaming/reformatting.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout_tmp
), fields AS (
SELECT
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TEXT) AS abandoned_checkout_url,
CAST(NULL AS TEXT) AS billing_address_address_1,
CAST(NULL AS TEXT) AS billing_address_address_2,
CAST(NULL AS TEXT) AS billing_address_city,
CAST(NULL AS TEXT) AS billing_address_company,
CAST(NULL AS TEXT) AS billing_address_country,
CAST(NULL AS TEXT) AS billing_address_country_code,
CAST(NULL AS TEXT) AS billing_address_first_name,
CAST(NULL AS TEXT) AS billing_address_last_name,
CAST(NULL AS TEXT) AS billing_address_latitude,
CAST(NULL AS TEXT) AS billing_address_longitude,
CAST(NULL AS TEXT) AS billing_address_name,
CAST(NULL AS TEXT) AS billing_address_phone,
CAST(NULL AS TEXT) AS billing_address_province,
CAST(NULL AS TEXT) AS billing_address_province_code,
CAST(NULL AS TEXT) AS billing_address_zip,
CAST(NULL AS BOOLEAN) AS buyer_accepts_marketing,
CAST(NULL AS TEXT) AS cart_token,
CAST(NULL AS TIMESTAMP) AS closed_at,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TEXT) AS currency,
CAST(NULL AS INT) AS customer_id,
CAST(NULL AS TEXT) AS customer_locale,
CAST(NULL AS INT) AS device_id,
CAST(NULL AS TEXT) AS email,
CAST(NULL AS TEXT) AS gateway,
CAST(NULL AS INT) AS id,
CAST(NULL AS TEXT) AS landing_site_base_url,
CAST(NULL AS INT) AS location_id,
CAST(NULL AS TEXT) AS name,
CAST(NULL AS TEXT) AS note,
CAST(NULL AS TEXT) AS phone,
CAST(NULL AS TEXT) AS presentment_currency,
CAST(NULL AS TEXT) AS referring_site,
CAST(NULL AS TEXT) AS shipping_address_address_1,
CAST(NULL AS TEXT) AS shipping_address_address_2,
CAST(NULL AS TEXT) AS shipping_address_city,
CAST(NULL AS TEXT) AS shipping_address_company,
CAST(NULL AS TEXT) AS shipping_address_country,
CAST(NULL AS TEXT) AS shipping_address_country_code,
CAST(NULL AS TEXT) AS shipping_address_first_name,
CAST(NULL AS TEXT) AS shipping_address_last_name,
CAST(NULL AS TEXT) AS shipping_address_latitude,
CAST(NULL AS TEXT) AS shipping_address_longitude,
CAST(NULL AS TEXT) AS shipping_address_name,
CAST(NULL AS TEXT) AS shipping_address_phone,
CAST(NULL AS TEXT) AS shipping_address_province,
CAST(NULL AS TEXT) AS shipping_address_province_code,
CAST(NULL AS TEXT) AS shipping_address_zip,
CAST(NULL AS TEXT) AS source_name,
CAST(NULL AS FLOAT) AS subtotal_price,
CAST(NULL AS BOOLEAN) AS taxes_included,
CAST(NULL AS TEXT) AS token,
CAST(NULL AS FLOAT) AS total_discounts,
CAST(NULL AS TEXT) AS total_duties,
CAST(NULL AS FLOAT) AS total_line_items_price,
CAST(NULL AS FLOAT) AS total_price,
CAST(NULL AS FLOAT) AS total_tax,
CAST(NULL AS INT) AS total_weight,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS INT) AS user_id,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
_fivetran_deleted AS is_deleted,
abandoned_checkout_url,
billing_address_address_1,
billing_address_address_2,
billing_address_city,
billing_address_company,
billing_address_country,
billing_address_country_code,
billing_address_first_name,
billing_address_last_name,
billing_address_latitude,
billing_address_longitude,
billing_address_name,
billing_address_phone,
billing_address_province,
billing_address_province_code,
billing_address_zip,
buyer_accepts_marketing AS has_buyer_accepted_marketing,
cart_token,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(closed_at AS TIMESTAMP) AS TIMESTAMP)) AS closed_at,
currency AS shop_currency,
customer_id,
customer_locale,
device_id,
email,
gateway,
id AS checkout_id,
landing_site_base_url,
location_id,
name,
note,
phone,
presentment_currency,
referring_site,
shipping_address_address_1,
shipping_address_address_2,
shipping_address_city,
shipping_address_company,
shipping_address_country,
shipping_address_country_code,
shipping_address_first_name,
shipping_address_last_name,
shipping_address_latitude,
shipping_address_longitude,
shipping_address_name,
shipping_address_phone,
shipping_address_province,
shipping_address_province_code,
shipping_address_zip,
source_name,
subtotal_price,
taxes_included AS has_taxes_included,
token,
total_discounts,
total_duties,
total_line_items_price,
total_price,
total_tax,
total_weight,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
user_id,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
abandoned_checkout_url | None | The recovery URL that's sent to a customer so they can recover their checkout. |
billing_address_address_1 | None | The street address of the billing address. |
billing_address_address_2 | None | An optional additional field for the street address of the billing address. |
billing_address_city | None | The city of the billing address. |
billing_address_company | None | The company of the person associated with the billing address. |
billing_address_country | None | The name of the country of the billing address. |
billing_address_country_code | None | The two-letter code (ISO 3166-1 alpha-2 format) for the country of the billing address. |
billing_address_first_name | None | The first name of the person associated with the payment method. |
billing_address_last_name | None | The last name of the person associated with the payment method. |
billing_address_latitude | None | The latitude of the billing address. |
billing_address_longitude | None | The longitude of the billing address. |
billing_address_name | None | The full name of the person associated with the payment method. |
billing_address_phone | None | The phone number at the billing address. |
billing_address_province | None | The name of the state or province of the billing address. |
billing_address_province_code | None | The two-letter abbreviation of the state or province of the billing address. |
billing_address_zip | None | The zip or postal code of the billing address. |
has_buyer_accepted_marketing | None | Whether the customer would like to receive email updates from the shop. This is set by the 'I want to receive occasional emails about new products, promotions and other news' checkbox during checkout. |
cart_token | None | The ID for the cart that's attached to the checkout. |
closed_at | None | The date and time (ISO 8601 format) when the checkout was closed. If the checkout was not closed, then this value is null. |
created_at | None | The date and time (ISO 8601 format) when the checkout was created. |
shop_currency | None | The three-letter code (ISO 4217 format) of the shop's default currency at the time of checkout. For the currency that the customer used at checkout, see `presentment_currency`. |
customer_id | None | ID of the customer with the abandoned checkout. |
customer_locale | None | The two or three-letter language code, optionally followed by a region modifier. Example values - en, en-CA. |
device_id | None | The ID of the Shopify POS device that created the checkout. |
None | The customer's email address. | |
gateway | None | The payment gateway used by the checkout. |
checkout_id | None | The ID for the checkout. |
landing_site_base_url | None | The URL for the page where the customer entered the shop. |
location_id | None | The ID of the physical location where the checkout was processed. |
name | None | Checkout order number. |
note | None | The text of an optional note that a shop owner can attach to the order. |
phone | None | The customer's phone number for receiving SMS notifications. |
presentment_currency | None | The three-letter code (ISO 4217 format) of the currency that the customer used at checkout. For the shop's default currency, see `currency`. |
referring_site | None | The website that referred the customer to the shop. |
shipping_address_address_1 | None | The street address of the shipping address. |
shipping_address_address_2 | None | An optional additional field for the street address of the shipping address. |
shipping_address_city | None | The city of the shipping address. |
shipping_address_company | None | The company of the person associated with the shipping address. |
shipping_address_country | None | The name of the country of the shipping address. |
shipping_address_country_code | None | The two-letter code (ISO 3166-1 alpha-2 format) for the country of the shipping address. |
shipping_address_first_name | None | The first name of the person associated with the shipping address. |
shipping_address_last_name | None | The last name of the person associated with the shipping address. |
shipping_address_latitude | None | The latitude of the shipping address. |
shipping_address_longitude | None | The longitude of the shipping address. |
shipping_address_name | None | The full name of the person associated with the shipping address. |
shipping_address_phone | None | The phone number at the shipping address. |
shipping_address_province | None | The name of the state or province of the shipping address. |
shipping_address_province_code | None | The two-letter abbreviation of the state or province of the shipping address. |
shipping_address_zip | None | The zip or postal code of the shipping address. |
source_name | None | Where the checkout originated. Valid values include `web`, `pos`, `iphone`, `android`. |
subtotal_price | None | The price of the checkout in _presentment_ (customer) currency before shipping and taxes. |
has_taxes_included | None | Boolean representing whether taxes are included in the price. |
token | None | A unique ID for a checkout. |
total_discounts | None | The total amount of discounts to be applied in presentment currency. |
total_duties | None | The total duties of the checkout in presentment currency. |
total_line_items_price | None | The sum of the prices of all line items in the checkout in _presentment_ (customer) currency. |
total_price | None | The sum of line item prices, all discounts, shipping costs, and taxes for the checkout in _presentment_ (customer) currency. |
total_tax | None | The sum of all the taxes applied to the checkout in _presentment_ (customer) currency. |
total_weight | None | The sum of all the weights in grams of the line items in the checkout. |
updated_at | None | The date and time (ISO 8601 format) when the checkout was last modified. |
user_id | None | The ID of the user who created the checkout. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query performs several operations on the 'stg_shopify__abandoned_checkout_discount_code' data. It starts by casting fields to specific data types, then applies data cleaning and standardization. The query converts timestamps to UTC, uppercases the 'code' field, and creates a unique index for each row. Finally, it deduplicates the data by keeping only the row with index 1 for each combination of checkout_id, code, and source_relation.
CleaningDeduplicationFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout_discount_code_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS FLOAT) AS amount,
CAST(NULL AS INT) AS checkout_id,
CAST(NULL AS TEXT) AS code,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS INT) AS discount_id,
CAST(NULL AS INT) AS index,
CAST(NULL AS TEXT) AS type,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
checkout_id,
UPPER(code) AS code,
discount_id,
amount,
type,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation,
CASE
WHEN checkout_id IS NULL AND code IS NULL AND index IS NULL
THEN ROW_NUMBER() OVER (PARTITION BY source_relation ORDER BY source_relation)
ELSE ROW_NUMBER() OVER (PARTITION BY checkout_id, UPPER(code), source_relation ORDER BY index DESC)
END AS index
FROM fields
)
SELECT
*
FROM final
WHERE
index = 1
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
amount | None | The amount of the discount in presentment currency. |
checkout_id | None | ID of the checkout. |
code | None | The discount code. |
created_at | None | When the checkout discount application was created. |
discount_id | None | ID of the discount. Deprecated, use `code` instead. |
type | None | The type of discount. Valid values - percentage, shipping, fixed_amount. (default - fixed_amount) |
updated_at | None | When the checkout's discount was last updated |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The LIMIT 0 clause ensures no rows are returned. This query appears to be a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify abandoned checkout discount codes.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs a data transformation on the 'stg_shopify__abandoned_checkout_shipping_line_tmp' table. It casts various columns to specific data types, renames some columns, and adds a new column 'source_relation'. The query also converts the '_fivetran_synced' timestamp to UTC timezone. The purpose seems to be standardizing and cleaning the data for further use.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__abandoned_checkout_shipping_line_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TEXT) AS carrier_identifier,
CAST(NULL AS INT) AS checkout_id,
CAST(NULL AS TEXT) AS code,
CAST(NULL AS TEXT) AS delivery_category,
CAST(NULL AS TEXT) AS delivery_expectation_range,
CAST(NULL AS INT) AS delivery_expectation_range_max,
CAST(NULL AS INT) AS delivery_expectation_range_min,
CAST(NULL AS TEXT) AS delivery_expectation_type,
CAST(NULL AS FLOAT) AS discounted_price,
CAST(NULL AS TEXT) AS id,
CAST(NULL AS INT) AS index,
CAST(NULL AS TEXT) AS phone,
CAST(NULL AS FLOAT) AS price,
CAST(NULL AS TEXT) AS requested_fulfillment_service_id,
CAST(NULL AS TEXT) AS source,
CAST(NULL AS TEXT) AS title,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS abandoned_checkout_shipping_line_id,
checkout_id,
index,
carrier_identifier,
code AS shipping_code,
delivery_category,
delivery_expectation_range,
delivery_expectation_range_max,
delivery_expectation_range_min,
delivery_expectation_type,
discounted_price,
phone,
price,
requested_fulfillment_service_id,
source,
title,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
carrier_identifier | None | A reference to the carrier service that provided the rate. Present when the rate was computed by a third-party carrier service. |
checkout_id | None | ID of the checkout that was abandoned. |
shipping_code | None | A reference to the shipping method. |
delivery_category | None | The general classification of the delivery method. |
delivery_expectation_range | None | Expected delivery date range. |
delivery_expectation_range_max | None | Latest expected delivery date. |
delivery_expectation_range_min | None | Earliest possible expected delivery date. |
delivery_expectation_type | None | Type of expected delivery. |
discounted_price | None | The pre-tax shipping price with discounts applied in _presentment_ (customer) currency. |
abandoned_checkout_shipping_line_id | None | Unique ID of the abandoned checkout shipping line. |
index | None | Index of the line amongst shipping lines for this checkout. |
phone | None | The phone number at the shipping address. |
price | None | The price of the shipping method in presentment currency. |
requested_fulfillment_service_id | None | The fulfillment service requested for the shipping method. Present if the shipping method requires processing by a third party fulfillment service. |
source | None | The channel where the checkout originated. Example value - shopify. |
title | None | The title of the shipping method. Example value - International Shipping. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The query is limited to 0 rows, effectively creating a template or schema for the 'stg_shopify__abandoned_checkout_shipping_line_tmp' model without actually returning any data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, initialized to NULL. It's likely used as a placeholder or template for further development or testing purposes.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs data cleaning and transformation on a Shopify collection dataset. It casts various fields to appropriate data types, renames some columns, and applies specific transformations such as converting the 'disjunctive' boolean field into a text field with 'disjunctive' or 'conjunctive' values. It also converts timestamp fields to UTC timezone. The query doesn't filter, deduplicate, or aggregate data, but focuses on standardizing and cleaning the existing data.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__collection_tmp
), fields AS (
SELECT
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS BOOLEAN) AS disjunctive,
CAST(NULL AS TEXT) AS handle,
CAST(NULL AS INT) AS id,
CAST(NULL AS TIMESTAMP) AS published_at,
CAST(NULL AS TEXT) AS published_scope,
CAST(NULL AS TEXT) AS rules,
CAST(NULL AS TEXT) AS sort_order,
CAST(NULL AS TEXT) AS title,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS collection_id,
_fivetran_deleted AS is_deleted,
CASE
WHEN disjunctive IS NULL
THEN NULL
WHEN disjunctive
THEN 'disjunctive'
ELSE 'conjunctive'
END AS rule_logic,
handle,
published_scope,
rules,
sort_order,
title,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(published_at AS TIMESTAMP) AS TIMESTAMP)) AS published_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
rule_logic | None | Whether the `rules` are disjunctive (logical `OR`) or conjunctive (logical `AND`) |
handle | None | A unique, human-readable string for the collection automatically generated from its title. This is used in themes by the Liquid templating language to refer to the collection. |
collection_id | None | The ID for the collection. |
published_at | None | The time and date (ISO 8601 format) when the collection was made visible. Returns null for a hidden collection. |
published_scope | None | Whether the collection is published to the Point of Sale channel. Valid values `web` (the collection is published to the Online Store channel but not published to the Point of Sale channel) and `global` (the collection is published to both the Online Store channel and the Point of Sale channel). |
rules | None | An array of rules that define what products go into the smart collection. Each rule (`column` -- `relation` --> `condition`) has these properties: - `column`: the property of a product being used to populate the smart collection. Ex: 'tag', 'type', 'vendor', 'variant_price', etc. - `relation`: The comparative relationship between the column choice, and the condition ('equals', 'contains', 'greater_than', etc.) - condition: Select products for a smart collection using a condition. Values are either strings or numbers, depending on the relation value. See the [Shopify docs](https://shopify.dev/api/admin-rest/2022-10/resources/smartcollection#resource-object) for more. |
sort_order | None | The order of the products in the collection. Valid values incline - `alpha-asc`: The products are sorted alphabetically from A to Z. - `alpha-des`: The products are sorted alphabetically from Z to A. - `best-selling`: The products are sorted by number of sales. - `created`: The products are sorted by the date they were created, from oldest to newest. - `created-desc`: The products are sorted by the date they were created, from newest to oldest. - `manual`: The products are manually sorted by the shop owner. - `price-asc`: The products are sorted by price from lowest to highest. - `price-desc`: The products are sorted by price from highest to lowest. |
title | None | The name of the collection |
updated_at | None | The date and time (ISO 8601 format) when the collection was last modified. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates a staging table for Shopify collection product data. It starts with a base CTE that selects all columns from a temporary table. Then, it defines a fields CTE that casts specific columns to desired data types, including setting some to NULL or empty string. Finally, it creates a final CTE that selects and formats these fields, particularly converting the _fivetran_synced timestamp to UTC. The query then selects all columns from this final CTE.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__collection_product_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS collection_id,
CAST(NULL AS INT) AS product_id,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
collection_id,
product_id,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
collection_id | None | ID referencing the `collection` the product belongs to. |
product_id | None | ID referencing the `product`. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The query limits the result to 0 rows, effectively creating a template or structure for the table without any actual data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of TEXT data type. It's likely used as a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify collection data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query transforms and cleans data from a Shopify customer staging table. It casts various fields to appropriate data types, standardizes email and state fields to lowercase, handles legacy and new marketing consent fields, converts timestamps to UTC, and renames some columns for clarity.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS BOOLEAN) AS accepts_marketing,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS DECIMAL(28, 6)) AS default_address_id,
CAST(NULL AS TEXT) AS email,
CAST(NULL AS TEXT) AS first_name,
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS TEXT) AS last_name,
CAST(NULL AS DECIMAL(28, 6)) AS orders_count,
CAST(NULL AS TEXT) AS phone,
CAST(NULL AS TEXT) AS state,
CAST(NULL AS BOOLEAN) AS tax_exempt,
CAST(NULL AS FLOAT) AS total_spent,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS BOOLEAN) AS verified_email,
CAST(NULL AS TIMESTAMP) AS email_marketing_consent_consent_updated_at,
CAST(NULL AS TEXT) AS email_marketing_consent_opt_in_level,
CAST(NULL AS TEXT) AS email_marketing_consent_state,
CAST(NULL AS TEXT) AS note,
CAST(NULL AS TIMESTAMP) AS accepts_marketing_updated_at,
CAST(NULL AS TEXT) AS marketing_opt_in_level,
CAST(NULL AS TEXT) AS currency,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS customer_id,
LOWER(email) AS email,
first_name,
last_name,
orders_count,
default_address_id,
phone,
LOWER(state) AS account_state,
tax_exempt AS is_tax_exempt,
total_spent,
verified_email AS is_verified_email,
note,
currency,
CASE
WHEN email_marketing_consent_state IS NULL
THEN CASE
WHEN accepts_marketing IS NULL
THEN NULL
WHEN accepts_marketing
THEN 'subscribed (legacy)'
ELSE 'not_subscribed (legacy)'
END
ELSE LOWER(email_marketing_consent_state)
END AS marketing_consent_state,
LOWER(COALESCE(email_marketing_consent_opt_in_level, marketing_opt_in_level)) AS marketing_opt_in_level,
CONVERT_TIMEZONE(
'UTC',
'UTC',
CAST(CAST(COALESCE(accepts_marketing_updated_at, email_marketing_consent_consent_updated_at) AS TIMESTAMP) AS TIMESTAMP)
) AS marketing_consent_updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
marketing_consent_state | None | Field indicating if the customer has consented to receive marketing material via email. Coalescing of the new `email_marketing_consent_state` field and the deprecated `accepts_marketing` field. Records with the old field will be marked with '(legacy)'. |
marketing_opt_in_level | None | The marketing subscription opt-in level, as described in the M3AAWG Sender Best Common Practices, that the customer gave when they consented to receive marketing material by email. |
marketing_consent_updated_at | None | The date and time when the customer consented to receive marketing material by email. If no date is provided, then the date and time when the consent information was sent is used. |
created_timestamp | None | The date and time when the customer was created. |
default_address_id | None | The default address for the customer. |
None | The unique email address of the customer. Attempting to assign the same email address to multiple customers returns an error. | |
first_name | None | The customer's first name. |
customer_id | None | A unique identifier for the customer. |
last_name | None | The customer's last name. |
orders_count | None | The number of orders associated with this customer. |
phone | None | The unique phone number (E.164 format) for this customer. Attempting to assign the same phone number to multiple customers returns an error. |
account_state | None | The state of the customer's account with a shop. |
is_tax_exempt | None | Whether the customer is exempt from paying taxes on their order. If true, then taxes won't be applied to an order at checkout. If false, then taxes will be applied at checkout. |
total_spent | None | The total amount of money that the customer has spent across their order history. |
updated_timestamp | None | The date and time when the customer information was last updated. |
is_verified_email | None | Whether the customer has verified their email address. |
currency | None | The three-letter code (ISO 4217 format) for the currency that the customer used when they paid for their last order. Defaults to the shop currency. Returns the shop currency for test orders. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
note | None | A note about the customer. |
This SQL query creates a staging table for Shopify customer tags. It starts with a base table, then defines the structure of the fields with null or empty values. Finally, it selects these fields, converting the '_fivetran_synced' timestamp to UTC. The query appears to be setting up a standardized structure for customer tag data, possibly as part of an ETL process.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__customer_tag_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS index,
CAST(NULL AS INT) AS customer_id,
CAST(NULL AS TEXT) AS value,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
customer_id,
index,
value,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
index | None | Index (starting from 1) representing when the tag was placed on the customer. |
customer_id | None | ID of the customer being tagged. |
value | None | Value of the tag. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify customer tag data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The LIMIT 0 clause ensures no rows are returned. This type of query is often used as a placeholder or to create a template for a table structure without populating it with data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates a staging table for Shopify discount codes. It starts with a base table, then defines a structure with NULL or empty values for all fields. The final step formats and transforms these fields, including converting timestamps to UTC, uppercasing the discount code, and renaming the ID field. The query is designed to prepare a consistent structure for discount code data, even when no actual data exists yet.
CleaningOther/* this model will be all NULL until you create a discount code in Shopify */
WITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__discount_code_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TEXT) AS code,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS INT) AS id,
CAST(NULL AS INT) AS price_rule_id,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS FLOAT) AS usage_count,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS discount_code_id,
UPPER(code) AS code,
price_rule_id,
usage_count,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
code | None | The case-insensitive discount code that customers use at checkout. Shopify recommends this map onto the associated `price_rule.title`. |
created_at | None | The date and time (ISO 8601 format) when the discount code was created. |
discount_code_id | None | The ID for the discount code. |
price_rule_id | None | The ID for the price rule that this discount code belongs to. |
updated_at | None | The date and time (ISO 8601 format) when the discount code was updated. |
usage_count | None | The number of times that the discount code has been redeemed. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty dataset with a single column named '_dbt_source_relation' of type TEXT. The query is designed to return no rows (LIMIT 0) and is intended as a placeholder until actual discount code data is created in Shopify.
Other/* this model will be all NULL until you create a discount code in Shopify */
SELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query stages data from a Shopify fulfillment source table. It casts columns to specific data types, renames some columns, converts timestamps to UTC, and standardizes the status field to lowercase. The query also adds a source_relation column, though its value is set to an empty string in this case.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__fulfillment_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS INT) AS id,
CAST(NULL AS INT) AS location_id,
CAST(NULL AS TEXT) AS name,
CAST(NULL AS INT) AS order_id,
CAST(NULL AS TEXT) AS service,
CAST(NULL AS TEXT) AS shipment_status,
CAST(NULL AS TEXT) AS status,
CAST(NULL AS TEXT) AS tracking_company,
CAST(NULL AS TEXT) AS tracking_number,
CAST(NULL AS TEXT) AS tracking_numbers,
CAST(NULL AS TEXT) AS tracking_urls,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS fulfillment_id,
location_id,
order_id,
name,
service,
shipment_status,
LOWER(status) AS status,
tracking_company,
tracking_number,
tracking_numbers,
tracking_urls,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
created_at | None | The date and time when the fulfillment was created. The API returns this value in ISO 8601 format. |
fulfillment_id | None | The ID for the fulfillment. |
location_id | None | The unique identifier of the location that the fulfillment was processed at. |
name | None | The uniquely identifying fulfillment name, consisting of two parts separated by a .. The first part represents the order name and the second part represents the fulfillment number. The fulfillment number automatically increments depending on how many fulfillments are in an order (e.g. #1001.1, #1001.2). |
order_id | None | The unique numeric identifier for the order. |
service | None | The fulfillment service associated with the fulfillment. |
shipment_status | None | The current shipment status of the fulfillment. Valid values include: - label_printed: A label for the shipment was purchased and printed. - label_purchased: A label for the shipment was purchased, but not printed. - attempted_delivery: Delivery of the shipment was attempted, but unable to be completed. - ready_for_pickup: The shipment is ready for pickup at a shipping depot. - confirmed: The carrier is aware of the shipment, but hasn't received it yet. - in_transit: The shipment is being transported between shipping facilities on the way to its destination. - out_for_delivery: The shipment is being delivered to its final destination. - delivered: The shipment was successfully delivered. - failure: Something went wrong when pulling tracking information for the shipment, such as the tracking number was invalid or the shipment was canceled. |
status | None | The status of the fulfillment. Valid values include: - pending: Shopify has created the fulfillment and is waiting for the third-party fulfillment service to transition it to 'open' or 'success'. - open: The fulfillment has been acknowledged by the service and is in processing. - success: The fulfillment was successful. - cancelled: The fulfillment was cancelled. - error: There was an error with the fulfillment request. - failure: The fulfillment request failed. |
tracking_company | None | The name of the tracking company. |
tracking_number | None | Primary tracking number for the order. |
tracking_numbers | None | A list of tracking numbers, provided by the shipping company. |
tracking_urls | None | The URLs of tracking pages for the fulfillment. |
updated_at | None | The date and time (ISO 8601 format) when the fulfillment was last modified. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The LIMIT 0 clause ensures no rows are returned. This query is likely used as a placeholder or template for generating a schema without actual data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs data transformation and cleaning on the Shopify inventory item data. It starts by selecting all columns from a temporary table, then casts them to specific data types. The query renames some columns, converts timestamps to UTC, and restructures the data for further use. It also adds a source_relation column, though it's set to an empty string in this case.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__inventory_item_tmp
), fields AS (
SELECT
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS FLOAT) AS cost,
CAST(NULL AS TEXT) AS country_code_of_origin,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS INT) AS id,
CAST(NULL AS TEXT) AS province_code_of_origin,
CAST(NULL AS BOOLEAN) AS requires_shipping,
CAST(NULL AS TEXT) AS sku,
CAST(NULL AS BOOLEAN) AS tracked,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS inventory_item_id,
sku,
_fivetran_deleted AS is_deleted, /* won't filter out for now */
cost,
country_code_of_origin,
province_code_of_origin,
requires_shipping AS is_shipping_required,
tracked AS is_inventory_quantity_tracked,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
cost | None | The unit cost of the inventory item. The shop's default currency is used. |
country_code_of_origin | None | The country code (ISO 3166-1 alpha-2) of where the item came from. |
created_at | None | The date and time (ISO 8601 format) when the inventory item was created. |
inventory_item_id | None | The ID of the inventory item. |
province_code_of_origin | None | The province code (ISO 3166-2 alpha-2) of where the item came from. The province code is only used if the shipping provider for the inventory item is Canada Post. |
is_shipping_required | None | Boolean representing whether a customer needs to provide a shipping address when placing an order containing the inventory item. |
sku | None | The unique SKU (stock keeping unit) of the inventory item. |
is_inventory_quantity_tracked | None | Boolean representing whether inventory levels are tracked for the item. If true, then the inventory quantity changes are tracked by Shopify. |
updated_at | None | The date and time (ISO 8601 format) when the inventory item was last modified. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of TEXT data type. It's likely used as a placeholder or template for further development or testing purposes in a dbt (data build tool) project.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query stages data from a temporary Shopify inventory level table. It casts columns to specific data types, renames the 'available' column to 'available_quantity', and converts 'updated_at' and '_fivetran_synced' timestamps to UTC. The query prepares the data for further processing or analysis by standardizing the format and structure of the inventory level information.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__inventory_level_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS available,
CAST(NULL AS INT) AS inventory_item_id,
CAST(NULL AS INT) AS location_id,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
inventory_item_id,
location_id,
available AS available_quantity,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
available_quantity | None | The available quantity of an inventory item at the inventory level's associated location. Returns null if the inventory item is not tracked. |
inventory_item_id | None | The ID of the inventory item associated with the inventory level. |
location_id | None | The ID of the location that the inventory level belongs to. |
updated_at | None | The date and time (ISO 8601 format) when the inventory level was last modified. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of TEXT data type, which is set to NULL. The query is limited to 0 rows, effectively creating a template or placeholder for the structure of the 'stg_shopify__inventory_level_tmp' model without containing any actual data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs a staging operation for Shopify location data. It starts by selecting all columns from a temporary table, then casts all fields to specific data types (creating null values in the process). The query then renames and reorganizes some columns, converts timezone for timestamp fields, and adds a source_relation column. This appears to be a data cleaning and standardization step in an ETL process.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__location_tmp
), fields AS (
SELECT
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS BOOLEAN) AS active,
CAST(NULL AS TEXT) AS address_1,
CAST(NULL AS TEXT) AS address_2,
CAST(NULL AS TEXT) AS city,
CAST(NULL AS TEXT) AS country,
CAST(NULL AS TEXT) AS country_code,
CAST(NULL AS TEXT) AS country_name,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS INT) AS id,
CAST(NULL AS BOOLEAN) AS legacy,
CAST(NULL AS TEXT) AS localized_country_name,
CAST(NULL AS TEXT) AS localized_province_name,
CAST(NULL AS TEXT) AS name,
CAST(NULL AS TEXT) AS phone,
CAST(NULL AS TEXT) AS province,
CAST(NULL AS TEXT) AS province_code,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS TEXT) AS zip,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS location_id,
name,
_fivetran_deleted AS is_deleted,
active AS is_active,
address_1,
address_2,
city,
country,
country_code,
country_name,
legacy AS is_legacy,
localized_country_name,
localized_province_name,
phone,
province,
province_code,
zip,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
is_active | None | Boolean representing whether the location is active. If true, then the location can be used to sell products, stock inventory, and fulfill orders. |
address_1 | None | The location's street address. |
address_2 | None | The optional second line of the location's street address. |
city | None | The city the location is in. |
country | None | The country the location is in (two-letter code). |
country_code | None | The two-letter code (ISO 3166-1 alpha-2 format) corresponding to country the location is in. |
country_name | None | Full name of the location's country. |
created_at | None | The date and time (ISO 8601 format) when the location was created. |
location_id | None | The ID of the location. |
is_legacy | None | Boolean representing whether this is a fulfillment service location. If true, then the location is a fulfillment service location. If false, then the location was created by the merchant and isn't tied to a fulfillment service. |
localized_country_name | None | The localized name of the location's country. |
localized_province_name | None | The localized name of the location's region. Typically a province, state, or district. |
name | None | The name of the location. |
phone | None | The phone number of the location. This value can contain special characters, such as - or +. |
province | None | The province, state, or district of the location. |
province_code | None | The province, state, or district code (ISO 3166-2 alpha-2 format) of the location. |
updated_at | None | The date and time (ISO 8601 format) when the location was last updated. |
zip | None | The zip or postal code. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify location data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query processes data from a staging table for Shopify metafields. It performs several operations including casting data types, cleaning and standardizing column values, creating new features, and deduplicating records. The query also converts timestamps to UTC and creates a flag for the most recent record for each metafield.
CleaningFeaturizationDeduplicationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__metafield_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TEXT) AS description,
CAST(NULL AS INT) AS id,
CAST(NULL AS TEXT) AS key,
CAST(NULL AS TEXT) AS namespace,
CAST(NULL AS INT) AS owner_id,
CAST(NULL AS TEXT) AS owner_resource,
CAST(NULL AS TEXT) AS type,
CAST(NULL AS TEXT) AS value_type,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS TEXT) AS value,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS metafield_id,
description,
namespace,
key,
value,
LOWER(COALESCE(type, value_type)) AS value_type,
owner_id AS owner_resource_id,
LOWER(owner_resource) AS owner_resource,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
LOWER(namespace || '_' || key) AS metafield_reference,
CASE
WHEN id IS NULL AND updated_at IS NULL
THEN ROW_NUMBER() OVER (PARTITION BY source_relation ORDER BY source_relation) = 1
ELSE ROW_NUMBER() OVER (PARTITION BY id, source_relation ORDER BY updated_at DESC) = 1
END AS is_most_recent_record,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
created_at | None | The date and time (ISO 8601 format) when the metafield was created. |
description | None | A description of the information that the metafield contains. |
metafield_id | None | The unique ID of the metafield. |
key | None | The key of the metafield. Keys can be up to 64 characters long and can contain alphanumeric characters, hyphens, underscores, and periods. |
namespace | None | A container for a group of metafields. Grouping metafields within a namespace prevents your metafields from conflicting with other metafields with the same key name. Must have between 3-255 characters. |
owner_resource_id | None | The unique ID of the resource that the metafield is attached to. |
owner_resource | None | The type of resource (table) that the metafield is attached to. |
value_type | None | The type of data that the metafield stores in the `value` field. Refer to the [list](https://shopify.dev/apps/metafields/types) of supported types. Coalescing of type and value_type |
updated_at | None | The date and time (ISO 8601 format) when the metafield was last updated. |
value | None | The data to store in the metafield. The value is always stored as a string, regardless of the metafield's type. |
metafield_reference | None | Combination of the namespace and key columns. This field is key for metafield mapping in downstream models. |
is_most_recent_record | None | Boolean indicating the most up to date record identified by the most recent updated_at field. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The query limits the result to 0 rows, effectively returning no data. This type of query is often used as a placeholder or template in data modeling processes, particularly in dbt (data build tool) projects.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query stages and transforms data from a Shopify orders table. It casts various fields to appropriate data types, renames some columns, converts timestamps to UTC, and applies some basic transformations like lowercasing email addresses. The query also filters out test orders and deleted records.
CleaningFilteringFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_tmp
), fields AS (
SELECT
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS TIMESTAMP) AS processed_at,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS DECIMAL(28, 6)) AS user_id,
CAST(NULL AS FLOAT) AS total_discounts,
CAST(NULL AS TEXT) AS total_discounts_set,
CAST(NULL AS FLOAT) AS total_line_items_price,
CAST(NULL AS TEXT) AS total_line_items_price_set,
CAST(NULL AS FLOAT) AS total_price,
CAST(NULL AS TEXT) AS total_price_set,
CAST(NULL AS TEXT) AS total_tax_set,
CAST(NULL AS FLOAT) AS total_tax,
CAST(NULL AS TEXT) AS source_name,
CAST(NULL AS FLOAT) AS subtotal_price,
CAST(NULL AS BOOLEAN) AS taxes_included,
CAST(NULL AS DECIMAL(28, 6)) AS total_weight,
CAST(NULL AS FLOAT) AS total_tip_received,
CAST(NULL AS TEXT) AS landing_site_base_url,
CAST(NULL AS DECIMAL(28, 6)) AS location_id,
CAST(NULL AS TEXT) AS name,
CAST(NULL AS TEXT) AS note,
CAST(NULL AS DECIMAL(28, 6)) AS number,
CAST(NULL AS DECIMAL(28, 6)) AS order_number,
CAST(NULL AS TEXT) AS cancel_reason,
CAST(NULL AS TIMESTAMP) AS cancelled_at,
CAST(NULL AS TEXT) AS cart_token,
CAST(NULL AS TEXT) AS checkout_token,
CAST(NULL AS TIMESTAMP) AS closed_at,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TEXT) AS currency,
CAST(NULL AS DECIMAL(28, 6)) AS customer_id,
CAST(NULL AS TEXT) AS email,
CAST(NULL AS TEXT) AS financial_status,
CAST(NULL AS TEXT) AS fulfillment_status,
CAST(NULL AS TEXT) AS referring_site,
CAST(NULL AS TEXT) AS billing_address_address_1,
CAST(NULL AS TEXT) AS billing_address_address_2,
CAST(NULL AS TEXT) AS billing_address_city,
CAST(NULL AS TEXT) AS billing_address_company,
CAST(NULL AS TEXT) AS billing_address_country,
CAST(NULL AS TEXT) AS billing_address_country_code,
CAST(NULL AS TEXT) AS billing_address_first_name,
CAST(NULL AS TEXT) AS billing_address_last_name,
CAST(NULL AS TEXT) AS billing_address_latitude,
CAST(NULL AS TEXT) AS billing_address_longitude,
CAST(NULL AS TEXT) AS billing_address_name,
CAST(NULL AS TEXT) AS billing_address_phone,
CAST(NULL AS TEXT) AS billing_address_province,
CAST(NULL AS TEXT) AS billing_address_province_code,
CAST(NULL AS TEXT) AS billing_address_zip,
CAST(NULL AS TEXT) AS browser_ip,
CAST(NULL AS BOOLEAN) AS buyer_accepts_marketing,
CAST(NULL AS TEXT) AS total_shipping_price_set,
CAST(NULL AS TEXT) AS shipping_address_address_1,
CAST(NULL AS TEXT) AS shipping_address_address_2,
CAST(NULL AS TEXT) AS shipping_address_city,
CAST(NULL AS TEXT) AS shipping_address_company,
CAST(NULL AS TEXT) AS shipping_address_country,
CAST(NULL AS TEXT) AS shipping_address_country_code,
CAST(NULL AS TEXT) AS shipping_address_first_name,
CAST(NULL AS TEXT) AS shipping_address_last_name,
CAST(NULL AS TEXT) AS shipping_address_latitude,
CAST(NULL AS TEXT) AS shipping_address_longitude,
CAST(NULL AS TEXT) AS shipping_address_name,
CAST(NULL AS TEXT) AS shipping_address_phone,
CAST(NULL AS TEXT) AS shipping_address_province,
CAST(NULL AS TEXT) AS shipping_address_province_code,
CAST(NULL AS TEXT) AS shipping_address_zip,
CAST(NULL AS BOOLEAN) AS test,
CAST(NULL AS TEXT) AS token,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS INT) AS app_id,
CAST(NULL AS INT) AS checkout_id,
CAST(NULL AS TEXT) AS client_details_user_agent,
CAST(NULL AS TEXT) AS customer_locale,
CAST(NULL AS TEXT) AS order_status_url,
CAST(NULL AS TEXT) AS presentment_currency,
CAST(NULL AS BOOLEAN) AS confirmed,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS order_id,
user_id,
total_discounts,
total_discounts_set,
total_line_items_price,
total_line_items_price_set,
total_price,
total_price_set,
total_tax_set,
total_tax,
source_name,
subtotal_price,
taxes_included AS has_taxes_included,
total_weight,
total_tip_received,
landing_site_base_url,
location_id,
name,
note,
number,
order_number,
cancel_reason,
cart_token,
checkout_token,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(cancelled_at AS TIMESTAMP) AS TIMESTAMP)) AS cancelled_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(closed_at AS TIMESTAMP) AS TIMESTAMP)) AS closed_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(processed_at AS TIMESTAMP) AS TIMESTAMP)) AS processed_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_timestamp,
currency,
customer_id,
LOWER(email) AS email,
financial_status,
fulfillment_status,
referring_site,
billing_address_address_1,
billing_address_address_2,
billing_address_city,
billing_address_company,
billing_address_country,
billing_address_country_code,
billing_address_first_name,
billing_address_last_name,
billing_address_latitude,
billing_address_longitude,
billing_address_name,
billing_address_phone,
billing_address_province,
billing_address_province_code,
billing_address_zip,
browser_ip,
total_shipping_price_set,
shipping_address_address_1,
shipping_address_address_2,
shipping_address_city,
shipping_address_company,
shipping_address_country,
shipping_address_country_code,
shipping_address_first_name,
shipping_address_last_name,
shipping_address_latitude,
shipping_address_longitude,
shipping_address_name,
shipping_address_phone,
shipping_address_province,
shipping_address_province_code,
shipping_address_zip,
token,
app_id,
checkout_id,
client_details_user_agent,
customer_locale,
order_status_url,
presentment_currency,
test AS is_test_order,
_fivetran_deleted AS is_deleted,
buyer_accepts_marketing AS has_buyer_accepted_marketing,
confirmed AS is_confirmed,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
WHERE
NOT COALESCE(is_test_order, FALSE) AND NOT COALESCE(is_deleted, FALSE)
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
app_id | None | The ID of the app that created the order. |
checkout_id | None | ID of the order's checkout. |
order_status_url | None | The URL pointing to the order status web page, if applicable. |
billing_address_address_1 | None | The street address of the billing address. |
billing_address_address_2 | None | An optional additional field for the street address of the billing address. |
billing_address_city | None | The city, town, or village of the billing address. |
billing_address_company | None | The company of the person associated with the billing address. |
billing_address_country | None | The name of the country of the billing address. |
billing_address_country_code | None | The two-letter code (ISO 3166-1 format) for the country of the billing address. |
billing_address_first_name | None | The first name of the person associated with the payment method. |
billing_address_last_name | None | The last name of the person associated with the payment method. |
billing_address_latitude | None | The latitude of the billing address. |
billing_address_longitude | None | The longitude of the billing address. |
billing_address_name | None | The full name of the person associated with the payment method. |
billing_address_phone | None | The phone number at the billing address. |
billing_address_province | None | The name of the region (province, state, prefecture, …) of the billing address. |
billing_address_province_code | None | The two-letter abbreviation of the region of the billing address. |
billing_address_zip | None | The postal code (zip, postcode, Eircode, …) of the billing address. |
browser_ip | None | The IP address of the browser used by the customer when they placed the order. |
has_buyer_accepted_marketing | None | Whether the customer consented to receive email updates from the shop. |
cancel_reason | None | The reason why the order was canceled. |
cancelled_timestamp | None | The date and time when the order was canceled. |
cart_token | None | The ID of the cart that's associated with the order. |
closed_timestamp | None | The date and time when the order was closed (archived). |
created_timestamp | None | The autogenerated date and time when the order was created in Shopify. |
currency | None | The three-letter code for the shop currency. |
customer_id | None | The ID of the order's customer. |
None | The customer's email address. | |
financial_status | None | The status of payments associated with the order. Can only be set when the order is created |
fulfillment_status | None | The order's status in terms of fulfilled line items. |
order_id | None | The ID of the order, used for API purposes. This is different from the order_number property, which is the ID used by the shop owner and customer. |
landing_site_base_url | None | The URL for the page where the buyer landed when they entered the shop. |
location_id | None | The ID of the physical location where the order was processed. |
name | None | The order name, generated by combining the order_number property with the order prefix and suffix that are set in the merchant's general settings. |
note | None | An optional note that a shop owner can attach to the order. |
number | None | The order's position in the shop's count of orders. Numbers are sequential and start at 1. |
order_number | None | The order 's position in the shop's count of orders starting at 1001. Order numbers are sequential and start at 1001. |
processed_timestamp | None | The date and time when an order was processed. This value is the date that appears on your orders and that's used in the analytic reports. |
referring_site | None | The website where the customer clicked a link to the shop. |
shipping_address_address_1 | None | The street address of the shipping address. |
shipping_address_address_2 | None | An optional additional field for the street address of the shipping address. |
shipping_address_city | None | The city, town, or village of the shipping address. |
shipping_address_company | None | The company of the person associated with the shipping address. |
shipping_address_country | None | The name of the country of the shipping address. |
shipping_address_country_code | None | The two-letter code (ISO 3166-1 format) for the country of the shipping address. |
shipping_address_first_name | None | The first name of the person associated with the shipping address. |
shipping_address_last_name | None | The last name of the person associated with the shipping address. |
shipping_address_latitude | None | The latitude of the shipping address. |
shipping_address_longitude | None | The longitude of the shipping address. |
shipping_address_name | None | The full name of the person associated with the payment method. |
shipping_address_phone | None | The phone number at the shipping address. |
shipping_address_province | None | The name of the region (province, state, prefecture, …) of the shipping address. |
shipping_address_province_code | None | The two-letter abbreviation of the region of the shipping address. |
shipping_address_zip | None | The postal code (zip, postcode, Eircode, …) of the shipping address. |
source_name | None | Where the order originated. Can be set only during order creation, and is not writeable afterwards. |
subtotal_price | None | The price of the order in the shop currency after discounts but before shipping, taxes, and tips. |
has_taxes_included | None | Whether taxes are included in the order subtotal. |
is_test_order | None | Whether this is a test order. |
token | None | A unique token for the order. |
total_discounts | None | The total discounts applied to the price of the order in the shop currency. |
total_line_items_price | None | The sum of all line item prices in the shop currency. |
total_price | None | The sum of all line item prices, discounts, shipping, taxes, and tips in the shop currency. Must be positive. |
total_tax | None | The sum of all the taxes applied to the order in th shop currency. Must be positive). |
total_weight | None | The sum of all line item weights in grams. |
updated_timestamp | None | The date and time (ISO 8601 format) when the order was last modified. |
user_id | None | The ID of the user logged into Shopify POS who processed the order, if applicable. |
total_shipping_price_set | None | The total shipping price of the order, excluding discounts and returns, in shop and presentment currencies. If taxes_included is set to true, then total_shipping_price_set includes taxes. |
checkout_token | None | A unique value when referencing the checkout that's associated with the order. |
customer_locale | None | A two-letter or three-letter language code, optionally followed by a region modifier. |
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
total_tip_received | None | The sum of all the tips in the order in the shop currency. |
client_details_user_agent | None | Details of the browsing client, including software and operating versions. |
total_tax_set | None | The total tax applied to the order in shop and presentment currencies. |
total_discounts_set | None | The total discounts applied to the price of the order in shop and presentment currencies. |
presentment_currency | None | The presentment currency that was used to display prices to the customer. |
source_relation | None | The ID of the order placed on the originating platform. This value doesn't correspond to the Shopify ID that's generated from a completed draft. |
total_line_items_price_set | None | The total of all line item prices in shop and presentment currencies. |
total_price_set | None | The total price of the order in shop and presentment currencies. |
is_confirmed | None | Whether the order is confirmed. |
This SQL query creates a staging table for Shopify order adjustments. It starts with a base table that is currently empty, then defines a structure with null or empty values for all fields. The query then renames some fields, converts the _fivetran_synced timestamp to UTC, and selects all columns from the final CTE. This appears to be a placeholder structure that will be populated with actual data once order adjustments are made in Shopify.
CleaningOther/* this model will be all NULL until you have made an order adjustment in Shopify */
WITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_adjustment_tmp
), fields AS (
SELECT
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS DECIMAL(28, 6)) AS order_id,
CAST(NULL AS DECIMAL(28, 6)) AS refund_id,
CAST(NULL AS FLOAT) AS amount,
CAST(NULL AS TEXT) AS amount_set,
CAST(NULL AS FLOAT) AS tax_amount,
CAST(NULL AS TEXT) AS tax_amount_set,
CAST(NULL AS TEXT) AS kind,
CAST(NULL AS TEXT) AS reason,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS order_adjustment_id,
order_id,
refund_id,
amount,
amount_set,
tax_amount,
tax_amount_set,
kind,
reason,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
order_adjustment_id | None | The unique numeric identifier for the order adjustment. |
order_id | None | Reference to the order which the adjustment is associated. |
refund_id | None | Reference to the refund which the adjustment is associated. |
amount | None | Amount of the adjustment. |
tax_amount | None | Tax amount applied to the order adjustment in the shop currency. |
kind | None | The kind of order adjustment (eg. refund, restock, etc.). |
reason | None | The reason for the order adjustment. |
amount_set | None | Amount set towards the order adjustment in presentment and shop currencies. |
tax_amount_set | None | Tax amount set towards the order adjustment in the shop and presentment currencies. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty temporary staging table for Shopify order adjustments. It selects a single NULL column named '_dbt_source_relation' with a TEXT data type and limits the result to 0 rows. The comment indicates that this model will remain empty until an order adjustment is made in Shopify.
Other/* this model will be all NULL until you have made an order adjustment in Shopify */
SELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs data cleaning and standardization on the 'stg_shopify__order_discount_code' model. It casts various fields to specific data types, converts the '_fivetran_synced' timestamp to UTC, and uppercases the 'code' field. The query also adds a 'source_relation' field, which is set to an empty string.
CleaningWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_discount_code_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS FLOAT) AS amount,
CAST(NULL AS TEXT) AS code,
CAST(NULL AS INT) AS index,
CAST(NULL AS INT) AS order_id,
CAST(NULL AS TEXT) AS type,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
order_id,
index,
UPPER(code) AS code,
type,
amount,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
amount | None | The amount that's deducted from the order total. |
code | None | This property returns the discount code that was entered at checkout. Otherwise this property returns the title of the discount that was applied. |
order_id | None | Associated order ID. |
type | None | The type of discount - `fixed_amount`, `percentage`, or `shipping`. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
index | None | Pairs with `order_id` to provide unique identifier for order discount code. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It doesn't retrieve any actual data and is likely used as a placeholder or template for further development.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs a staging operation for Shopify order line data. It starts by selecting all columns from a temporary table, then casts them to specific data types in the 'fields' CTE. The 'final' CTE renames some columns, applies minor transformations (like converting boolean fields to 'is_' prefixed columns), and converts the '_fivetran_synced' timestamp to UTC. The query ultimately selects all columns from the 'final' CTE.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS DECIMAL(28, 6)) AS fulfillable_quantity,
CAST(NULL AS TEXT) AS fulfillment_status,
CAST(NULL AS BOOLEAN) AS gift_card,
CAST(NULL AS DECIMAL(28, 6)) AS grams,
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS DECIMAL(28, 6)) AS index,
CAST(NULL AS TEXT) AS name,
CAST(NULL AS DECIMAL(28, 6)) AS order_id,
CAST(NULL AS FLOAT) AS pre_tax_price,
CAST(NULL AS TEXT) AS pre_tax_price_set,
CAST(NULL AS FLOAT) AS price,
CAST(NULL AS TEXT) AS price_set,
CAST(NULL AS DECIMAL(28, 6)) AS product_id,
CAST(NULL AS DECIMAL(28, 6)) AS quantity,
CAST(NULL AS BOOLEAN) AS requires_shipping,
CAST(NULL AS TEXT) AS sku,
CAST(NULL AS BOOLEAN) AS taxable,
CAST(NULL AS TEXT) AS tax_code,
CAST(NULL AS TEXT) AS title,
CAST(NULL AS FLOAT) AS total_discount,
CAST(NULL AS TEXT) AS total_discount_set,
CAST(NULL AS DECIMAL(28, 6)) AS variant_id,
CAST(NULL AS TEXT) AS variant_title,
CAST(NULL AS TEXT) AS variant_inventory_management,
CAST(NULL AS TEXT) AS vendor,
CAST(NULL AS TEXT) AS properties,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS order_line_id,
index,
name,
order_id,
fulfillable_quantity,
fulfillment_status,
gift_card AS is_gift_card,
grams,
pre_tax_price,
pre_tax_price_set,
price,
price_set,
product_id,
quantity,
requires_shipping AS is_shipping_required,
sku,
taxable AS is_taxable,
tax_code,
title,
total_discount,
total_discount_set,
variant_id,
variant_title,
variant_inventory_management,
vendor,
properties,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
fulfillable_quantity | None | The amount available to fulfill, calculated as follows: quantity - max(refunded_quantity, fulfilled_quantity) - pending_fulfilled_quantity - open_fulfilled_quantity |
fulfillment_status | None | How far along an order is in terms line items fulfilled. |
is_gift_card | None | Whether the item is a gift card. If true, then the item is not taxed or considered for shipping charges. |
grams | None | The weight of the item in grams. |
order_line_id | None | The ID of the line item. |
name | None | The name of the product variant. |
order_id | None | The ID of the related order. |
price | None | The price of the item before discounts have been applied in the shop currency. |
product_id | None | The ID of the product that the line item belongs to. Can be null if the original product associated with the order is deleted at a later date. |
quantity | None | The number of items that were purchased. |
is_shipping_required | None | Whether the item requires shipping. |
sku | None | The item's SKU (stock keeping unit). |
is_taxable | None | Whether the item was taxable. |
title | None | The title of the product. |
total_discount | None | The total amount of the discount allocated to the line item in the shop currency. |
variant_id | None | The ID of the product variant. |
vendor | None | The name of the item's supplier. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
index | None | Index of the order line. |
pre_tax_price | None | The pre tax price of the line item in shop currency. |
pre_tax_price_set | None | The pre tax price of the line item in shop currency and presentment currency. |
price_set | None | The price of the line item in shop and presentment currencies. |
tax_code | None | Tax code applied to the line item. As multiple taxes can apply to a line item, we recommend referring to `stg_shopify__tax_line`. |
total_discount_set | None | The total amount allocated to the line item in the presentment currency. |
variant_title | None | The title of the product variant. |
variant_inventory_management | None | The fulfillment service that tracks the number of items in stock for the product variant. |
properties | None | Line item properties. |
This SQL query creates a staging table for Shopify order line refunds. It starts with a base table, then defines a fields CTE that casts all columns to specific data types with NULL values. The final CTE renames some columns and converts the _fivetran_synced timestamp to UTC. The query is designed to create a structure for order line refund data, even when no actual refunds have been made yet in Shopify.
CleaningOther/* this model will be all NULL until you have made an order line refund in Shopify */
WITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_line_refund_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS DECIMAL(28, 6)) AS location_id,
CAST(NULL AS DECIMAL(28, 6)) AS order_line_id,
CAST(NULL AS DECIMAL(28, 6)) AS subtotal,
CAST(NULL AS TEXT) AS subtotal_set,
CAST(NULL AS DECIMAL(28, 6)) AS total_tax,
CAST(NULL AS TEXT) AS total_tax_set,
CAST(NULL AS FLOAT) AS quantity,
CAST(NULL AS DECIMAL(28, 6)) AS refund_id,
CAST(NULL AS TEXT) AS restock_type,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS order_line_refund_id,
location_id,
order_line_id,
subtotal,
subtotal_set,
total_tax,
total_tax_set,
quantity,
refund_id,
restock_type,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
order_line_refund_id | None | The unique identifier of the line item in the refund. |
location_id | None | TThe unique identifier of the location where the items will be restockedBD |
order_line_id | None | The ID of the related line item in the order. |
quantity | None | The quantity of the associated line item that was returned. |
refund_id | None | The ID of the related refund. |
restock_type | None | How this refund line item affects inventory levels. |
subtotal | None | Subtotal amount of the order line refund in shop currency. |
total_tax | None | The total tax applied to the refund in the shop currency. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
subtotal_set | None | The subtotal of the refund line item in shop and presentment currencies. |
total_tax_set | None | The total tax of the line item in shop and presentment currencies. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. The query is designed to return no rows (LIMIT 0) and is likely used as a placeholder or template for future data when order line refunds are made in Shopify.
Other/* this model will be all NULL until you have made an order line refund in Shopify */
SELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for further operations in a dbt (data build tool) project, specifically for staging Shopify order line data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates a staging table for Shopify order note attributes. It starts with a base CTE that selects all columns from a temporary table. Then, it defines a fields CTE that casts specific columns to desired data types and adds a source_relation column. Finally, it creates a final CTE that selects and formats the columns, including converting the _fivetran_synced timestamp to UTC. The query then selects all columns from the final CTE.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_note_attribute_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TEXT) AS name,
CAST(NULL AS INT) AS order_id,
CAST(NULL AS TEXT) AS value,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
order_id,
name,
value,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
name | None | Name of the attribute. |
order_id | None | ID referencing the order the note attribute belongs to. |
value | None | Value of the attribute. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The LIMIT 0 ensures no rows are returned. This is likely used as a placeholder or template for a staging table in a dbt (data build tool) project.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs a staged transformation of Shopify order shipping line data. It starts by selecting all columns from a temporary staging table, then casts these columns to specific data types in the 'fields' CTE. The 'final' CTE renames some columns, creates a boolean field 'is_third_party_required' based on the 'requested_fulfillment_service_id' column, and converts the '_fivetran_synced' timestamp to UTC. The query primarily focuses on cleaning and structuring the data for further use.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_shipping_line_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TEXT) AS carrier_identifier,
CAST(NULL AS TEXT) AS code,
CAST(NULL AS TEXT) AS delivery_category,
CAST(NULL AS FLOAT) AS discounted_price,
CAST(NULL AS TEXT) AS discounted_price_set,
CAST(NULL AS INT) AS id,
CAST(NULL AS INT) AS order_id,
CAST(NULL AS TEXT) AS phone,
CAST(NULL AS FLOAT) AS price,
CAST(NULL AS TEXT) AS price_set,
CAST(NULL AS TEXT) AS requested_fulfillment_service_id,
CAST(NULL AS TEXT) AS source,
CAST(NULL AS TEXT) AS title,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS order_shipping_line_id,
order_id,
carrier_identifier,
code,
delivery_category,
discounted_price,
discounted_price_set,
phone,
price,
price_set,
NOT requested_fulfillment_service_id IS NULL AS is_third_party_required,
source,
title,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
carrier_identifier | None | A reference to the carrier service that provided the rate. Present when the rate was computed by a third-party carrier service. |
code | None | A reference to the shipping method. |
delivery_category | None | The general classification of the delivery method. |
discounted_price | None | The pre-tax shipping price with discounts applied in shop currency. |
discounted_price_set | None | The pre-tax shipping price with discounts applied (JSON) in presentment and shop currencies. |
order_shipping_line_id | None | A globally-unique identifier. |
order_id | None | ID of the associated order. |
phone | None | The phone number at the shipping address. |
price | None | Returns the price of the shipping line in shop currency. |
price_set | None | Returns the price of the shipping line (JSON) in shop and presentment currencies. |
is_third_party_required | None | The fulfillment service requested for the shipping method. Present if the shipping method requires processing by a third party fulfillment service. |
source | None | Returns the rate source for the shipping line. |
title | None | Returns the title of the shipping line. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, which is set to NULL. The query is limited to 0 rows, effectively creating a template or placeholder for the structure of the 'stg_shopify__order_shipping_line_tmp' model without actually returning any data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates a staging table for Shopify order shipping tax line data. It starts with a base table, then defines a fields CTE that sets up the structure with NULL or empty values for each column. The final CTE selects these fields, converting the _fivetran_synced timestamp to UTC. The main purpose appears to be setting up a consistent structure for this data, possibly as part of a larger ETL process.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_shipping_tax_line_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS index,
CAST(NULL AS INT) AS order_shipping_line_id,
CAST(NULL AS FLOAT) AS price,
CAST(NULL AS TEXT) AS price_set,
CAST(NULL AS FLOAT) AS rate,
CAST(NULL AS TEXT) AS title,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
order_shipping_line_id,
index,
price,
price_set,
rate,
title,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
index | None | Index (from 1) representing the order of shipping lines per order. |
order_shipping_line_id | None | ID of the order shipping line this record is associated with. |
price | None | The amount of tax, in shop currency, after discounts and before returns. |
price_set | None | The amount of tax, in shop and presentment currencies, after discounts and before returns (JSON). |
rate | None | The proportion of the line item price that the tax represents as a decimal. |
title | None | The name of the tax. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of TEXT data type. It's likely used as a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify order shipping tax line data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates a staging table for Shopify order tags. It starts with a base table, then defines the structure of the fields with appropriate data types. The final step converts the _fivetran_synced timestamp to UTC. The query doesn't perform any filtering, deduplication, or aggregation, but it does some basic data type casting and timezone conversion.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_tag_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS index,
CAST(NULL AS INT) AS order_id,
CAST(NULL AS TEXT) AS value,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
order_id,
index,
value,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
index | None | Index (starting from 1) representing when the tag was placed on the order. |
order_id | None | ID of the order being tagged. |
value | None | Value of the tag. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify order tag data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for further operations in a dbt (data build tool) project, specifically for a staging model related to Shopify orders.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates a staging table for Shopify order URL tags. It starts with a base table, then defines a fields structure with specific data types for each column. The final step converts the '_fivetran_synced' timestamp to UTC timezone. The query primarily focuses on data type casting and minimal transformation.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__order_url_tag_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TEXT) AS key,
CAST(NULL AS INT) AS order_id,
CAST(NULL AS TEXT) AS value,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
order_id,
key,
value,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
key | None | Key of the tag pair. |
order_id | None | ID of the order url being tagged. |
value | None | Value of the tag. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for further development or as a temporary staging table in a data pipeline.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query stages data from a Shopify price rule source table. It casts fields to appropriate data types, renames some columns for clarity, and converts timestamp fields to UTC. The query also adds a source_relation field and selects all columns from the transformed data.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__price_rule_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS allocation_limit,
CAST(NULL AS TEXT) AS allocation_method,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TEXT) AS customer_selection,
CAST(NULL AS TIMESTAMP) AS ends_at,
CAST(NULL AS INT) AS id,
CAST(NULL AS BOOLEAN) AS once_per_customer,
CAST(NULL AS FLOAT) AS prerequisite_quantity_range,
CAST(NULL AS FLOAT) AS prerequisite_shipping_price_range,
CAST(NULL AS FLOAT) AS prerequisite_subtotal_range,
CAST(NULL AS FLOAT) AS prerequisite_to_entitlement_purchase_prerequisite_amount,
CAST(NULL AS INT) AS quantity_ratio_entitled_quantity,
CAST(NULL AS INT) AS quantity_ratio_prerequisite_quantity,
CAST(NULL AS TIMESTAMP) AS starts_at,
CAST(NULL AS TEXT) AS target_selection,
CAST(NULL AS TEXT) AS target_type,
CAST(NULL AS TEXT) AS title,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS INT) AS usage_limit,
CAST(NULL AS FLOAT) AS value,
CAST(NULL AS TEXT) AS value_type,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS price_rule_id,
allocation_limit,
allocation_method,
customer_selection,
once_per_customer AS is_once_per_customer,
prerequisite_quantity_range AS prereq_min_quantity,
prerequisite_shipping_price_range AS prereq_max_shipping_price,
prerequisite_subtotal_range AS prereq_min_subtotal,
prerequisite_to_entitlement_purchase_prerequisite_amount AS prereq_min_purchase_quantity_for_entitlement,
quantity_ratio_entitled_quantity AS prereq_buy_x_get_this,
quantity_ratio_prerequisite_quantity AS prereq_buy_this_get_y,
target_selection,
target_type,
title,
usage_limit,
value,
value_type,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(starts_at AS TIMESTAMP) AS TIMESTAMP)) AS starts_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(ends_at AS TIMESTAMP) AS TIMESTAMP)) AS ends_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
allocation_limit | None | The number of times the discount can be allocated on the cart - if eligible. For example a Buy 1 hat Get 1 hat for free discount can be applied 3 times on a cart having more than 6 hats, where maximum of 3 hats get discounted - if the allocation_limit is 3. Empty (null) allocation_limit means unlimited number of allocations. |
allocation_method | None | The allocation method of the price rule. Valid values include `each` (the discount is applied to each of the entitled items. For example, for a price rule that takes $15 off, each entitled line item in a checkout will be discounted by $15) and `across` (the calculated discount amount will be applied across the entitled items. For example, for a price rule that takes $15 off, the discount will be applied across all the entitled items). |
created_at | None | The date and time (ISO 8601 format) when the price rule was created. |
customer_selection | None | The customer selection for the price rule. Valid values include `all` (the price rule is valid for all customers) and `prerequisite` (the customer must either belong to one of the customer segments specified by customer_segment_prerequisite_ids, or be one of the customers specified by prerequisite_customer_ids). |
ends_at | None | The date and time (ISO 8601 format) when the price rule ends. Must be after starts_at. |
price_rule_id | None | The ID for the price rule. |
is_once_per_customer | None | Boolean representing whether the generated discount code will be valid only for a single use per customer. This is tracked using customer ID. |
prereq_min_quantity | None | If `customer_selection` is `prerequisite`, the minimum number of items for the price rule to be applicable. The quantity of an entitled cart item must be greater than or equal to this value. |
prereq_max_shipping_price | None | If `customer_selection` is `prerequisite`, the maximum shipping price for the price rule to be applicable. The shipping price must be less than or equal to this value |
prereq_min_subtotal | None | If `customer_selection` is `prerequisite`, the minimum subtotal for the price rule to be applicable. The subtotal of the entitled cart items must be greater than or equal to this value for the discount to apply. |
prereq_min_purchase_quantity_for_entitlement | None | If `customer_selection` is `prerequisite`, the prerequisite purchase for a Buy X Get Y discount. The minimum purchase amount required to be entitled to the discount. |
prereq_buy_x_get_this | None | If `customer_selection` is `prerequisite`, in a Buy/Get ratio for a Buy X Get Y discount, this is the offered 'get' quantity. |
prereq_buy_this_get_y | None | If `customer_selection` is `prerequisite`, in a Buy/Get ratio for a Buy X Get Y discount, this defines the necessary 'buy' quantity. |
starts_at | None | The date and time (ISO 8601 format) when the price rule starts. |
target_selection | None | The target selection method of the price rule. Valid values include `all` (the price rule applies the discount to all line items in the checkout) and `entitled` (the price rule applies the discount to selected entitlements only). |
target_type | None | The target type that the price rule applies to. Valid values include `line_item` (the price rule applies to the cart's line items) and `shipping_line` (the price rule applies to the cart's shipping lines). |
title | None | The title of the price rule. This is used by the Shopify admin search to retrieve discounts. It is also displayed on the Discounts page of the Shopify admin for bulk discounts. Shopify recommends that this map onto the associated `discount_code.code`. |
updated_at | None | The date and time (ISO 8601 format) when the price rule was updated. |
usage_limit | None | The maximum number of times the price rule can be used, per discount code. |
value | None | The value of the price rule. If if the value of `target_type` is `shipping_line`, then only -100 is accepted. The value must be negative. |
value_type | None | The value type of the price rule. Valid values include `fixed_amount` (applies a discount of value as a unit of the store's currency. For example, if value is -30 and the store's currency is USD, then $30 USD is deducted when the discount is applied) and `percentage` (applies a percentage discount of value. For example, if value is -30, then 30% will be deducted when the discount is applied). If `target_type` is `shipping_line`, then only `percentage` is accepted. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of TEXT data type, initialized as NULL. The LIMIT 0 clause ensures no rows are returned. This query appears to be a placeholder or template, possibly used for schema definition or testing purposes in a dbt (data build tool) project.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs data transformation and cleaning on a Shopify product staging table. It casts various fields to specific data types, renames some columns, and converts timestamp fields to UTC timezone. The query also adds a 'source_relation' field and an 'is_deleted' flag based on the '_fivetran_deleted' column.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_tmp
), fields AS (
SELECT
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TEXT) AS handle,
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS TEXT) AS product_type,
CAST(NULL AS TIMESTAMP) AS published_at,
CAST(NULL AS TEXT) AS published_scope,
CAST(NULL AS TEXT) AS title,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS TEXT) AS vendor,
CAST(NULL AS TEXT) AS status,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS product_id,
handle,
product_type,
published_scope,
title,
vendor,
status,
_fivetran_deleted AS is_deleted,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(published_at AS TIMESTAMP) AS TIMESTAMP)) AS published_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
is_deleted | None | Whether the record has been deleted in the source system. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
created_timestamp | None | The date and time when the product was created. |
handle | None | A unique human-friendly string for the product. Automatically generated from the product's title. |
product_id | None | An unsigned 64-bit integer that's used as a unique identifier for the product. Each id is unique across the Shopify system. No two products will have the same id, even if they're from different shops. |
product_type | None | A categorization for the product used for filtering and searching products. |
published_timestamp | None | The date and time (ISO 8601 format) when the product was published. Can be set to null to unpublish the product from the Online Store channel. |
published_scope | None | Whether the product is published to the Point of Sale channel. |
title | None | The name of the product. |
updated_timestamp | None | The date and time when the product was last modified. |
vendor | None | The name of the product's vendor. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
status | None | The status of the product. Valid values: - active: The product is ready to sell and is available to customers on the online store, sales channels, and apps. By default, existing products are set to active. - archived: The product is no longer being sold and isn't available to customers on sales channels and apps. - draft: The product isn't ready to sell and is unavailable to customers on sales channels and apps. By default, duplicated and unarchived products are set to draft. |
This SQL query processes data from a staging table for Shopify product images. It casts various fields to specific data types, renames some columns, converts timestamp fields to UTC, and filters out deleted records. The query also adds a source_relation field and restructures the data for further use.
CleaningFilteringWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_image_tmp
), fields AS (
SELECT
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS INT) AS height,
CAST(NULL AS INT) AS id,
CAST(NULL AS INT) AS position,
CAST(NULL AS INT) AS product_id,
CAST(NULL AS TEXT) AS src,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS TEXT) AS variant_ids,
CAST(NULL AS INT) AS width,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS product_image_id,
product_id,
height,
position,
src,
variant_ids,
width,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
WHERE
NOT COALESCE(_fivetran_deleted, FALSE)
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
created_at | None | The date and time when the product image was created. The API returns this value in ISO 8601 format. |
height | None | Height dimension of the image which is determined on upload. |
product_image_id | None | Unique numeric identifier of the product image. |
position | None | The order of the product image in the list. The first product image is at position 1 and is the "main" image for the product. |
product_id | None | The id of the product associated with the image. |
src | None | Specifies the location of the product image. This parameter supports URL filters that you can use to retrieve modified copies of the image. |
updated_at | None | The date and time when the product image was last modified. The API returns this value in ISO 8601 format. |
variant_ids | None | An array of variant ids associated with the image. |
width | None | Width dimension of the image which is determined on upload. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, initialized to NULL. The LIMIT 0 clause ensures no rows are returned. This is likely used as a template or placeholder for a staging table in a dbt (data build tool) project.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs a basic transformation on data from a staging table. It casts several columns to specific data types, including setting some to NULL or empty string. The query also converts the '_fivetran_synced' timestamp to UTC timezone. The purpose seems to be standardizing the data structure and preparing it for further use.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_tag_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS index,
CAST(NULL AS INT) AS product_id,
CAST(NULL AS TEXT) AS value,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
product_id,
index,
value,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
index | None | Index (starting from 1) representing when the tag was placed on the product. |
product_id | None | ID of the product being tagged. |
value | None | Value of the tag. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of TEXT data type, initially set to NULL. The query is limited to 0 rows, effectively creating a template or skeleton for the 'stg_shopify__product_tag_tmp' model without any actual data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for further operations in a dbt (data build tool) model.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs data cleaning and transformation on a Shopify product variant dataset. It starts by casting columns to specific data types, renames some columns, and performs time zone conversions for timestamp fields. The query also coalesces inventory quantity fields and adjusts the boolean representation of the 'taxable' field.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__product_variant_tmp
), fields AS (
SELECT
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS DECIMAL(28, 6)) AS product_id,
CAST(NULL AS DECIMAL(28, 6)) AS inventory_item_id,
CAST(NULL AS DECIMAL(28, 6)) AS image_id,
CAST(NULL AS TEXT) AS title,
CAST(NULL AS FLOAT) AS price,
CAST(NULL AS TEXT) AS sku,
CAST(NULL AS DECIMAL(28, 6)) AS position,
CAST(NULL AS TEXT) AS inventory_policy,
CAST(NULL AS FLOAT) AS compare_at_price,
CAST(NULL AS TEXT) AS fulfillment_service,
CAST(NULL AS TEXT) AS inventory_management,
CAST(NULL AS BOOLEAN) AS taxable,
CAST(NULL AS TEXT) AS barcode,
CAST(NULL AS FLOAT) AS grams,
CAST(NULL AS DECIMAL(28, 6)) AS old_inventory_quantity,
CAST(NULL AS DECIMAL(28, 6)) AS inventory_quantity,
CAST(NULL AS FLOAT) AS weight,
CAST(NULL AS TEXT) AS weight_unit,
CAST(NULL AS TEXT) AS option_1,
CAST(NULL AS TEXT) AS option_2,
CAST(NULL AS TEXT) AS option_3,
CAST(NULL AS TEXT) AS tax_code,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS variant_id,
product_id,
inventory_item_id,
image_id,
title,
price,
sku,
position,
inventory_policy,
compare_at_price,
fulfillment_service,
inventory_management,
taxable AS is_taxable,
barcode,
grams,
COALESCE(inventory_quantity, old_inventory_quantity) AS inventory_quantity,
weight,
weight_unit,
option_1,
option_2,
option_3,
tax_code,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
barcode | None | The barcode, UPC, or ISBN number for the product. |
compare_at_price | None | The original price of the item before an adjustment or a sale in shop currency |
created_timestamp | None | The date and time (ISO 8601 format) when the product variant was created. |
fulfillment_service | None | The fulfillment service associated with the product variant. |
grams | None | The weight of the product variant in grams. |
variant_id | None | The unique numeric identifier for the product variant. |
image_id | None | The unique numeric identifier for a product's image. The image must be associated to the same product as the variant. |
inventory_item_id | None | The unique identifier for the inventory item, which is used in the Inventory API to query for inventory information. |
inventory_management | None | The fulfillment service that tracks the number of items in stock for the product variant. |
inventory_policy | None | Whether customers are allowed to place an order for the product variant when it's out of stock. |
inventory_quantity | None | An aggregate of inventory across all locations. To adjust inventory at a specific location, use the InventoryLevel resource. |
option_1 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
option_2 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
option_3 | None | The custom properties that a shop owner uses to define product variants. You can define three options for a product variant: option1, option2, option3. |
position | None | The order of the product variant in the list of product variants. The first position in the list is 1. The position of variants is indicated by the order in which they are listed. |
price | None | The price of the product variant. |
product_id | None | The unique numeric identifier for the product. |
sku | None | A unique identifier for the product variant in the shop. Required in order to connect to a FulfillmentService. |
is_taxable | None | Whether a tax is charged when the product variant is sold. |
tax_code | None | This parameter applies only to the stores that have the Avalara AvaTax app installed. Specifies the Avalara tax code for the product variant. |
title | None | The title of the product variant. The title field is a concatenation of the option1, option2, and option3 fields. You can only update title indirectly using the option fields. |
updated_timestamp | None | The date and time when the product variant was last modified. Gets returned in ISO 8601 format. |
weight | None | The weight of the product variant in the unit system specified with weight_unit. |
weight_unit | None | The unit of measurement that applies to the product variant's weight. If you don't specify a value for weight_unit, then the shop's default unit of measurement is applied. Valid values: g, kg, oz, and lb. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of TEXT data type. It's likely used as a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify product variant data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates a staging table for Shopify refund data. It starts with a base CTE that selects all columns from a temporary table. Then, it defines a fields CTE that casts all columns to specific data types, initially setting them to NULL or empty strings. Finally, it selects and renames some columns, converts timestamp fields to UTC, and prepares the data for further processing.
CleaningOther/* this model will be all NULL until you have made a refund in Shopify */
WITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__refund_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS TEXT) AS note,
CAST(NULL AS DECIMAL(28, 6)) AS order_id,
CAST(NULL AS TIMESTAMP) AS processed_at,
CAST(NULL AS BOOLEAN) AS restock,
CAST(NULL AS TEXT) AS total_duties_set,
CAST(NULL AS DECIMAL(28, 6)) AS user_id,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS refund_id,
note,
order_id,
restock,
total_duties_set,
user_id,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(processed_at AS TIMESTAMP) AS TIMESTAMP)) AS processed_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
refund_id | None | The unique numeric identifier for the refund. |
created_at | None | Timestamp of the date when the refund was created. |
processed_at | None | Timestamp of the date when the refund was processed. |
note | None | User generated note attached to the refund. |
restock | None | Boolean indicating if the refund is a result of a restock. |
user_id | None | Reference to the user id which generated the refund. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
total_duties_set | None | Record representing total duties set for the refund. |
order_id | None | Reference to the order which the refund is associated. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates a placeholder or dummy table for the 'model.shopify_source.stg_shopify__refund_tmp' model. It selects a single column named '_dbt_source_relation' with a NULL value of TEXT type, but limits the result to 0 rows. This effectively creates an empty table structure that can be used as a template or placeholder until actual refund data is available in Shopify.
Other/* this model will be all NULL until you create a refund in Shopify */
SELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query processes data from a Shopify shop source table. It performs type casting, renames some columns, and applies various transformations to clean and standardize the data. The query also creates new boolean fields from existing data and converts timezone for timestamp fields. It's primarily focused on data cleaning and light featurization.
CleaningFeaturizationWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__shop_tmp
), fields AS (
SELECT
CAST(NULL AS BOOLEAN) AS _fivetran_deleted,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TEXT) AS address_1,
CAST(NULL AS TEXT) AS address_2,
CAST(NULL AS BOOLEAN) AS checkout_api_supported,
CAST(NULL AS TEXT) AS city,
CAST(NULL AS TEXT) AS cookie_consent_level,
CAST(NULL AS TEXT) AS country,
CAST(NULL AS TEXT) AS country_code,
CAST(NULL AS TEXT) AS country_name,
CAST(NULL AS BOOLEAN) AS county_taxes,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TEXT) AS currency,
CAST(NULL AS TEXT) AS customer_email,
CAST(NULL AS TEXT) AS domain,
CAST(NULL AS BOOLEAN) AS eligible_for_card_reader_giveaway,
CAST(NULL AS BOOLEAN) AS eligible_for_payments,
CAST(NULL AS TEXT) AS email,
CAST(NULL AS TEXT) AS enabled_presentment_currencies,
CAST(NULL AS TEXT) AS google_apps_domain,
CAST(NULL AS BOOLEAN) AS google_apps_login_enabled,
CAST(NULL AS BOOLEAN) AS has_discounts,
CAST(NULL AS BOOLEAN) AS has_gift_cards,
CAST(NULL AS BOOLEAN) AS has_storefront,
CAST(NULL AS TEXT) AS iana_timezone,
CAST(NULL AS INT) AS id,
CAST(NULL AS FLOAT) AS latitude,
CAST(NULL AS FLOAT) AS longitude,
CAST(NULL AS TEXT) AS money_format,
CAST(NULL AS TEXT) AS money_in_emails_format,
CAST(NULL AS TEXT) AS money_with_currency_format,
CAST(NULL AS TEXT) AS money_with_currency_in_emails_format,
CAST(NULL AS TEXT) AS myshopify_domain,
CAST(NULL AS TEXT) AS name,
CAST(NULL AS BOOLEAN) AS password_enabled,
CAST(NULL AS TEXT) AS phone,
CAST(NULL AS TEXT) AS plan_display_name,
CAST(NULL AS TEXT) AS plan_name,
CAST(NULL AS BOOLEAN) AS pre_launch_enabled,
CAST(NULL AS TEXT) AS primary_locale,
CAST(NULL AS TEXT) AS province,
CAST(NULL AS TEXT) AS province_code,
CAST(NULL AS BOOLEAN) AS requires_extra_payments_agreement,
CAST(NULL AS BOOLEAN) AS setup_required,
CAST(NULL AS TEXT) AS shop_owner,
CAST(NULL AS TEXT) AS source,
CAST(NULL AS BOOLEAN) AS tax_shipping,
CAST(NULL AS BOOLEAN) AS taxes_included,
CAST(NULL AS TEXT) AS timezone,
CAST(NULL AS TIMESTAMP) AS updated_at,
CAST(NULL AS TEXT) AS weight_unit,
CAST(NULL AS TEXT) AS zip,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS shop_id,
name,
_fivetran_deleted AS is_deleted,
address_1,
address_2,
city,
province,
province_code,
country,
country_code,
country_name,
zip,
latitude,
longitude,
CASE WHEN county_taxes IS NULL THEN FALSE ELSE county_taxes END AS has_county_taxes,
currency,
enabled_presentment_currencies,
customer_email,
email,
domain,
phone,
timezone,
iana_timezone,
primary_locale,
weight_unit,
myshopify_domain,
cookie_consent_level,
shop_owner,
source,
tax_shipping AS has_shipping_taxes,
CASE WHEN taxes_included IS NULL THEN FALSE ELSE taxes_included END AS has_taxes_included_in_price,
has_discounts,
has_gift_cards,
has_storefront,
checkout_api_supported AS has_checkout_api_supported,
eligible_for_card_reader_giveaway AS is_eligible_for_card_reader_giveaway,
eligible_for_payments AS is_eligible_for_payments,
google_apps_domain,
CASE
WHEN google_apps_login_enabled IS NULL
THEN FALSE
ELSE google_apps_login_enabled
END AS is_google_apps_login_enabled,
money_format,
money_in_emails_format,
money_with_currency_format,
money_with_currency_in_emails_format,
plan_display_name,
plan_name,
password_enabled AS is_password_enabled,
pre_launch_enabled AS is_pre_launch_enabled,
requires_extra_payments_agreement AS is_extra_payments_agreement_required,
setup_required AS is_setup_required,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(updated_at AS TIMESTAMP) AS TIMESTAMP)) AS updated_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
is_deleted | None | Boolean representing whether the record was soft-deleted in Shopify. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
address_1 | None | The shop's street address. |
address_2 | None | The optional second line of the shop's street address. |
has_checkout_api_supported | None | Boolean representing whether the shop is capable of accepting payments directly through the Checkout API. |
city | None | The shop's city. |
cookie_consent_level | None | The cookie consent level defined on the shop's online store. |
country | None | The shop's country. In most cases, this value matches the country_code. |
country_code | None | The two-letter country code corresponding to the shop's country. |
country_name | None | The shop's normalized country name. |
has_county_taxes | None | Boolean representing whether the shop is applying taxes on a per-county basis. Only applicable to shops based in the US. Either `true` or `null` (not false, according to Shopify API docs). |
created_at | None | The date and time (ISO 8601) when the shop was created. |
currency | None | The three-letter code (ISO 4217 format) for the shop's default currency. |
customer_email | None | The contact email used for communication between the shop owner and the customer. |
domain | None | The shop's domain. |
is_eligible_for_card_reader_giveaway | None | Boolean representing whether the shop is eligible to receive a free credit card reader from Shopify. |
is_eligible_for_payments | None | Boolean representing whether the shop is eligible to use Shopify Payments. |
None | The contact email used for communication between Shopify and the shop owner. | |
enabled_presentment_currencies | None | An array of of enabled currencies (ISO 4217 format) that the shop accepts. Merchants can enable currencies from their Shopify Payments settings in the Shopify Admin. |
google_apps_domain | None | The GSuite URL for the store, if applicable. |
is_google_apps_login_enabled | None | Boolean representing whether the GSuite login is enabled. Shops with this feature will be able to log in through the GSuite login page. Valid values are `true` and `null`. |
has_discounts | None | Boolean representing whether any active discounts exist for the shop. |
has_gift_cards | None | Boolean representing whether any active gift cards exist for the shop. |
has_storefront | None | Boolean representing whether this shop has an online store. |
iana_timezone | None | The name of the timezone assigned by the [IANA](https://www.iana.org/time-zones). |
shop_id | None | The ID for the shop. A 64-bit unsigned integer. |
latitude | None | The latitude of the shop's location. |
longitude | None | The longitude of the shop's location. |
money_format | None | A string representing the way currency is formatted when the currency isn't specified. |
money_in_emails_format | None | A string representing the way currency is formatted in email notifications when the currency isn't specified. |
money_with_currency_format | None | A string representing the way currency is formatted when the currency is specified. |
money_with_currency_in_emails_format | None | A string representing the way currency is formatted in email notifications when the currency is specified. |
myshopify_domain | None | The shop's .myshopify.com domain. |
name | None | The name of the shop. |
is_password_enabled | None | Boolean representing whether the password protection page is enabled on the shop's online store. |
phone | None | The contact phone number for the shop. |
plan_display_name | None | The display name of the Shopify plan the shop is on. |
plan_name | None | The name of the Shopify plan the shop is on. |
is_pre_launch_enabled | None | Boolean representing whether the pre-launch page is enabled on the shop's online store. |
primary_locale | None | The shop's primary locale, as configured in the language settings of the shop's theme. |
province | None | The shop's normalized province or state name. |
province_code | None | The two- or three-letter code for the shop's province or state. |
is_extra_payments_agreement_required | None | Boolean representing whether the shop requires an extra Shopify Payments agreement. |
is_setup_required | None | Boolean representing whether the shop has any outstanding setup steps. |
shop_owner | None | The username of the shop owner. |
source | None | The handle of the partner account that referred the merchant to Shopify, if applicable. |
has_shipping_taxes | None | Boolean representing whether taxes are charged for shipping. Valid values are true or false. |
has_taxes_included_in_price | None | Boolean representing whether applicable taxes are included in product prices. Valid values are true or null. |
timezone | None | The name of the timezone the shop is in. |
updated_at | None | The date and time (ISO 8601) when the shop was last updated. |
weight_unit | None | The default unit of weight measurement for the shop. |
zip | None | The shop's zip or postal code. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT, initialized to NULL. The LIMIT 0 clause ensures no rows are returned. This is likely used as a template or placeholder for a staging table in a dbt (data build tool) project.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query creates a staging table for tax line data from a Shopify source. It starts with a base table, then defines a fields CTE that casts columns to specific data types, including several NULL casts. The final CTE selects these fields, converts the _fivetran_synced timestamp to UTC, and includes a source_relation column. The main query then selects all columns from the final CTE.
CleaningOtherWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__tax_line_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS INT) AS index,
CAST(NULL AS INT) AS order_line_id,
CAST(NULL AS FLOAT) AS price,
CAST(NULL AS TEXT) AS price_set,
CAST(NULL AS FLOAT) AS rate,
CAST(NULL AS TEXT) AS title,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
index,
order_line_id,
price,
price_set,
rate,
title,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
index | None | The index of the tax line. |
order_line_id | None | The order line that this tax line is associated with. |
price | None | The amount of tax, in shop currency, after discounts and before returns. |
price_set | None | The amount of tax, in shop and presentment currencies, after discounts and before returns. |
rate | None | The proportion of the line item price that the tax represents as a decimal. |
title | None | The name of the tax. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for a staging table in a dbt (data build tool) project, specifically for Shopify tax line data.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query performs data transformation and cleaning on a Shopify tender transaction dataset. It starts by selecting all columns from a temporary staging table, then defines a set of fields with specific data types. The query then selects and renames certain fields, converts timestamps to UTC, and filters out test transactions. The result is a cleaned and standardized version of the tender transaction data.
CleaningFilteringWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__tender_transaction_tmp
), fields AS (
SELECT
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS FLOAT) AS amount,
CAST(NULL AS TEXT) AS currency,
CAST(NULL AS INT) AS id,
CAST(NULL AS INT) AS order_id,
CAST(NULL AS TEXT) AS payment_method,
CAST(NULL AS TIMESTAMP) AS processed_at,
CAST(NULL AS TEXT) AS remote_reference,
CAST(NULL AS BOOLEAN) AS test,
CAST(NULL AS INT) AS user_id,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS transaction_id,
order_id,
amount,
currency,
payment_method,
remote_reference,
user_id,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(processed_at AS TIMESTAMP) AS TIMESTAMP)) AS processed_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
WHERE
NOT COALESCE(test, FALSE)
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
amount | None | The amount of the tender transaction in the shop's currency. |
currency | None | The three-letter code (ISO 4217 format) for the currency used for the tender transaction. |
transaction_id | None | The ID of the transaction. |
order_id | None | The ID of the order that the tender transaction belongs to. |
payment_method | None | Information about the payment method used for this transaction. Valid values include: - credit_card - cash - android_pay - apple_pay - google_pay - samsung_pay - shopify_pay - amazon - klarna - paypal - unknown - other |
processed_at | None | The date and time (ISO 8601 format) when the tender transaction was processed. |
remote_reference | None | The remote (gateway) reference associated with the tender. |
user_id | None | The ID of the user logged into the Shopify POS device that processed the tender transaction, if applicable. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for further operations in a dbt (data build tool) project, specifically for the 'stg_shopify__tender_transaction_tmp' model in the Shopify source.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|
This SQL query stages data from a Shopify transactions source table. It casts various fields to specific data types, renames some columns, converts timezone for timestamp fields to UTC, and filters out test transactions. The query prepares the data for further processing or analysis by standardizing the structure and ensuring data quality.
CleaningFilteringWITH base AS (
SELECT
*
FROM TEST.PUBLIC_stg_shopify.stg_shopify__transaction_tmp
), fields AS (
SELECT
CAST(NULL AS DECIMAL(28, 6)) AS id,
CAST(NULL AS DECIMAL(28, 6)) AS order_id,
CAST(NULL AS DECIMAL(28, 6)) AS refund_id,
CAST(NULL AS DECIMAL(28, 6)) AS amount,
CAST(NULL AS TIMESTAMP) AS created_at,
CAST(NULL AS TIMESTAMP) AS processed_at,
CAST(NULL AS DECIMAL(28, 6)) AS device_id,
CAST(NULL AS TEXT) AS gateway,
CAST(NULL AS TEXT) AS source_name,
CAST(NULL AS TEXT) AS message,
CAST(NULL AS TEXT) AS currency,
CAST(NULL AS DECIMAL(28, 6)) AS location_id,
CAST(NULL AS DECIMAL(28, 6)) AS parent_id,
CAST(NULL AS TEXT) AS payment_avs_result_code,
CAST(NULL AS TEXT) AS payment_credit_card_bin,
CAST(NULL AS TEXT) AS payment_cvv_result_code,
CAST(NULL AS TEXT) AS payment_credit_card_number,
CAST(NULL AS TEXT) AS payment_credit_card_company,
CAST(NULL AS TEXT) AS kind,
CAST(NULL AS TEXT) AS receipt,
CAST(NULL AS DECIMAL(28, 6)) AS currency_exchange_id,
CAST(NULL AS DECIMAL(28, 6)) AS currency_exchange_adjustment,
CAST(NULL AS DECIMAL(28, 6)) AS currency_exchange_original_amount,
CAST(NULL AS DECIMAL(28, 6)) AS currency_exchange_final_amount,
CAST(NULL AS TEXT) AS currency_exchange_currency,
CAST(NULL AS TEXT) AS error_code,
CAST(NULL AS TEXT) AS status,
CAST(NULL AS BOOLEAN) AS test,
CAST(NULL AS DECIMAL(28, 6)) AS user_id,
CAST(NULL AS TIMESTAMP) AS _fivetran_synced,
CAST(NULL AS TIMESTAMP) AS authorization_expires_at,
CAST(NULL AS TEXT) AS authorization_code,
CAST('' AS TEXT) AS source_relation
FROM base
), final AS (
SELECT
id AS transaction_id,
order_id,
refund_id,
amount,
device_id,
gateway,
source_name,
message,
currency,
location_id,
parent_id,
payment_avs_result_code,
payment_credit_card_bin,
payment_cvv_result_code,
payment_credit_card_number,
payment_credit_card_company,
kind,
receipt,
currency_exchange_id,
currency_exchange_adjustment,
currency_exchange_original_amount,
currency_exchange_final_amount,
currency_exchange_currency,
error_code,
status,
user_id,
authorization_code,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(created_at AS TIMESTAMP) AS TIMESTAMP)) AS created_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(processed_at AS TIMESTAMP) AS TIMESTAMP)) AS processed_timestamp,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(authorization_expires_at AS TIMESTAMP) AS TIMESTAMP)) AS authorization_expires_at,
CONVERT_TIMEZONE('UTC', 'UTC', CAST(CAST(_fivetran_synced AS TIMESTAMP) AS TIMESTAMP)) AS _fivetran_synced,
source_relation
FROM fields
WHERE
NOT COALESCE(test, FALSE)
)
SELECT
*
FROM final
Name | Type | Comment |
---|---|---|
transaction_id | None | The ID for the transaction. |
order_id | None | The ID for the order that the transaction is associated with. |
refund_id | None | The ID associated with a refund in the refund table. |
amount | None | The amount of money included in the transaction. |
authorization_code | None | The authorization code associated with the transaction. |
created_timestamp | None | The date and time when the transaction was created. |
processed_timestamp | None | The date and time when a transaction was processed. |
device_id | None | The ID for the device. |
gateway | None | The name of the gateway the transaction was issued through. |
source_name | None | The origin of the transaction. |
message | None | A string generated by the payment provider with additional information about why the transaction succeeded or failed. |
currency | None | The three-letter code (ISO 4217 format) for the currency used for the payment. |
location_id | None | The ID of the physical location where the transaction was processed. |
parent_id | None | The ID of an associated transaction. |
payment_avs_result_code | None | The response code from the address verification system. |
payment_credit_card_bin | None | The issuer identification number (IIN), formerly known as bank identification number (BIN) of the customer's credit card. |
payment_cvv_result_code | None | The response code from the credit card company indicating whether the customer entered the card security code, or card verification value, correctly. |
payment_credit_card_number | None | The customer's credit card number, with most of the leading digits redacted. |
payment_credit_card_company | None | The name of the company that issued the customer's credit card. |
kind | None | The transaction's type. |
receipt | None | A transaction receipt attached to the transaction by the gateway. |
currency_exchange_id | None | The ID of the adjustment. |
currency_exchange_adjustment | None | The difference between the amounts on the associated transaction and the parent transaction. |
currency_exchange_original_amount | None | The amount of the parent transaction in the shop currency. |
currency_exchange_final_amount | None | The amount of the associated transaction in the shop currency. |
currency_exchange_currency | None | The shop currency. |
error_code | None | A standardized error code, independent of the payment provider. |
status | None | The status of the transaction. |
test | None | Whether the transaction is a test transaction. |
user_id | None | The ID for the user who was logged into the Shopify POS device when the order was processed, if applicable. |
_fivetran_synced | None | The time when a record was last updated by Fivetran. |
source_relation | None | The schema or database this record came from if you are making use of the `shopify_union_schemas` or `shopify_union_databases` variables, respectively. Empty string if you are not using either of these variables to union together multiple Shopify connectors. |
authorization_expires_at | None | The date and time (ISO 8601 format) when the Shopify Payments authorization expires. |
This SQL query creates an empty result set with a single column named '_dbt_source_relation' of type TEXT. It's likely used as a placeholder or template for further data manipulation or as a starting point for a more complex query.
OtherSELECT
CAST(NULL AS TEXT) AS _dbt_source_relation
LIMIT 0
Name | Type | Comment |
---|