Capturing Every Change From Shopify’s Sharded Monolith

T1Covered by 1 source · first reported by Stripe / Shopify Eng / Netflix TechBlog / Dropbox Tech / Slack Eng / Spotify Eng at 12 Mar 2021, 18:51 UTC

Shopify is moving from existing query-based CDC (Change-Data Capture) tooling (Longboat) to an immutable, append-only, log-based mechanism. This has required a complete re-platforming, involving a shift to new, open source technology such as Kafka Connect and Debezium.

dev

Sources

• T1Capturing Every Change From Shopify’s Sharded MonolithStripe / Shopify Eng / Netflix TechBlog / Dropbox Tech / Slack Eng / Spotify Eng12 Mar 2021, 18:51