Insights
Product, design, AI, and engineering perspectives from our team.
Measuring Data Engineering Team Productivity: A Framework
Most engineering teams track DORA metrics for software delivery, but data engineering needs a different lens. This article sets out a practical framework covering throughput, pipeline reliability, and stakeholder satisfaction for growing data functions.
From Spreadsheets to Data Infrastructure: A Migration Path
Many growing Australian companies still run finance and operations reporting on spreadsheets long after they've outgrown them. This guide covers the signs it's time to move, what a first data infrastructure build should include, and how to sequence the migration without disrupting month-end.
Data Product Management: Treating Datasets Like Products
Most internal datasets have no owner, no documented SLA, and no versioning discipline — which is why downstream teams stop trusting them and start duplicating them. This piece lays out a lightweight framework for treating a handful of critical datasets as products, without committing to a full data mesh rebuild.
Change Data Capture for AI-Ready Data Pipelines
Change data capture (CDC) keeps analytics, search, and AI features in sync with source systems in near real time — but it's not free. This explainer covers how CDC works, where it fits in an AI-ready data stack, and when the operational overhead is actually worth it for an Australian engineering team.
Data Infrastructure Consulting: Building the AI Foundation
Most growing Australian organisations don't have a data problem — they have a data infrastructure problem. This post covers the modern data stack (Snowflake, BigQuery, Databricks), ELT vs ETL trade-offs, data contract patterns, and Australian data sovereignty considerations, with a practical look at consolidating five warehouses into one.