Naming conventions, yml files, and folder structures — Dbt is a hot tool in the analytics world, and it only continues to become more and more popular. It is used by analysts and analytics engineers alike to run modular code in a way that is faster and more dependable. Organization is a key component of using dbt in…

A new tool that does what Airflow does but better — When I first joined my current company my first major project was deciding on a data stack we wanted to use for analytics. I researched the best solutions for ingestion, warehousing, transformation, and orchestration. All the others were fairly straightforward but the orchestration part stumped me. Sure, Airflow may be…

How this tool is changing the way data analysts work — If you’ve heard of an analytics engineer or know about data modeling, then you’re probably familiar with dbt (if you aren’t familiar, read what it is here). It is quite the hot tool right now because it makes the data team’s life easier! And isn’t that what we all want…

Save time, increase transparency, and provide high-quality data — As an analytics engineer, I am constantly tracking down owners of various datasets I see referenced in old data models. There are a ton of Google sheets floating around, being used in complex data models, yet nobody really knows how they are being used. I typically have to message multiple…

Everything you need to know about transforming your data — why you should do it, the benefits of this practice, and tips for modeling your data — If you’re a data analyst or analytics engineer, then you are familiar with the process of receiving a request from someone in a business team. Typically, they give you the key metrics they need to track, then you go through the process of deciding which datasets are needed, what calculations…

How data-diff can save you hours of time validating data models — If you’ve ever had to migrate your data warehouse to a new platform or account then you know how time-consuming and painful this can be. Recently, I’ve had to migrate my data from one Snowflake account to another, reconfiguring all of the data ingestion pipelines and orchestration. You quickly realize…

What they are and how to implement them — dbt is all the rage right now for data transformation. And, there’s a reason why! dbt makes it easy to write modular code, flipping the traditional process of writing data models on its head. You no longer have to repeat the same lines of code over and over again, making…

Learn how to detect data quality issues on data syncs with re_data — Have you ever been in a situation where you’ve been using a data model for months only to find out that the quality of the data was compromised? Unfortunately, this tends to be a fairly common problem in the modern data stack world. It can be easy to let data…

Data warehouse, ingestion, transformation, orchestration, and monitoring Modern data stacks have grown in popularity since we realized how important data really is to a business. They came about as we moved from on-prem data storage to the cloud, allowing us to access our data quicker and easier than ever. These…