From the course: Data Engineering: dbt for SQL
Join today to access over 25,500 courses taught by industry experts.
Incremental materializations
From the course: Data Engineering: dbt for SQL
Incremental materializations
- [Instructor] While table and view strategies are likely the most commonly seen strategies, there's one particular strategy that can be very powerful if used properly. That is the incremental strategy. As you know, DBT offers a few materialization strategies. These traditional materializations in DBT involve creating intermediary tables that start the results of complex SQL queries. But let's say you have a table that's growing in perpetuity. For example, you decide to store all of your sales in a sales table. You know that this table every day will have new sales appended to it. In a regular table transformation, every child table of sales and intermediate table before the final nodes in the pipeline are updated. In DBT, these intermediate tables are materialized in their entirety. Whenever a DBT job runs, regardless of whether or not the underlying data has changed meaning you would have to reprocess and regenerate…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.