28 Jul Using macros in dbtvault – Staging
Staging in dbtvault
Welcome to a new series of videos about dbtvault, in which we will be showing you tips and tricks with the tool. This is the first video in the series “Using macros provided in dbtvault” and Alex – who is the lead developer of dbtvault will show you how to create a prime staging view and how to create the two main configurations in the prime stage layer: Derived columns and Hashed columns.
“This is the first video of many – the next videos will be on Hubs, Links and Satellites. We have much more planned, and you have a suggestion please reach out on our community slack channel.
The first step is to create a prime staging view which comes right after the raw staging layer where your metadata has landed. There are lots of things that you can do in the prime stage view – however here we focus on two types of columns.
Alex is using the TPCH dataset as an example – and uses a YAML metadata approach, as it’s easier to read when we have more detailed metadata. This is just Alex’s preference however, you can provide metadata in any way to the stage macro, there is no difference in the outcome.
After the prime stage view has been setup – Alex moves onto creating the derived columns – you can find the code he is using in the dbtvault documentation or just type it out as shown in the video. You can find the dbtvault documentation here.
Towards the end of the video – Alex shows how to create hashed columns. This is one of the most powerful features that the dbtvault stage macro has to offer. It provides a standard hashing algorithm that you can apply to any column with many versatile configurations. You can read more about hashed columns in the dbtvault documentation.
This is the first video of many – the next videos will be on Hubs, Links and Satellites. We have much more planned, and you have a suggestion please reach out on our community slack channel.
If you want to find out more about dbtvault, find us on all popular social media outlets (LinkedIn, Facebook, Twitter) and look out for announcements as we are continually working on updates to improve our package!
Lastly, we offer paid support for dbtvault – which you can find out more about by clicking here.