To finish off the system we needed to orchestrate the process of extracting, transforming loading and visualising the data with the added function of emailing the reports as well.

There’s quite a lot to airflow and I highly recommend you take a look at the following udemy course

I’ll post a link to my code here

It’s a very basic example of using airflow however I found it to be a great experience with the amount of customisation you can do with it.

I enjoyed doing this project because I gave me the opportunity to use new technologies and deliver a relatively small project end to end.

This article follows on from my previous post

So once the star schema is loaded into our database(I’m using google big query — in the following post I will go over how I loaded that in the airflow section)

We will generate the assets /charts for the reports —…

This article follows on from my previous article

In the past when dealing with pre-formatted reports I’ve either had to ask for direct access to the reporting database and when thats not been possible i’ve had to rely on vba/macros or manual intervention to cleanse the report into a…


Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store