To finish off the system we needed to orchestrate the process of extracting, transforming loading and visualising the data with the added function of emailing the reports as well.

There’s quite a lot to airflow and I highly recommend you take a look at the following udemy course

I’ll post a link to my code here

It’s a very basic example of using airflow however I found it to be a great experience with the amount of customisation you can do with it.

I enjoyed doing this project because I gave me the opportunity to use new technologies and deliver a relatively small project end to end.