Create a robust pipeline for an Actuarial Team. The pipeline needed to include migration from Oracle to Trino. This needed to support the team’s data processing needs and provide them with accurate and up-to-date data to back up their decision-making processes.
By using Trino, Airflow, and DBT as core stack, we reduced the time and effort required to extract meaningful insights from our data and make informed decisions based on the most current information. We were also able to create incremental models using DBT to overcome this so Actuarial users don’t have to reload and analyse all their data every time a change is made.
DBT
Airflow
Great Expectations
AWS
