A datalake house project build with Spark, Airflow, Docker, Minio and much more !

Data Pipeline built using Airflow, dbt, Kakfa, Spark, GCP, Docker, Terraform

Streaming Data Pipeline for NYC taxi Rides build using kafka and Pyspark locally running on docker containers

My Learning from DataTalksClub data-engineering-zoomcamp

Data Pipeline for unstructured data on AWS with spark