In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our cloud provider.
-
Updated
Dec 18, 2023
#
modern-data-engineering
Here are 3 public repositories matching this topic...
Extract Load Transform (ELT) framework is a metadata based batch orchestration framework for modern data platforms. Implemented using Azure PaaS data services. Common ingestion and transformation patterns available out of box. Reusable code can be easily extended to cater to custom patterns.
devops
ci-cd
delta
elt
dataplatform
etl-framework
elt-framework
modern-data-engineering
metadata-driven-controls
-
Updated
Oct 9, 2024 - TSQL
This curated list brings together powerful open-source tools, frameworks, and resources for data engineering 🛠️ and data science 📈. It started with inspiration from pracdata's awesome-open-source-data-engineering list 🌟 and has grown into a resource to organize and explore the ever-evolving data landscape 🌐.
-
Updated
Oct 25, 2024
Improve this page
Add a description, image, and links to the modern-data-engineering topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the modern-data-engineering topic, visit your repo's landing page and select "manage topics."