Skip to content

Latest commit

 

History

History
50 lines (31 loc) · 1.9 KB

README.md

File metadata and controls

50 lines (31 loc) · 1.9 KB
Raw

Smogboard v1

shields shields shields

An end-to-end data engineering project using data sources from Smogon Stats and PokeAPI.

[UPDATE] Dashboard will no longer display updated stats after Jan 2024, I ran out of credits and was charged $100 by Google :(

🧑‍🔧 Data Pipeline

  • Mage was in charge of my ETL processes and scheduling/orchestration because people were hyping it as the next Airflow.
  • BigQuery handled my Data Warehousing needs and it was effortless connecting it to Looker Studio as a live source.
  • Yes it's nothing fancy, but hey - it works.

pipeline

🌟 Data Model

  • I used Kimball's approach for data warehousing (transaction-based) and created the star schema below.
  • Again, nothing fancy because I didn't want to overcomplicate this project.

model

🚀 Demo

  • The final result (yes it needs some work):

dashboard

Click here to interact with it!

💻 Built With

Technologies used in the project:

  • Python
  • Mage
  • Google Cloud Platform
  • BigQuery
  • Looker Studio

👀 What's Next

  • Including statistics for other generations and battle formats
  • Improve on the dashboard's design and naming consistency of fields
  • Enhance dashboard visuals with sprites and type-based coloring
  • Add staging via Cloud Storage

🛡️ License

This project is licensed under the MIT License.